Augmented reality (AR) is a field of computer research which deals with the combination of real-world and computer-generated data (virtual reality), where computer graphics objects are blended into real footage in real time. Presently, most AR research is concerned with the use of live video imagery which is digitally processed and “augmented” by the addition of computer-generated graphics. (More from Wikipedia…) This quick guide assumes you have a web cam, printer, and some programming experience.

What frameworks are out there?

C++. Most of what you’ll find online is based on the ARToolKit (Augmented Reality Toolkit) written in C++. It’s important to note that the ARToolKit was released under the GNU GPL but commercial licenses are also available. There’s also the NyARToolkit in C++ which boasts ports to Java, Android, C#, and ActionScript3. I don’t speak Japanese, however, so I left this road largely untraveled. I don’t actually speak much C++ either, but if you’re an advanced programmer, ARToolKitPlus is probably for you. And they’ve done some impressive marker tracking too.

Java / Processing. Java implementations typically use the JMyron library for vision processing which only seems to work for PCs. Kevin Cox of Jibberia has compiled a Mactel JMyron version but this didn’t work for me either; particularly frustrating because this rules out developing in Processing. (If someone on a Mactel has better luck, let me know.) If you’re on a Windows box then check out Bryan Chung’s Simple ARToolKit Library for Processing. JMyron is also available for Max MSP and Adobe Director but I’ve yet to try them.

Flash / ActionScript3. Saqoosha offers the definitive Flash implementation called FLARToolkit which is built from the NyARToolkit’s Java implementation and uses the Papervision3D library. His FlarToolkit Start-up Guide is a handy resource. Eric Socolofsky of Transmote released FLARManager this past April which simplifies building on top of the FLARToolkit. If you’re a Flex Builder user the BasicFLARToolKit by John Lindquist at PV3D.org has some useful examples and source code. For more specific questions or concerns check the FLARToolkit Google User Group.

But I want it NOW!

The quickest road to AR is through the SquidderKit. It’s a Flash implementation that includes the FLARToolkit, Papervision3D, and FLARManager all in one package that can compile in Flash, no Flex Builder required. It handles multiple markers and includes a PDF of them for you to print and test with. A compiled SWF is included so you can give it a spin right away before even looking at the code. Generally I’m not a fan of Flash, but it enables you can crank out a demo and rapidly publish it to the masses (that also have web cams and printers) all through a web browser. After all, the strength of Augmented Reality is in experiencing it. Canned demo videos just feel like bland After Effects magic.

Custom Markers

Again, Squidder brings the goods with a blog entry titled For Nerds Only: Custom FLAR Markers Explained from March 2009. The article points to Tarotaro’s ARToolKit Marker Generator which you can download or use online. It requires Flash Player 10, but will partially work in Flash Player 9 while failing silently, truly frustrating! I’ve had trouble running the SWF locally (unclear why), but it runs fine online. And don’t worry, the “.pat” extension is just convention. You can name the file however you’d like. Alternatively there’s Saqoosha’s MarkerGenerator, an Adobe AIR application, but Tarotaro’s app gives you more control.

Calibration, Lighting, Tracking

These are the topics I’m just beginning to investigate. If you’ve been playing with AR code already you’ve probably wondered about the camera calibration files. Their sole purpose is to correct lens distortion so markers can be read correctly in physical space and positioned appropriately in virtual space. If you’re interested, you can do your own camera calibrations.

Right now I’m wrestling with variations in lighting and how dimly lit spaces can absolutely kill a demo. (Consider yourself warned!) The two things I’m looking into are Makc’s AdaptiveToBinFilter (AS3) for lighting issues and Justin Windle’s article on Better Motion Tracking (AS2/3).