Next Tuesday, November 17th, Stewart will deliver a brief lecture at the Experimental Media and Performing Arts Center (EMPAC) in Troy, New York. His topic will be the Terre Natale (Exits Two) exhibition piece created by Diller Scofidio + Renfro, et al. His lecture is just one part of an evening of lectures under the banner Full Immersion: Art and the 360-degree screen, lead off by artist and scientist Bernd Lintermann of the Institute for Visual Media at ZKM, Karlsruhe. Be sure to see Eric Ameres, highly accomplished video engineer, and Thomas Soetens and Kora Van Den Bulcke along with their piece They Watch. The evening begins at 7pm in Studio 2.

Description from the Full Immersion page: An exceptional panel of international artists, engineers, and producers representing the evolving field of works created for the 360° panoramic screen. The panelists in Full Immersion – Stewart Smith, Bernd Linterman, Eric Ameres, and Thomas Soetens and Kora Van Den Bulcke of Workspace Unlimited – come from the visual arts, film, and data presentation. Some specialize in grand gestures, some write code. What they have in common is expertise in realizing works that surround the viewer, with effects that may be subtle, spectacular, or unsettling. In addition to talking about their field, they’ll also be showing up-to-the-minute examples of what’s been accomplished in it. The result will boggle not just the mind, but the eye and ear.

Bronson is a light-weight data visualization framework written in Processing by Stewart Smith and Robert Gerard Pietrusko specifically for the Terre Natale exhibition in Paris (November 2008). It was named in honor of actor Charles Bronson for his no-nonsense, brute-force approach to problem solving.

Bronson is capable of generating lengthy movies by dynamically loading and unloading complex animation instructions. It was built from scratch to satisfy some abnormal project specifications: a movie resolution of 6,800 × 768 pixels running at 30 frames-per-second, divided across six projectors onto a curved, panoramic target surface. (That’s over 150 million pixels per second.) Bronson outputs large, lossless movies that are then reformatted by a projection-curving and edge-blending system developed by ZKM.

Bronson is currently receiving some under-the-hood adjustments in anticipation of Terre Natale‘s travel to Copenhagen this December. More information to follow.

What frameworks are out there?

C++. Most of what you’ll find online is based on the ARToolKit (Augmented Reality Toolkit) written in C++. It’s important to note that the ARToolKit was released under the GNU GPL but commercial licenses are also available. There’s also the NyARToolkit in C++ which boasts ports to Java, Android, C#, and ActionScript3. I don’t speak Japanese, however, so I left this road largely untraveled. I don’t actually speak much C++ either, but if you’re an advanced programmer, ARToolKitPlus is probably for you. And they’ve done some impressive marker tracking too.

Java / Processing. Java implementations typically use the JMyron library for vision processing which only seems to work for PCs. Kevin Cox of Jibberia has compiled a Mactel JMyron version but this didn’t work for me either; particularly frustrating because this rules out developing in Processing. (If someone on a Mactel has better luck, let me know.) If you’re on a Windows box then check out Bryan Chung’s Simple ARToolKit Library for Processing. JMyron is also available for Max MSP and Adobe Director but I’ve yet to try them.

Flash / ActionScript3. Saqoosha offers the definitive Flash implementation called FLARToolkit which is built from the NyARToolkit’s Java implementation and uses the Papervision3D library. His FlarToolkit Start-up Guide is a handy resource. Eric Socolofsky of Transmote released FLARManager this past April which simplifies building on top of the FLARToolkit. If you’re a Flex Builder user the BasicFLARToolKit by John Lindquist at PV3D.org has some useful examples and source code. For more specific questions or concerns check the FLARToolkit Google User Group.

But I want it NOW!

The quickest road to AR is through the SquidderKit. It’s a Flash implementation that includes the FLARToolkit, Papervision3D, and FLARManager all in one package that can compile in Flash, no Flex Builder required. It handles multiple markers and includes a PDF of them for you to print and test with. A compiled SWF is included so you can give it a spin right away before even looking at the code. Generally I’m not a fan of Flash, but it enables you can crank out a demo and rapidly publish it to the masses (that also have web cams and printers) all through a web browser. After all, the strength of Augmented Reality is in experiencing it. Canned demo videos just feel like bland After Effects magic.

Custom Markers

Again, Squidder brings the goods with a blog entry titled For Nerds Only: Custom FLAR Markers Explained from March 2009. The article points to Tarotaro’s ARToolKit Marker Generator which you can download or use online. It requires Flash Player 10, but will partially work in Flash Player 9 while failing silently, truly frustrating! I’ve had trouble running the SWF locally (unclear why), but it runs fine online. And don’t worry, the “.pat” extension is just convention. You can name the file however you’d like. Alternatively there’s Saqoosha’s MarkerGenerator, an Adobe AIR application, but Tarotaro’s app gives you more control.

Calibration, Lighting, Tracking

These are the topics I’m just beginning to investigate. If you’ve been playing with AR code already you’ve probably wondered about the camera calibration files. Their sole purpose is to correct lens distortion so markers can be read correctly in physical space and positioned appropriately in virtual space. If you’re interested, you can do your own camera calibrations.

Right now I’m wrestling with variations in lighting and how dimly lit spaces can absolutely kill a demo. (Consider yourself warned!) The two things I’m looking into are Makc’s AdaptiveToBinFilter (AS3) for lighting issues and Justin Windle’s article on Better Motion Tracking (AS2/3).

Terre Natale has recently been featured in the International Herald Tribune, France’s Le Monde, and Esquire Magazine’s “Best and Brightest” issue with the article Four Innovative Mapmakers Re-inventing the Very Idea of Maps. You may also visit the Terre Natale project on Stewdio.

Congratulations to Ben Rubin and his Ear Studio for their work on Exits I and much gratitude for their contributions to Exits II. A very special thank-you to Bernd Lintermann for contributing his panoramic projection system and incredibly sharp eyes. And finally, we are in great debt to our Cartier counterparts Francois Gemenne, Ilana Shamoon, and Marta Niedzwiecka who assisted us above and beyond the call of duty to ensure Exits II told a cohesive and well-translated story.

See Stewdio’s Terre Natale project page at http://stewdio.org/work/terrenatale.