Augmented reality: the next generation

Mobile augmented reality (AR) was a favorite of the tech press in 2009, a year that saw the unveiling of mobile AR browsers from Wikitude, Layar and Metaio. Now that the dust has settled, let’s take a look at the current state of the technology and where it will go from here.

Mobile AR browsers currently work by using a combination of the mobile phone’s camera, compass and GPS data to identify the user’s location and field of view, retrieve data based on the geographical coordinates, and overlay that data over the camera view. This first generation of mobile AR technology had a number of drawbacks which the AR companies are currently tackling.

Accuracy problems

The GPS on a mobile phone normally only gives a position within around 20 meters, while the iPhone’s compass orientation is only accurate to around 20 degrees. This can lead to problems in determining exactly what’s within the camera’s view. Real and augmented objects may also be poorly aligned with each other, so the virtual AR objects end up “floating” in the view rather than being solidly anchored to real objects. This becomes a big problem if, for example, two restaurants with different review ratings sit beside each other. AR company Metaio uses optical tracking mechanisms to stabilize the relative accuracy of the viewpoint and reduce the “floating” effect.

In the longer term, the company is working on markerless tracking to improve accuracy in identifying objects. This works by creating a “signature” from a photograph of a building or other landscape object in various lighting and weather conditions and mapping the signature to the object’s geographical coordinates. The object can then be identified up to an order of magnitude more accurately than when using GPS and compass data alone.

Metaio is partnering with Earthmine, which provides street-level images in which each pixel is mapped to the corresponding 3D coordinate — latitude, longitude and elevation. Metaio’s markerless tracking implementation is currently in the prototype phase but should be available in products by the end of 2010.

Line of sight

The next problem is line of sight. The GPS position doesn’t tell the AR system where nearby objects are in relation to each other. You may see augmented reality information about a building that is close to your position but obscured by another building. Both Layar and Metaio have been working on this problem. Layar did a demonstration at last year’s PICNIC conference that used “invisible walls” to simulate a real-life object in the line of sight and blank out augmented objects behind it. Metaio is also working on the related problems of scale and depth perception. Virtual objects that are far away should be smaller than those closer to the user, just like in the real world.

As more information becomes available, the AR view can quickly become cluttered. Metaio is working on techniques like clustering objects so that individual objects are only visible when a user moves closer to them. However, Mobile AR products do not yet use any standard data formats for representing points of interest and other AR data, so different browsers will organize data in different ways. AR products are also “walled gardens”; unlike the online world, where different browsers will show you the same web page, you can’t by default view the same information with a Layar, Wikitude or Metaio browser. For mobile AR to really go mainstream, standard data formats and interoperability will have to be established.

According to Claire Boonstra, one of the co-founders of Layar, the next step for mobile AR is moving from “functional AR to experience AR”. Most current users are AR enthusiasts and other early adopters. The general public seems to regard AR with more trepidation. The mobile AR companies therefore need to convert the initial wow factor into an engaging user experience for a larger audience.

Metaio CTO Peter Meier agrees that the user experience needs to improve and more emphasis must be placed on design as well as technology. Layar is banking on features like POI to POI (linking points of interest together), which would allow multiple locations to be linked together in an experience like the Beatles AR discovery tour shown above. Metaio is putting a lot of work into into social features in an effort to create more engagement.

What’s next?

So what’s next for mobile AR? Both Layar and Metaio were reluctant to tell me what’s coming up in the next versions of their products but would talk about general trends. Peter Meier of Metaio said there has been increasing demand to do recognition tasks like those performed by Google Googles. Googles is an Android application that takes a photograph of an object like a book or landmark. It then uses image recognition to identify the object and returns information about it to the user. AR on mobile phones is mainly applied at street level right now, but it could be very interesting to also see AR data for smaller objects such as retail items. Systems using markers (a kind of printed tag) like Metaio’s Lego Box project have been around for a while, but Googles could replace markers with image recognition.

Claire Boonstra of Layar says that AR interfaces in glasses are the next frontier. Video eyewear company Vuzix is bringing out its Wrap 920AR glasses in Q2 of this year. As as well as allowing you to view video in the glasses like other products in the Vuzix range, the glasses have two embedded cameras, a 6-Degree of freedom tracker (which identifies the field of view), and a compass.

Combining an AR view from these glasses with a mobile phone to retrieve the data could be a dream scenario for mobile AR, but there is one technological barrier — mobile phone support for external displays. Apple iPhones and iPods, for example, block the mobile device from sending video display to an external monitor like the Vuzix Wrap eyewear. In order for developers to use an external display, they have to jail-break the iPhone, which is obviously not practical in a commercial product. Incidentally, Vuzix also recently started an AR training division, which services industries like aviation, medicine and the military.

Finally, Metaio’s plans for 2010 include creating the world’s first “AR city”. This is an attempt to move AR into the mainstream by blanketing a specific geographic location in the US with a high-density of AR data. Such an environment could be a giant test lab for the fledgling AR industry. Watch this space.