Reality made larger than life

Reality made larger than life
By Alan Cane

Published: January 19 2010 16:41 | Last updated: January 19 2010 16:41

Audiences gasp at what they see: a presenter stands in front of them with nothing in his hands. Yet the large screens on either side of the stage show him holding a flower.

As he waves his clenched fist around, it is as if the flower were really there: it moves in perfect time. Then suddenly it becomes a light sabre; later, the presenter holds a model car and a helicopter, which – on the large screens – appears to fly around the lecture theatre.

A growing number of such awe-inspiring demonstrations of what is labelled “augmented reality” are appearing on the internet and businesses are being encouraged to consider the potential uses of this seamless interaction of the real and the virtual.

Advertising, product design simulation and visualisation, architect’s modelling and – importantly in today’s market – entertainment and sophisticated computer games are among the areas expected to find uses for augmented reality.

In fact, there are already some well-known examples. In televised sport, for instance, advertisers’ logos and advertisements appear on football and cricket pitches where no such logos exist in the physical world; they have been written on the pitch virtually.

Similarly, viewers can estimate how far a long jumper has progressed by comparing lines drawn digitally in the sand indicating the best jump so far, the world record and so on.

Outside sport, environmentalists can hold a pattern on a piece of paper in front of a videocamera and, on a screen, see it transformed into a three dimensional model of an electricity grid, for example.

A number of companies have already launched AR systems. Layar, for example, has combined the Global Positioning System (GPS) with a camera and a digital compass to create a system that enables users to identify their surroundings, extract information about the locality and combine it with their real-world view on the mobile device’s screen.

Metaio, a German group, has developed a product to help engineers service mechanical systems by creating a digital image that overlays the physical work in front of them through a head-mounted display. Such systems can act as real-time manuals, for example, displaying every move a mechanic needs to make to repair an engine as they work.

Other applications have yet to find public acceptance and some could prove controversial.

Daphna Steinmetz, chief innovation officer for Comverse, the telecommunications software group, describes research in her laboratory which could see an end to business cards: people would merely point their mobile phone at an individual to enable face recognition software to identify them and bring to the phone everything known about them on the internet.

“The user, with one click, will be able to see the tweets of this person or view their profile in Facebook. We would also add the ability to generate a message, create a phone call or voice message or add to an address book so that immediate communication can be created.”

The question remains whether people will be happy to have their lives exposed in this way to anyone with a mobile phone. Comverse is aware of this as it works to bring the application to market.

In essence, augmented reality is a kind of digital trompe l’oeil which overlays facts and figures from the internet and other sources on to the real world to create a combined image rich in functional benefits.

But in spite of some spectacular demonstrations, in practice, much AR is in a nascent form, awaiting technological advances to make commercial progress.

Industry watchers and investors alike, however, are excited by its potential.

AR is an old idea: the concept was first broached by a cinematographer, Morton Heilig, in 1957 and the term coined by Tom Caudell at Boeing in 1992.

It comprises two broad areas – “object level” AR, where physical objects are augmented by additional data or graphics (this would include computer screen-based AR) and “location level” AR where the users’ view of their surroundings is enriched through additional information.

The widespread development and acceptance of both, however, was hindered by a lack of appropriate technology. Kelly Dempski, director of research at Accenture’s Sophia Antipolis laboratories, points out that displays and tracking technology had been poor.

But then something happened: “The average consumer now has a piece of technology – a phone with a good screen and rich graphics, a camera and a variety of tracking mechanisms ranging from GPS to compasses to onboard image recognition.

“Suddenly everyone has an AR platform in their pocket and businesses are just beginning to find new uses for this platform.”

So the mobile phone was, it seems, the silver bullet.

The re-emergence of AR, however, depended on the maturation of a number of technologies: image sensors, video cameras, and displays – either head-mounted, handheld, fixed or spatial (the last involving the projection of digital information on to physical objects).

It also depended on an array of sensory devices – accelerometers, digital compasses, GPS sensors, wireless sensors and gyroscopes – and, crucially, communications networks and key databases. For example, the Apple iPhone 3GS, complete with compass and accelerometers, is very much the model for mobile AR.

According to Ken Blakeslee, an independent consultant who has worked with a number of AR companies, the tipping point has been reached in technological development: “We are in the very early stages of development in AR, but we can move rapidly. The key thing is that the databases exist.”

He sees potential business applications in retail, vehicle repair, safety and real estate, among others.

Retail has been an early adopter of the technology. David Grunwald of Deloitte, the consultancy, says many consumer technologies are coming together to create “a basket of readiness” for AR applications, including smartphones, Flash software, 3D bar codes and the like, as well as social media: “Together they are producing a rich and fertile set of applications for retail,” he says.

Examples include an online application from Holition, a London-based software developer, whose technology allows prospective buyers of expensive goods such as jewellery and watches to try them – even if they don’t exist. An image of the customer and the item are united on screen in a montage. It enables customers to try on items at home or in-store.

Lynne Murray, the group’s head of design, says it is working now to include clothing in the application.

Significant hurdles have yet to be overcome, however. Robin Gear, manager of the innovation unit for PA Consulting, points out that “registration” – aligning digital data with the real world view while it is moving – remains a problem, together with the interaction between the virtual image and the surrounding environment.

Rob Gonda of Sapient Interactive says this is because the processing power to superimpose digital elements on top of real time video captured on a webcam or mobile camera is not sufficient to give the illusion of a seamless image.

John Spindler of ADC, the US networking group, points to a more basic difficulty – the capacity of wireless networks to carry data in the volumes generated by AR: “You have to have the infrastructure in place from a network point of view to support these applications.”

In Mr Spindler’s view, to support AR and other data-heavy applications, US wireless carriers will have to invest heavily and create a different network topology with smaller cells.

Professor Jonathan Raper of the Information Science Department at City University London, underlines the technological questions of localisation in space and time which continue to dog development: “In my view, AR is in a waiting room, still looking for the right formula that engages the masses.”

He has no doubt, however, that most progress will be made in the immediate future in location-based AR (he is editor-in-chief of the Journal of Location Based Services) and he is generally optimistic about the future of AR technology: “The steps that AR needs now to occur are improvements in positioning integrity and positioning speed and pervasiveness of good positioning.

“The step that we can envisage making this happen is the second constellation of global positioning satellites which is expected to go live in a few years.”

Copyright The Financial Times Limited 2010. Print a single copy of this article for personal use. Contact us if you wish to print more to distribute to others.