Pantomime launches Creatures AR app with shared augmented reality

Pantomime, a maker of augmented reality technology, has unveiled its new Creatures AR app with shared augmented reality.

Sebastopol, Calif.-based Pantomime has come up with a way of viewing virtual reality imagery without wearing goggles. The new technology is part of the promise of virtual reality and augmented reality, which are expected to become a $120 billion market by 2020, according to tech advisor Digi-Capital.

The new app is available on iOS. It enables realistic first-person 3D interaction and real-time multiplayer over Wi-Fi. The company argues that its version of AR is a significant advance over the popular Pokémon Go mobile game introduced this July, which simply shows individual animated characters superimposed over live video as a targets for Poké Balls.

“Pokémon Go launched a new era where Augmented Reality isn’t about costly headsets or vision processing,” said Pantomime cofounder and CEO David Levitt, in a statement. “Now Pantomime Creatures AR lets anyone interact in 3D, turning our phones and tablets into paddles and scoops to use in augmented worlds. Since no new hardware is needed, these consumer experiences will continue to be the largest and fastest-growing augmented reality markets.”

Pantomime’s software detects the way you are holding a tablet and figures out which point of view you should see through the tablet’s screen. If you turn the tablet in a new direction, you’ll see a different view. The tech makes use of motion-sensing hardware on the devices. You can cradle, grip, or swing the device in your hand, and you’ll see your own motion have an effect in the virtual world.

Pantomime Creatures AR provides real-time 3D, with first-person graphics and navigation that puts users in vivid 3D worlds. It has realistic physics, where users interact with multiple 3D creatures simultaneously as they toss them food, scoop up and pour creatures and objects from virtual cups, and knock them into a creature collection basket with virtual snowballs.

Above: Pantomime’s software lets you interact with a virtual world.

Image Credit: Pantomime

The Creatures AR app also has synchronous multiplayer, where users on the same Wi-Fi network can play together, seeing each other’s 3D movements in a shared world in real time.

Creatures AR lets you use a tablet or phone to view a 3D-animated world and the virtual objects in it. An object that you hold in your hand in front of a camera can show up in the virtual world, too. Your animated hands can also show up in the virtual world, as viewed through a screen. Devices can contact and collide with virtual objects, while their screens become mobile views into the virtual scene. Users can tip, turn, tumble, walk, paddle and swing their devices through the scene, stroke the screen to move and turn, and tap the screen to throw objects, in a portable experience that requires no external sensors or advance setup.

Creatures AR lets users attach a virtual Cup to their device to carry creatures and food. Users can fill a cup with food, scoot under flying creatures to catch them, scoop things up from the virtual play rink, and pour out the contents by tipping their device over as they view the scene through it. Thus real devices, interactive objects, and animated characters are seamlessly combined in physically realistic 3D worlds. I’ve seen some of these in a past demo.

Last year, Pantomime launched a new twist on the virtual world with an app it has dubbed Pantomime Playground. The first game to debut on top of Pantomime Playground was Pantomime Bug Farm, which lets you use your iPad to maneuver around a table with bugs on it. You can use the iPad to view the bugs from different angles and can even squish them with it.

“We call it virtual worlds for the rest of us,” said Levitt, in an interview with GamesBeat last year. “You can reach into a virtual world using the tablets, screens, and computers we already have.”

Levitt studied artificial intelligence at MIT under A.I. innovator Marvin Minsky. In addition to his work with Minsky, Levitt also spent time at VPL Research, one of the first VR companies, which Jaron Lanier founded in 1984. That first go-round for virtual reality failed, but Levitt reawakened to the possibilities after Oculus VR started grabbing the limelight with a revival of the medium. He started his company before Facebook bought Oculus for $2 billion.

In most virtual-reality and augmented-reality systems, you must wear goggles with a stereoscopic display over your eyes, so you can immerse yourself in a virtual world. But Pantomime’s technology turns this inside out. Ordinary tablets, smartphones, and computers become the portal into a VR space, as if their screens are made of glass. Your device is a part of the virtual scene, and you can use it as a paddle, swinging it back and forth. You can use a Mac screen with your iPad to view your tablet’s position in the virtual world, so long as both devices are on the same Wi-Fi network.

Levitt formed his new Sebastopol, California-based company in 2014 with Don Hopkins, who formerly worked on The Sims. Eric Hedman, another game creator who worked on The Sims expansion packs, subsequently joined them. Advisors include Arthur van Hoff of cinematic-virtual-reality leader Jaunt and video game visionary Nolan Bushnell. Pantomime has filed for patents — and has been awarded one — on its technology, and it is working with game and entertainment companies to build apps on top of it.