Pantomime lets VR headset and smartphone users share a virtual space

A startup called Pantomime came up with a nifty system of enabling people to peer into animated virtual worlds without the need to wear virtual reality headsets. And now it is adding support for VR headsets so that multiple people can be in the same VR space more easily.

The new technology is very innovative when it comes to making VR more social, and it’s one more offering that could help fulfill the promise of virtual reality and augmented reality, which are expected to become a $120 billion market by 2020, according to tech advisor Digi-Capital.

As we wrote in December, David Levitt, the CEO of Pantomime, and his team have created a new twist on the virtual world with a new app dubbed Pantomime Playground. It lets you use a Mac computer or an iOS device as a kind of window into the VR space. The first game to debut on top of Pantomime Playground is Pantomime Bug Farm, which lets you use your iPad to maneuver around a table with bugs on it. You can use the iPad to view the bugs from different angles and even squish them. And now the company says it will support a variety of headsets in its shared experience as well.

“Today Pantomime software is turning virtual reality from a passive solo experience into an interactive, social one,” said Levitt, in a statement. “We’re introducing two advances: reaching into worlds interactively using mobile devices while wearing a VR headset, and playing with other users over wireless networks — all with no new hardware. And since Pantomime works across many devices and makes headsets optional, it lets people who don’t have a VR headset play together in virtual worlds with people who do — providing an inexpensive consumer on-ramp to VR.”

Pantomime’s software detects the way you are holding a tablet and figures out which point of view you should see through the tablet’s screen. If you turn the tablet in one direction, you’ll see a different view.

The tech makes use of motion-sensing hardware on the devices. You can cradle, grip, or swing the device in your hand, and you’ll see your own motion have an effect in the virtual world. You can, for instance, push your tablet flat on the table to squish ants, praying mantises, spiders, or scorpions in the Pantomime Bug Farm game. You can grab a tablet screen from any edge of the tablet and swing it back and forth. The platform can sense the presence of the swinging tablet like a motion-sensing Wii controller.

With today’s announcement being made at the Vision VR Summit in Los Angeles, you can now wear a VR headset to see into the same virtual space that your friends are viewing on a Mac or other device.

Levitt studied artificial intelligence at MIT under A.I. innovator Marvin Minsky. In addition to his work with Minsky, Levitt also spent time at VPL Research, one of the first VR companies, which Jaron Lanier founded in 1984. That first go-round for virtual reality failed, but Levitt reawakened to the possibilities after Oculus VR started grabbing the limelight with a revival of the medium. He started his company before Facebook bought Oculus for $2 billion.

In most virtual-reality and augmented-reality systems, you must wear goggles with a stereoscopic display over your eyes, so you can immerse yourself in a virtual world. But Pantomime’s technology turns this inside out. Ordinary tablets, smartphones, and computers become the portal into a VR space, as if their screens are made of glass. Your device is a part of the virtual scene, and you can use it as a paddle, swinging it back and forth. You can use a Mac screen with your iPad to view your tablet’s position in the virtual world so long as both devices are on the same Wi-Fi network.

Levitt formed his new Sebastopol, California-based company in 2014 with Don Hopkins, who formerly worked on The Sims. Eric Hedman, another game creator who worked on The Sims expansion packs, subsequently joined them. Advisors include Arthur van Hoff of cinematic-virtual-reality leader Jaunt and video game visionary Nolan Bushnell. Pantomime has filed for patents — and has been awarded one — on its technology, and it is working with game and entertainment companies to build apps on top of it.

The possibilities are tantalizing. Levitt showed a video of how you can have three Macs and an iPad interacting with each other in the same virtual world. I used it in person, and it was a pretty magical experience. I was swinging an iPad back and forth, using it to squish bugs on the table. As I turned the iPad and pointed it in different directions, I could see different parts of the 3D scene.

The software, now available in the App Store and Mac App Store as Pantomime Bug Farm, lets people reach in and play with virtual creatures — evading, shoving, and squashing them. You can fling the bugs across the table, toss realistic objects like ping-pong balls at them, feed them, and grow them to enormous size. Multiple devices can access and share the 3D scene in Bug Farm. Players of any age can interact with Bug Farm, even four-year-olds and five-year-olds, Levitt said.

Bug Farm is just one scene that will be available in Pantomime Playground. Eventually, individuals will be able to create their own original scenes. The app is free at the moment.

Levitt said that Pantomime resolves some key problems with VR. It can reduce the cost because people can use devices they already have. They can also use it on the run and network with other players. Plus, they avoid the potential isolation or nausea that goggles can cause. It’s not a “nerds only” solution, Levitt said.

Today at the AR/VR Vision Summit in Hollywood, the company said it will demonstrate multiple users of the Samsung Gear VR headset playing in shared virtual worlds and reaching in with ordinary consumer smartphones and tablets. A half-dozen virtual worlds showed how users can push and paddle balls and other realistic objects, knock over dominos, play scoring games, dance together with expressive body language, and squash virtual bugs against their real tables, while seeing virtual versions of the devices they’re really holding – held in their animated virtual hands.

Above: Pantomime lets users with a Samsung Gear VR share a virtual space with users on smartphones, tablets, or PCs.

Image Credit: Pantomime

In Pantomime’s new demonstrations and videos, two, three and four headset users will be able to see each others’ virtual heads and hands, pitch and paddle balls to one another, play competitive games, and see themselves and others in virtual mirrors. Users with only a smart phone or tablet can see and be seen by other players as reaching into the shared virtual worlds.

Pantomime Playground for Mobile VR is now available to beta testers using the Samsung Gear VR. Versions will be available later this quarter for mobile headsets compatible with Google Cardboard software. Pantomime Playground apps for Android and Windows are likewise available to beta testers now.

“With Pantomime, anyone with a tablet, smart phone, computer, OR a VR headset can reach in and play with other people in virtual worlds,” Levitt said.

Above: Pantomime lets multiple people share a virtual space and, uh, kiss. Sort of.

Image Credit: Pantomime