Facebook wants to make social VR indistinguishable from real life

Facebook is conducting a lot of research into how to capture social interaction in virtual reality so that it will seem real.

In addition to a demo on how to do selfies in a 360-degree VR photo, Facebook showed how researchers are trying to make social interaction in VR indistinguishable from real life, so that you can meet with friends or see family members for the first time or go to concerts on virtual dates.

“Social VR done right will let us define our tribe … not based on proximity, but by personal choice,” said Yaser Sheikh, a Facebook Oculus VR researcher and associate professor at Carnegie Mellon University, during a talk at the F8 event in San Francisco. “It will change how we live in this world and with each other.”

Sheikh said his Carnegie Mellon students built a studio, Panoptic, where they conduct experiments of capturing the motion of people and their social interactions. The researchers try to combine hundreds of videos captured in the studio without the use of motion capture markers.

Sheikh said that he was inspired to do some of this work with the birth of his daughter, during a moment when he looked across at his older son holding a baby and capturing a smile. He wishes that all of his relatives, some who live in distant locations, could experience the same thing and get to know his children. VR is just getting started with the launch of Facebook’s Oculus Rift VR headset and the Samsung Gear VR powered by Oculus, but it has a long way to go before it can meet these challenges, Sheikh said.

Above: Yaser Sheikh

Image Credit: Facebook

At some point, Sheikh said, “the technology disappears and you simply interact.” He calls this “achieving social presence in VR.” and he said, “Doing lifelike virtual reactions will be a transformative achievement.”

You’ll be able to play poker with friends as if you were in the same room, or do a job interview, meet your relatives, go to a concert with friends.

“Proximity would no longer determine who you spend your time with,” he said.

Above: A captured avatar of Yaser Sheikh’s head.

Image Credit: Facebook

One of the tasks ahead is to make realistic avatars. As we know from video games, developers can create extremely realistic faces now. But it takes an awful lot of skill and post-production techniques to make those faces so real. With VR, you want to be able to do this on the fly, in real-time. It has to be deeply convincing and genuine, Sheikh said.

Those avatars have to be able to capture subtle motions in a unobtrusive way. It won’t work with a ton of expensive equipment on your head. It has to be cheap, instantaneous, and accurate. You have to be able to play a game like Mafia (also known as Werewolf), where a group of people have to guess who is a member of the Mafia among them, based on their body language under questioning.

Facebook will also have to figure out how to predict social interactions and get avatars to anticipate motions that are about to happen.

Right now, Facebook can distinguish head movements and hand gestures with the Rift and its Oculus Touch controls. But it has a long way to go to capturing more than that.

“This is nowhere near solved,” Sheikh said.

Above: Facebook is working on capturing lifelike motions.

Image Credit: Facebook