The DeanBeat: Epic graphics guru Tim Sweeney foretells how we can create the open Metaverse

Tim Sweeney stitched together an interesting vision of the future in a talk this week at the VRX virtual reality conference in San Francisco. He wants to build the Metaverse, the virtual world envisioned by Neal Stephenson in the novel Snow Crash in 1992. That novel’s vision of a pervasive cyberspace where we live, work, and play has inspired many a startup including Second Life maker Linden Lab and Stephenson’s current employer, augmented reality glasses maker Magic Leap.

Above: Snow Crash

Image Credit: UploadVR

Many people have tried and failed to build the Metaverse, and I came across two new companies in the past week that are invoking the name in their own pitches. But Sweeney is unique, in that he has been a graphics wizard for the past couple of decades and he enjoys both predicting technology’s path and thinking about its ramifications.

I interviewed him about his speech after he gave it. I also bounced Sweeney’s predictions off another speaker, Tim Leland, vice president of technology management at mobile technology manufacturer Qualcomm. Sweeney told me he forgot about the Metaverse, particularly as early virtual worlds didn’t take off, but he has given it serious thought lately as VR came back.

Sweeney predicted that perfect augmented reality glasses would make TVs obsolete, as the smartglasses will deliver the equivalent of a 40-feet screen in front of your eyes. He also rattled Microsoft’s cage in calling for the platform owner to make Windows more open.

Those past comments were visible in Sweeney’s latest prognostications, and the newest predictions were wonderfully lucid. I suspect he is right in that it will take all of us, not just a single company, to make this happen.

Sweeney started close to home, nothing that the company spent tens of thousands of dollars earlier this year to create a real-time demo where a voice actress moved and spoke at the same time as a 3D-animated avatar named Senua from the upcoming Ninja Theory game Hellblade. Working with Cubic Motion, 3Lateral, and others, Epic was able to create an extremely realistic demo that showed Senua moving and speaking at the same time the voice actress did.

This kind of realistic avatar is possible because of outstanding performance of 3D graphics chips as well as realistic simulations that bring to live digital characters. Graphics technology has gotten 100,000 times better in the past twenty years.

“We are not more than several years away from having a human that is not that much different from reality,” Sweeney said. “This is going to change more over the next two years than it has over the past 20 years.”

Those digital humans were an essential part of Stephenson’s Metaverse, which is the cyberspace equivalent of our own world. But Sweeney also believes that the social part of the Metaverse is important too. He sees the beginning of this shared social environment in Oculus’ virtual reality demo of the Toy Box, where you can use avatars and touch controls to play with toys along with someone else in the same virtual reality room.

“The Oculus toy box is the beginning of a social experience,” Sweeney said. “That’s the first step. The next step is to have outward facing cameras and inward-facing cameras that pick up the movements of your face.”

When you can capture facial expressions accurately, communicating in the virtual world becomes as good as it can be in the real world. And while game developers have had a nasty time duplicating humans in video games over decades, Sweeney noted that the task at hand for the Metaverse is a lot easier because real humans will provide the realistic motions and expressions that the avatars in the Metaverse will mimic. You simply have to capture real people, not try to completely reproduce them and simulate them, Sweeney said in our interview. Sensors are delivering this to us today.

Right now, motion capture technologies are quite expensive, as are tethered virtual reality headsets. Sweeney envisions the hardware required for VR will eventually be reduced to a pair of Oakley-style sunglasses, with 8K resolution graphics per eye. You need about 640 million pixels for something like this, Sweeney calculated, and that should be doable, given that Intel can already put billions of transistors on a chip. You’ll take it wherever you go.

“The device will be cheaper than a PC because it weighs far less and uses less material,” Sweeney said.

Sweeney thinks this may take five to seven years, particularly as tech companies develop new displays that are made just for VR and AR, rather than using the displays that were originally developed for phones.

Above: Tim Leland, vice president of technology management at Qualcomm.

Image Credit: Dean Takahashi

Tim Leland at Qualcomm said it will be challenging to get this task done, but he agreed that purpose-built displays for VR and AR will be more efficient than borrowing the tech designed for smartphones. He thinks the ultimate consumer device will combine AR and VR in a single pair of glasses, untethered and mobile, with outstanding 3D graphics and power efficiency.

“Those resolutions will be there,” Leland said, regarding the 8K spectacles. “Some of the brute force methods in the console and PC space will need to change to meet the requirements for these thin and light devices if you want them to run for hours at a time. Some of the nice visuals where you spend an extra four or five watts to get nice hair color, that’s probably not the right decision.”

The “human implications of this will be profound,” Sweeney said. “We can put a headset on and communicate with anybody in the world in real-time and it will be like being the same room with them.”

It’s going to take digital human technology, virtual reality and virtual worlds, presence (the feeling that you are there), and a Facebook-like ubiquitous social network.

This Metaverse ideas has been growing in science fiction since the 1960s, and it’s only partly about gaming. It will be the place where engage in e-commerce and it will affect all aspects of life. We’ll move from the giant TV screens and clunky mice and keyboards to systems that are incredibly realistic, he said.

We’ll have all the ills of the real world in the Metaverse, like porn, crime, harassment and other things we already encounter in the Internet today.

Above: Ping pong in VR

Image Credit: Oculus

“This virtual economy might exceed the real economy some day, and it could be a utopia or dystopia when we think about it,” Sweeney said. “It’s an economy that is very different from the one we have today.”

Artificial intelligence could take away millions of jobs. But living in a virtual economy, which creates jobs such as esports careers and virtual consumption, could also provide millions of jobs. And it could be more sustainable, as it doesn’t consume as much resources as jobs in the physical world. Think Second Life or Minecraft, where you can make your own career creating digital goods. In the real world, you might have a simple home. But in the virtual world, you could live in a giant oasis built from virtual currency and giving you all of the status and riches that you lust after.

“You don’t need iron mines and oil wells to make virtual goods,” Sweeney said.

But that virtual world has to be open, and it has to evolve on open standards as the Internet has, Sweeney argued. You don’t want a closed system like Facebook, iPhone, Android, or Twitter, as they lock you into their closed systems. They monitor what you are doing and they will take a cut of your every transaction and sell your private information to the highest bidder. People were tricked into joining these closed systems the first time around, but the next time, we need something open, Sweeney said.

Above: Second Life

Image Credit: Linden Lab

“This Metaverse is going to be far more pervasive and powerful than anything else,” Sweeney said. “If one central company gains control of this, they will become more powerful than any government and be a god on Earth. What we want is not a company but a protocol, that anyone can implement.”

There are ways to make that happen. Companies can realize that they can’t provide every part of the Metaverse themselves, and that they can make a lot more money if they band together to build it.

We can use blockchain as the foundation for e-commerce, for both security and privacy. We could tap something dubbed an “interplanetary file system” for unlimited storage, as it will be distributed via the BitTorrent protocol across all computers and use public key cryptography to keep it private, Sweeney said. We’ll have to define the Metaverse equivalent of a web site, a place that anyone can visit.

“Between all of these between all of these things, we have the ability to build this Metaverse with existing game engines like Crytek, Unity, and Unreal,” Sweeney said.

Sweeney concluded, “I’m excited and a little bit scared about it and looking forward to being a part of this.”

Sweeney said that you’ll be able to get exercise in the Metaverse, so we won’t all simply turn into blobs, and it will be social too, so we won’t all be completely isolated.

“We have to be patient, and we have to take many attempts to succeed,” Sweeney said.