Epic’s graphics guru Tim Sweeney shares his vision for perfect augmented reality

Disclosure: The organizers of ChinaJoy paid my way to Shanghai. Our coverage remains objective.

SHANGHAI — Tim Sweeney, chief executive of Epic Games, is one of the rare seers who understands both technology and games. He’s been pioneering graphics for 25 years, and his company of game designers has brought us titles such as Unreal Tournament and Gears of War.

Epic also creates the Unreal Engine 4, a tool that makes it easier for other game developers to create beautiful games with amazing worlds and realistic human characters. Epic’s Unreal is the engine of the game industry, and Sweeney is the one of the brains that makes it happen.

That’s why he’s one of the most interesting seers in games. He gave a talk at this year’s ChinaJoy 2015 game trade show, an event attended by an estimated 250,000 people in Shanghai. At first, as I sat in the room in the Kerry Hotel, there were just nine people in the audience. Sweeney, of all people, had technical difficulties. And by the time he was ready to talk, the room had magically become full of listeners.

Tuning in via live translation, they were a lucky bunch. Sweeney talked about his predictions for mobile game graphics, better web games, virtual reality, the game engine wars, and augmented reality. By his calculation, computers have gotten 100,000 times faster while he’s been making games. But augmented reality is going to be the best revolution yet. And he figures that if augmented reality is perfected, we’ll have the equivalent of 40-feet displays on our eyeballs. That will make TVs, tablet displays, and smartphone displays of all kinds obsolete.

Afterward, in Epic’s booth in one of ten giant halls, I interviewed Sweeney about his predictions. Here’s an edited transcript of our talk.

Above: Epic Games touted Unreal Engine 4 at a big booth at ChinaJoy 2015.

Image Credit: Dean Takahashi

GamesBeat: It was an interesting talk. That was a big thought again, about AR possibly eliminating all kinds of screens. It’s not good for the glassmakers of the world.

Sweeney: If you think about it, it’s going to be a real boon. Everyone will have an awesome television that’s 40 feet wide and you can display it anywhere in your house. The amount of material needed to manufacture AR will be much smaller than TVs or computers or any of these other devices. It’ll be very economical.

GamesBeat: You suggested that the quality is getting to a point where screens are less and less something we need to look at. The images on the glasses can be as good as real life.

Sweeney: We’re definitely on track to that. There have been a lot of studies on exactly what is the resolution of the human visual system. It’s fairly clear that it’s less than 8000 pixels by 8000 pixels. That’s the maximum quality of graphics we can appreciate. You could go to a display like that right now if you were sufficiently motivated.

Chipmaking has been driven by Moore’s Law. They’re building billion-transistor chips. But there hasn’t been as much drive behind displays until recent times, until the Retina display revolution. As that technology is pushed harder, we’re just a couple of generations away from displays that are 8K by 8K per eye times two eyes. There you have a display that’s indistinguishable from reality.

The real interesting things with VR and AR start to happen when you have custom-manufactured display devices aimed just at those applications. Right now everybody’s devices are reusing mobile phone displays, augmenting them with lenses and other systems to purpose them for VR. Once you have custom display systems, the level of realism is going to go up very quickly.

GamesBeat: After 10 years, say we don’t need screens anymore. What are some of the steps along the path to making that happen?

Sweeney: The immediate step for content creators is VR. VR is manufacturable. We can build devices with high quality and a very large field of view. The augmented reality content revolution starts with VR, and that’s probably the home of it for the next three or four years. Then we can expect to have very high quality AR devices coming online.

The industry is going to have to go through a period of reinvention. Every application designed for 2D interfaces will be obsoleted by AR. You’ll be able to achieve far better usability if you design everything for the AR experience using AR input — motion and gestures and other very fine input sources. Think of every type of application you use in daily life and the types of applications used by professionals. You can start to imagine how they might work in the future.

Think of modeling, for example, which is done in 2D with programs like 3D Studio Max and Maya, using very complicated user interfaces with very complicated mouse commands. Instead, you’ll be able to reach out and sculpt your object directly, paint on it with your paintbrush, and zoom in and out in a very natural way.

GamesBeat: I’ve heard people talk about trying to do glasses with VR on one side and AR on the other. They become see-through at some point. Is that another step in the middle that somebody could achieve?

Sweeney: Perfection is a long way off, just because of the difficulty of building these things. We’re going to see a lot of progress and a lot of interesting intermediate steps. Right now with VR we have HTC, Oculus, Sony’s Morpheus, and Samsung with Gear VR, which is probably the most interesting idea, reusing an existing cell phone with some lens hardware to give a VR experience where you don’t have to buy a computing device. That’ll combine with the research into HoloLens and MagicLeap and a lot of other creative ideas.

It’s going to be like any other period of revolution. We’ll see a lot of experimentation. A lot of people will try ideas. Some of them will work and some won’t, but everyone will learn rapidly. In 10 years, you’re at the destination.