Epic’s Tim Sweeney believes virtual reality will evolve more like console games than mobile

It seems like we just can’t get enough of Tim Sweeney, chief executive of Epic Games.

Sweeney has been a video game graphics guru for more than two decades. During that time, he has seen computing technology get 100,000 times faster. He has such a command of tech that it’s fun to pick his brain about what will happen in the future.

We had a chance to talk to him about his views in a fireside chat at our GamesBeat 2015 event. But we also caught up with him for a deeper dive into virtual reality and augmented reality at the VRX conference this week in San Francisco. Epic makes game engine tools for would-be VR developers, but it also is working on cool VR demos, such as Bullet Train, that show developers the way forward in this young and growing sector of the gaming industry, which tech adviser Digi-Capital expects to be a $150 billion industry by 2020.

Sweeney is a big fan of VR, but he has some cautionary advice for startups. And he worries about overinvestment at a time when the market is just getting started. But in the long run, Sweeney sees how VR is going to change just about everything. Here’s an edited transcript of our interview.

GamesBeat: We’ve heard a lot about the “metaverse,” in reference to the virtual world from author Neil Stephenson’s sci-fi novel Snow Crash. I’m curious about what you think this is actually going to be.

Tim Sweeney: With at least 250 companies working on VR now, a large number of them are working on realizing the Metaverse, this science-fiction idea of virtual reality and all that it leads to. It’s an interesting topic. There’s been a lot of technical writing about what’s possible. But we have to be careful.

We’ve learned a lot about the workings of the internet from social networks and massively multiplayer games. These are real world incarnations of something like the Metaverse. But they’re going to have some fairly significant differences. Like most real-world things, they’re not going to be quite so utopian. They’re going to be shaped by market forces and social forces.

We’ve seen all these attempts eventually degenerate into chat-oriented universes rather than game-oriented universes. Having rule systems and game systems in place to guide players is a big part of it.

GamesBeat: How is AR and VR gaming going to develop? Is it going to be more like the mobile gaming market or more like consoles?

Sweeney: That’s the fundamental question. Here in the Silicon Valley area, I think a lot of people are approaching this from the point of view of the mobile gaming revolution and trying to extrapolate that to VR. But VR is very different. The mobile gaming experience is fundamentally limited by this very small screen in front of you that occupies maybe 15 degrees of your field of view. The PC and console experience you have a 45-degree field of view.

VR is 120 degrees, your entire view space. The expectations of users on the platform will rise to levels we’ve never seen before. The console and high-end PC businesses have a lot more to teach us about development in this business than the mobile business. We’re going to see large teams producing some very novel, high-end experiences. They’re going to redefine the state of the art in photorealism.

GamesBeat: When we had our history of gaming here, it was pretty orderly in some ways: arcade games, PC games, console games, and you eventually got to mobile games. Virtual reality is fairly disorderly. We have VR appearing first on mobile. Samsung is shipping its Gear VR headset on November 20 for its phones. How are we going to have an expected unfolding of this technology, given that mobile is coming at the same time as the PC experience?

Sweeney: Mobile is going to introduce a lot of gamers to VR for the first time, which is a valuable thing. But everybody who gets into this market is going to start comparing experiences between the different platforms, and high-end experiences are going to dominate. There’s going to be more of a race to the top, to deliver the best, highest-powered gaming experiences.

That can be done on a mobile platform, but you’re going to need very powerful devices. You’re going to need advances in technology. You’ll have a lot of horses vying to deliver experiences are competitive.

GamesBeat: Do you think that’s going to drive a high-quality experience on mobile?

Sweeney: Absolutely. We’ve only scratched the surface of what’s possible there. The limitations of the smaller screens on these devices have diminished the value of pushing the GPU technology harder than we’ve been pushing it, because you don’t need to produce something excellent if it’s going to be displayed on these tiny screens.

When you’re completely immersed in this full field of view of experience, you’re going to value the high quality experiences much more than the low quality ones. We’re going to see an entire revolution take place in the field of mobile devices.

As far as today’s mobile devices, they’re limited by weight and form factor. You need to hold it in the hand as you use it. The future form factors for mobile VR and AR could be something that fits in your pocket, with a more powerful battery and GPU sending a signal to glasses on your head. We’ll see some major innovations in form factor there.

GamesBeat: How realistic does it have to be?

Sweeney: Michael Abrash said that your eye is going to be able to distinguish resolution in games up to about 24K horizontal resolution. That’s dramatically higher than anything built today. It’s six times higher than a 4K display. It becomes extremely interesting once you get from current resolutions — about 1200 pixels at high resolution per eye, which is about the same experience as playing DOOM on a 14” monitor 20 years ago – to 4K displays per eye.

At that point you have the ability to display high quality text for user interfaces and a combination of 2D and 3D content that’s extraordinarily impressive and acceptable to users. It’s the difference between a Retina display and a legacy display.

GamesBeat: If that’s how good it can get, do you worry that in the meantime we’ll have such poor quality stuff that people just won’t take to it?

Sweeney: We’re going to be in early adopter mode for at least the next two years. Now is the time to be investing.

If you look at the economics of the business, right now there are more than a million indie mobile developers worldwide producing 400,000 games per year. Maybe one percent of them actually make a living from their work. These 250 VR startups that have formed in the last couple of years, they’re ideally positioned to become leaders in their field. They have the opportunity. Some will succeed and some will fail, but now is the time to be making an investment.

There isn’t a whole lot of economic potential. You’re not going to be making a billion-dollar game for the next couple of years. But you can build an audience and make a name for yourself and create what needs to be carried forward to a leadership position when this reaches an exponential growth phase. The companies that are forged now are going to be the big winners in five or six years when the consumer market develops.

GamesBeat: You had a lot of discussion this year on the topic of the “indiepocalypse.” How does that tie to your views on the move to the new platform?

Sweeney: This is something we must avoid in the transition to VR, bringing in a million indie developers without any real hope of economic opportunity for most of them. In that regard, the indie revolution—it’s a small number of games making tens of millions of dollars, and then anyone who isn’t in the top one half of one percent isn’t earning any money.

The traditional game industry has provided a more fruitful model. There’s a much better correlation between high production value games and reliable revenue sources. You have reviewers and other marketing staples that drive gamers to a wide variety of games. There’s also a helpful stratification of the market into more genres of games, each reaching their own audience. It’s not just 400,000 games all competing for the same top 10 chart. It’s a much richer experience than that.

The other thing we’re seeing, which we’ve been participating by supporting modding—There was an indie game released that sold two million copies in its first couple of months of release. It was a game that supported modding and built its own modding community. For all these indie developers out there, there’s going to be a major business opportunity in building and expanding content for existing games. Instead of 400,000 games competing for the top 10, why not have a market with 50 top games in different genres and with different appeals, and hundreds of developers contributing content to all those games?

Valve announced earlier this year that Steam Workshop content creators made more than $40 million in the past year. That’s a real opportunity. I think it will become a much more viable model for a lot of the smaller studios.

GamesBeat: User-generated content is an escape route, in that case?

Sweeney: I think so. The market is becoming more functional and there are opportunities in stratification. There’s a wide variety of products and a rich ecosystem with different roles for people who want to be a part of that ecosystem.

GamesBeat: What’s your general view of tech? I think you’d said that in your two decades making games, processing power has increased 100,000 times. What are you expecting from advances in technology going forward?

Sweeney: We’re at an astonishing point already. We’re able to render these VR scenes with extremely high fidelity. The quality of rendering in modern games, and not even games — in architecture and product visualization — is astonishing.

We still have about two decades of major progress to go there. VR is going to be driving it. VR is going to require much more investment in GPU technology, battery technology and display technology. Everything we’re doing nowadays at Epic we’re doing to build the best VR experiences possible and support growth on the triple-A side.

GamesBeat: What do you think about the different pieces around VR that have to be developed still? The progress we’re making with hand gestures and body detection, what you’ll do with your legs, 3D audio — how do you feel about the rest of the technology around VR, beyond headsets?

Sweeney: We’re past the starting line now with the basic VR headsets — Oculus, HTC, Sony. They’re solid. They provide a starting point from here on. The motion controllers provide the first step toward direct input – going from moving a joystick to actually being able to reach out and manipulate things in a way that your brain has been doing since birth.

The next steps are going to happen along all axes. Huge advancements in display technology are needed, huge advancements in processing power. We also have to do a much better job of putting pixels where they matter. Even though your ideal display screen in VR would have to be 24,000 by 24,000 pixels to be indistinguishable from reality, your eye only has about 20 million photoreceptors. We have to draw pixels in the right place, where you can perceive them, by tracking your eyes. We’re going to need gesture-based tracking, so that you’re tracking not only one point of position or rotation for each hand, but the individual fingers, so you can reach out and interact with scenes.

Technology is going to be much of a race than people realize now. We’re thinking about games and we’re thinking about movies, but when you think about the content creation process — and this is something we’re building into the Unreal engine right now – instead of looking at your scenes on a 2D monitor and modeling them using a 2D interface with a lot of complicated shortcuts, you’ll be able to reach out and manipulate a scene. Pinch to zoom in, pinch to zoom out, look around, focus on objects—You’ll be able to do much more just using gestures. It will make the bar for getting into art, modeling, game design and everything else far lower than ever before. It’s going to work the way your brain works.

GamesBeat: Since it’s so hard to develop VR applications that don’t make you sick, do you feel more responsibility as the engine maker to show the way?

Sweeney: Absolutely. This is how we’ve approached every generation of technology. You start out with very early hardware prototypes. We build a series of tech demos, each one more advanced and building on the previous lessons, to learn what’s possible and show the world some of the opportunities that are out there. You can download the Unreal engine for free, download the Showdown demo for free, and see the content as we built it and play around with it.

This is the very first phase of our development for the new platform. The next phase is building games. Just as our early tech demos in the Unreal Engine 3 generation led to the development of Gears of War, we’re building on our arsenal of techniques for future Epic games.

GamesBeat: If I was a developer seeing your Bullet Time demo, I’d be a little intimidated. You did that with 12 people over 10 weeks.

Sweeney: Exactly. But it’s still small enough that an indie team could achieve something like that and then turn it into a full game and ship it in less than a year.

GamesBeat: That was just something to entertain us for a few minutes. How do you plan to build a game that people keep coming back to?

Sweeney: A lot of the work is in building a framework. Once you’ve built the new waves of enemies and the new types of weapons, it’s a pretty well-understood process. With the effort we put into this demo, I think we could ship is at a small scope, but high quality game.

GamesBeat: What do you see as the consequences of some of the technology in VR and AR? You’ve been saying this year that if you have perfect AR and a perfect 40-foot screen in front of your eyes all the time, why would you ever need a TV or a screen on your smartphone or a movie theater? It sounds like that would be fairly disruptive to the tech world.

Sweeney: Disruptive, but also empowering. The nice thing about involvement in this wave of the future — imagine your ugly sunglasses with a 4K display for each eye. Over time the hardware will be cost-reduced to the point where if you live in Africa and all you have access to is a cell tower and a power source to charge your device—With a $100 device you could have a far better entertainment experience than anybody can buy for any price right now.

That’s maybe 15 years out. But we’re about to see some big changes. It’s going to move away from this weird state the industry has been in the last few years with the mobile revolution. We went from super high-end games dominating the market to very simple ones. You have to realize that phenomenon arose because we have these ubiquitous devices with really tiny screens that only allow for a very limited game experience. After years of experimentation we found these awesome ways of monetizing 2D games that don’t have a lot of depth.

Once you go back to a fully immersive, better than console experience, it’s going to bring hardcore, super high fidelity gaming back to several billion new gamers to this audience that’s grown over the last few years. It’s going to be revolutionary for the whole industry.

GamesBeat: You said you saw Magic Leap under NDA and they were doing things you didn’t think were possible. Can you narrow that down a little bit?

Sweeney: Who’s heard about the Xerox PARC laboratory from the ‘60s and ‘70s? It was before my time. But I feel like what I saw there, it was like an extension. I hadn’t thought some of that stuff was possible, but they were doing it right there. They had the devices in their lap. They were making it work. It felt like if you teleported back to 1972 and saw the first mouse, the first graphical user interface, the future of computing right there.

This is probably happening in a lot of other places besides Magic Leap that I haven’t seen. But that’s part of the revolution that’s happening right now.

Question: You mentioned editing and altering VR. Can you expand more on what you’re trying and thinking about in that area for developers?

Sweeney: We’re all building VR content. It’s pretty ridiculous that we’re using 2D monitors and mice to do it. It’ll take something that interprets gestures to do this, but you’ll be able to reach out with your motion controllers, one in each hand, and directly manipulate the environment.

I imagine it’ll work a lot like manipulating your iPhone. One button on each controller moves the world. You can pan the world around you and pinch the view, so you can zoom way out and then view it like you’re Godzilla from far above. You could zoom way down to human scale and paint fine detail on lawns. You’d have geography editing, painting with colors, all the techniques that you use now in a combination of Photoshop and 3D Studio Max for modeling. You’d use this with a motion controller and a bunch of buttons that bring up different tools.

As you’re in that VR environment, you’d need to be able to pull up and interact with traditional user interface outlets, which for now are a Windows-like user interface. The goal, when this has come to full fruition, is to be able to do live editing of full VR scenes from all directions without ever leaving the VR environment, including switching over and writing some code or drawing some shader notes in an individual shader system. We’ll be able to bring that all together into a cohesive VR experience.

While it’s not related to the 3D aspects of VR, there’s a lot of basic research to do in user interfaces. We’re doing a lot of experimentation to make that work. I think that’s going to come a lot sooner than later in a basic, usable form.

GamesBeat: Sounds like Tony Stark designing the Iron Man suit in the movies.

Sweeney: We’re very close to that time.

Question: Do you think, because of VR, there’s going to be any particular category of games that will utilize it especially well and capture a lot more market share?

Sweeney: Some of the first smartphone games were ports of shooters, and they didn’t work very well. The touch screen wasn’t any substitute for a joystick. We’re going to see the same thing with VR. You can’t run around at 30 miles an hour and turn on a dime without barfing pretty quickly. So we’ll have to see. We’ve experimented with a visceral shooter experience in the Bullet Time demo. It also looks very promising for tabletop games, where instead of being immersed in an environment, you have an awesome gaming set in front of you that you can reach out and manipulate, scaling up and scaling down, pinch to zoom, using a controller to interact with it.

The great thing about VR is that the input is going to be so drastically better than mouse input or games with touch controls on smartphones. It’ll make it much easier to build stuff and express yourself freely. That’s going to reach a much higher level of importance than on past gaming platforms.