Epic’s graphics guru Tim Sweeney on AR/VR, game engine wars, and simulating humans

Above: Tim Sweeney, the CEO of Epic Games, has been making video games for two decades.

Image Credit: Michael O'Donnell/VentureBeat

GamesBeat: As tech gets more powerful, making games on top of that tech can get harder and harder. How do you keep tools simple and making games more accessible?

Sweeney: Einstein had an axiom on that. He said, “Keep it as simple as possible, but not any simpler than that.” Our experience building games is that it’s easy to start building a game now. It’s an order of magnitude harder than that to ship a game. It’s three orders of magnitude harder to ship a game that’s commercially successful.

What we do in all of our efforts is optimize for the ultimate outcome of success. Building tools that favor being able to build and maintain over a life cycle, scale, debug, ship, and maintain large-scale games. This necessarily means tools that have a steeper learning curve, but it also means tools that are able to reach these high magnitudes of success with a higher probability.

With Unreal there are multiple games that have made more than a billion dollars in their franchise lifetime with the engine. It comes from this high-end console background and it’s worked its way down to be more usable. It’s an interesting contrast to coming from the other direction.

GamesBeat: What’s your update on the game engine wars? Unreal comes from triple-A on down while Unity has bubbled up from the bottom. What’s the competitive situation like?

Sweeney: As far as wars go, this is a pretty gentlemanly war. There are two major engines used widely now, Unity and Unreal. They’re both thriving. Since we made Unreal free at GDC this year, about seven months ago, a million users have chosen Unreal. These are serious developers building serious games. We’ve been stunned by the growth. John Riccitiello recently announced two million monthly active users with Unity.

That’s amazing, that the game industry has progressed to this point compared to a few years ago, when it was this exclusive club of super-elite developers. It’s a much larger market now. If there’s a war happening, it’s between the game developers who are trying to get their games noticed and reach their audience. It’s far harder for them and their businesses than it is for us at Epic and Unity as engine providers. It’s on us to be able to provide them with the tools they need to be able to achieve those levels of success.

GamesBeat: Is your approach very different from Unity’s?

Sweeney: The first three generations of Unreal had their own scripting language, UnrealScript, which I designed. It was a great way to get started with a small engine. With a bit of coding you could get a project up and running quickly. But what we found over the years was that it didn’t scale up to a larger system.

Let’s face it. We have a C++ operating system at the foundation. We have a C++ game engine built on top of that. Unreal is written in C++. Unity’s written in C++. If you’re building your game in C++ then you have access to the entire technology stack, completely unfettered. The benefits of that, having full engine source – go to unrealengine.com and you can download the full Unreal source nowadays – having the full ability to debug and understand the entire technology stack is really important once your project reaches a certain scale and complexity.

At times you need to dig in and figure out what’s really happening there. It’s a more expert level of development than the sandbox scripting language approach that Unity has now and we had in the previous generation, but we feel that at a certain point it is necessary in scaling. I made the decision to kill UnrealScript, my pet programming language, and move to a pure C++ engine.

Above: Epic Games’ demo for Oculus Rift virtual reality headset.

Image Credit: Epic Games

GamesBeat: Do I have to be geekier, then, to make games on Unreal?

Sweeney: The neat thing about Unreal is that there’s also this visual programming system called Blueprints. Without writing a single line of code, you can build a small indie-scale project. A number of games have been shipped without any programming whatsoever in Unreal.

But for programmers—To move from a scripting language to C++ is a learning curve. It’s a rite of passage in the industry. But there is a magic to it. By writing code in that manner, you have complete access to the entire operating system and game engine. You have code that can run 20 times faster than an interpretive scripting language. When you’re aiming at higher-end experiences – games with lots of objects, VR and AR where you’re trying to run at 90 or 120 frames per second to keep up with the hardware – it’s a matter of life or death for your project.

GamesBeat: We had Sinjin Bain earlier talking about how game engines need to be brought into the modern age, how cloud computing and other factors need to be applied to game engines. Does that resonate?

Sweeney: If you look at all the major successful games, they have a large component running in the cloud, running game logic and persistent stuff. They have a graphics engine running on your device, be it PC or mobile. For the foreseeable future, we’re going to see that architecture. We’ve moved an awful lot to the cloud. We still need to run a lot on the client, though.

There’s been a theory that video streaming will power gaming “soon,” in just a few years. It’s always been just a few years. With AR and VR, you’re pushing toward a more mobile form factor and a much lower latency requirement for games. It’s pushing the possibility of cloud rendering out even further. So there’s always going to be a client-server separation there. It’s important for the game engine to be pragmatic about that tradeoff.

Above: A Boy and His Kite demo of Unreal Engine 4.

Image Credit: Epic Games

GamesBeat: As far as your graphics experience goes, when you look at a modern game with motion capture and face capture and so forth these days, what do you notice that can still be improved?

Sweeney: The way that high-end triple-A games solve this problem is by brute force application of tens of millions of dollars to motion capture, facial animation, and acting. It’s a process that works well, if you look at games like Call of Duty and what’s being done with massive budgets. The problem is that it’s woefully uneconomical for most developers.

The real challenge in that area, in simulating more realistic humans and other game systems, is not just how to reach the maximum level of quality that’s theoretically attainable within the engine, but how to make it as economical as possible to do that. Everybody is always going to be under budgetary pressure to create experiences.

All of the efforts we’re undertaking in that area are to make it as straightforward as possible to make human animations – creating reusable libraries of animation, reusable skeletons – so that with as little work as possible you can take an existing human, modify it to do what you want, and have it largely work.

Above: Samantha, played by Hayden Panettiere, in Until Dawn.

Image Credit: Sony

GamesBeat: I love the faces in Sony’s Until Dawn from this summer, but as soon as the people move, the motion of the bodies tends to give it away. It’s not just getting the faces right.

Sweeney: Yeah. There you get into the problem of simulating human intelligence. Having a conversation with a game character is going to result in them having to do what a human would do in that situation. It’s a difficult problem and nobody knows the answer to it yet. It’s not just rendering, where if you get enough computing power, we know the laws of physics, and we can simply simulate the flow of light. With intelligence we don’t know the algorithms yet.

You can talk about neural nets and deep learning, but nobody knows how to apply this yet to anything more than just a brute force algorithm. It’s one of the major problems we’re going to face over the next decade.

GamesBeat: Are you optimistic that we’ll eventually get to simulating reality, simulating believable humans?

Sweeney: That’s the hardest problem we face in the industry. I do believe that within the next decade, everybody’s smartphones, desktop computers, televisions, all these computing devices will be replaced with augmented reality devices that are as convenient as glasses. That will be a fairly natural, incremental transition over that time period. But I feel like simulating human intelligence is more than a decade out. It’s a problem we’re likely to be grappling with for the rest of our lifetimes.