Disclosure: The organizers of the Siggraph Business Symposium covered part of my expenses to Anaheim. Our coverage remains objective.
Every game company needs a super geek like Scott Cronce. As vice president of technology at Electronic Arts, the publisher of series like the first-person shooter Battlefield and city-builder SimCity, Cronce has to look at the newest technologies and figure out what strategy EA should pursue in terms of spending its own R&D dollars. He has to be a seer, finding the path through the icebergs so that EA doesn’t become the Titanic, blindsided by technological change.
[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":784328,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"games,","session":"C"}']As new platforms emerge and new consoles like the upcoming Microsoft Xbox One and Sony PlayStation 4 debut, Cronce has to evaluate them and advise the executive team about the geek details. Cronce has done this in various tech roles for more than 24 years at EA. He joined in 1988 as technical director in the simulations group and has worked on dozens of games.
We caught up with him at the Siggraph Business Symposium. Here’s an edited transcript of our interview.
Scott Cronce: We have two now, Frostbite and Ignite, and they come in various flavors depending on what you’re doing. As a developer, what that means is that you already have something to start with that’s owned by EA. It doesn’t mean you have everything. If you’re trying to stretch that engine beyond what it was intended for, or add a new feature to it, you still have some work to do. But at least you’re building on something that was already successful.
If you were going to do a startup and buy an engine, the one thing I would recommend is that you never buy an engine that hasn’t shipped a game. That proves out everything related to whether the engine works.
GamesBeat: That’s the old lesson that EA got from buying Criterion, right? But that actually did work with some games.
Cronce: That was a tactic that was a bad idea. Don’t try to converge onto an engine during a console transition and make it just the one for the entire company.
Over the years we’ve gone from a game engine per title to two large game engines that are used and supported across the entire company. One for Sports and one for Games. But that has to happen from the strength of the engine.
[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":784328,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"games,","session":"C"}']
Above: Scott Cronce
GamesBeat: If you have the engine, are you halfway there?
Cronce: It’s not so much that. What it means is that you have the runtime engine and all the pipelines associated with it. It’s like starting with all the rigging ready to go.
GamesBeat: You set the boundaries of the game by choosing your engine.
Cronce: Right. But at the same time, you can also decide to do something that extends it beyond them. You make sure that’s where you put your engineering dollars. Depending on what that is, it may be a branch from that engine specifically for a genre of games. Or it may be something that will fold back into the main engine so everybody can benefit from that feature.
[aditude-amp id="medium2" targeting='{"env":"staging","page_type":"article","post_id":784328,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"games,","session":"C"}']
Cost, of course, is always a consideration. Often that doesn’t mean you’re going to make a cheaper game, though. You’re going to spend the dollars someplace else, on the content in the game itself. That’s a good thing.
The other obvious benefit it that now you have a lot of people who know how to use the same pipeline. If you can get multiple people and teams using the pipeline, you can have a much more fluid movement of people back and forth, with very little startup time.
GamesBeat: A valuable asset, then, becomes something like a team that has worked with an engine before and shipped a game. That’s far more valuable an asset than what startups have.
Cronce: Much more. If you want to speed something up or slow something down, you can do that now. Instead of the normal month or two of startup time—If someone’s done this before on another game team, and you need to get a game out on time or ship it quicker, you have the freedom to add those people to it without the startup time.
[aditude-amp id="medium3" targeting='{"env":"staging","page_type":"article","post_id":784328,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"games,","session":"C"}']
GamesBeat: At Sony, Mark Cerny did a backward-looking analysis on the PlayStation 3 and what they learned for the PlayStation 4. He said that the time to prototype had gone from one or two months on the PlayStation to six months or a year with the Cell processor on PS3. Until the prototypes of the console itself shipped, there was no way to get started on games.
Above: PS 4
Cronce: During that last console transition, the trick of being able to just increase the clock speed to get more power didn’t work anymore. Both Sony and Microsoft had to break that cycle if they wanted to have a sub-$500 machine with a lot of power. Intel too, right? We’d hit the heat to performance barrier on the chips. We had to do multiple cores.
The entire industry was going through this at the same time. How do you break up a game, which is typically a very linear processing issue, into multiple CPUs? The Cell was just a little bit further away from what people were used to. It did take us a couple of years to figure out how to properly get performance out of it. That’s the problem everybody had in the last generation of games. It turns out that everybody was going to have to go through it at some point in time.
GamesBeat: He said that this time around, with x86, the tools are out there. Everyone knows how to make PC games already. The time to prototype should be nearly instantaneous. The part before you can start iterating on things has gone away. So his argument is that, theoretically, it’s much easier to make games in this generation. Your starting point may set a very high bar that becomes a difficult game to make, I guess, but your starting point is better than it was before.
[aditude-amp id="medium4" targeting='{"env":"staging","page_type":"article","post_id":784328,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"games,","session":"C"}']
Cronce: You’ve got Xbox, PlayStation, PC, and they all have identical x86-style architecture. Wii U is just a little different. That means it’s familiar. I would say it slightly differently than Mark does. It’s not so much that the architecture itself implies that it’s easy to do. It’s just that because everyone has so many years now with multiple cores and with x86 PC architecture, it’s well-known. It’s not this massive learning curve for getting a game engine running on an alien piece of hardware.
Cronce: The majority of the R&D is done in the game studio teams. It’s part of the culture. It’s driven by-product design, or sometimes by the freedom to just say, “See if you can go make this cool water effect happen. See if you can get this radiosity effect working within the pipeline.” One thing is being able to get it up and running so people can see it. The second step is to figure out how we put it into the pipeline of the game. Unless you can do that, it’s not going to work. People will come up with some really cool stuff, but it completely breaks the pipeline, and that means it might not make it in this year.
[aditude-amp id="medium5" targeting='{"env":"staging","page_type":"article","post_id":784328,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"games,","session":"C"}']
GamesBeat: It sounds like applied research in the sense that you’re working within the context of having chosen a game engine. You’re not going to go make a new engine.
Cronce: We used to have research with no development. We’ve gotten away from that. If it’s pure research and it’s too far away from when the intended generation is going to ship, you often make cool inspirational demos, but they don’t have any effect on anything that might be shipping.
Cronce: Microsoft still does a lot of research. It makes sense, if you’re making a console, to continue to do research and try to find out which of your ideas is going to be big. That’s part of what they do. Software ends up being a lot simpler. The cycle of idea to demo to production of that R&D is much faster. With hardware, you think, “Okay, I want to have this visual control system, like the Kinect.” Then you’ll find different companies that do it. None of them may have everything you want, so then you have to go build it. That might be two, three, four years of work.
GamesBeat: Do you have some people who actually are technologists, who are still in some ways centralized R&D?
Cronce: Yeah, that’s EA Tech. EA Tech is what’s left of the Criterion organization. They’ve gotten into a very healthy spot. They have tech, like our animation engine that everyone uses. They have the physics engine that’s used by most game teams, and all the commodity software. We try to build a simple, easy-to-understand API that can then work on all platforms. That’s the majority of what they’re doing. They’re trying to figure out, “Okay, if I want to be able to do one function, I want that function to work on multiple platforms. I don’t want to have to think too much about it.”
You can’t do that for everything, but for a lot of things you can. By doing that, it makes it simple to have a few developers create something that’s going to save time for thousands of developers. It’s something we’ve always had in some form, since the very beginning. In the early days, when we were a publisher, all the engineering we had internally was specifically for those tools and cross-platform tech.
GamesBeat: Then you have things like EA’s Origin online-games service. Is that coming out of a business group, or is that part of R&D?
Cronce: We really don’t have pure R&D. It’s coming out of the business group. They’re responsible for producing those pieces. We’ve found that R&D that’s not applied to either feature effect or a problem is not useful.
The trend you’re seeing more and more is that the service side of running a game is becoming a larger component. You have two choices. You can have each game team do it themselves, or you can look at what the common pieces are, look at the future of what you want to have, and then build that separately.
GamesBeat: As games go digital, then, there’s this associated task of providing that infrastructure.
Cronce: It’s also the other trends, like large data. You want to be able to know how your games are being consumed. You want to know what users like or don’t like about them. It’s part of that big-data component — being able to use the telemetry in games not just for development, but also when they’re live as well.
Cronce: We do. Right now everyone’s going through the cycle of learning these platforms that they’re in the process of finishing. That means spending a lot of resources on creating what will be the new engines that we use for this next generation of consoles. Each team has to figure out how they’re going to allocate that across their team.
If I have a franchise, and I’m shipping both a gen-four and a current-gen and a PC title, I have to figure out how I’m going to spend my R&D dollars in order to set myself up for the future. We put that down at the level of the individual game, the individual franchise, the individual division, and the label. They get to figure out what the best way is for them to do that.
Above: Microsoft’s Xbox One console.
GamesBeat: What are the tech things that would keep you up at night, the challenges that create risk for the company?
Cronce: Luckily, after close to 25 years, I can still go to sleep. [laughs] But I’d say the larger issues include trying to deal with much more complicated online systems than we’ve dealt with in the past. We’ve always liked the idea where the server side of the game has foreknowledge about what’s happening in the game, so we can maximize the number of players in it and make it into a living world. Most of the people in the game industry have done that by buying bigger and faster hardware and doing it all locally. The new challenge is that we have to now break up those problems into smaller pieces.
It’s similar to what you saw when we had to move to multiple cores in the PS3 and Xbox 360 days. We’re now having to figure out how to do that with servers, and it’s slightly different. How do you break your logic of your game, which is normally running on heavy hardware in a backend server, into a bunch of smaller virtual components that are all running on cloud systems like Amazon?
GamesBeat: Is everybody going that way? Microsoft is the one that’s talked about cloud processing.
Cronce: Very few people, I think, are heavily investing in it right now. It’s something that we see as a challenge that we have to solve, in two ways. We don’t want to run our own server hardware, but we have to do so today. There isn’t anything out there that’s approaching the type of low latency that our gamers require. But we think that the horizon is changing. We’re in a model where we’re not going to be able to buy the hardware necessary to have thousands or hundreds of thousands of people playing at the same time.
That’s on one side. The other side is the mobile, free-to-play market. You don’t want to build a server system where you’re spending a whole bunch of money supporting all the people who aren’t actually paying for the game. It’s both a cost issue and a technology issue when you’re thinking about free-to-play.
Cronce: We always learn from our mistakes. Same thing with The Simpsons: Tapped Out when it originally came out. What that says is that we, as an industry creating games, haven’t mastered that kind of broad server-side scalability. You see that from everybody. Almost always, when a large online game comes out, there are troubles in the first week or so.
GamesBeat: That was hard for me to understand from the outside. We hear all the rhetoric, at least, about things like Amazon Web Services, which was built to alleviate these kinds of crushes. You can go get some computing power from them if there’s a peak in demand that your own servers can’t handle.
Cronce: That’s right. But what happens is that sometimes, there’s nothing like a good live demo to let you know when a portion of the architecture isn’t scalable. If you have anything that’s not scalable, then it doesn’t matter how many VMs you spin up. You’re not going to be able to solve the problem.
It’s the same thing you see on websites. Somebody says you’re going to build a new fancy car this way, you get a few million hits, and boom, the website goes down. One piece that nobody thought about turns out to be the core. It’s not just a games problem. But it is unique to games because we actually have to have a different kind of realtime low-latency backend, and it has to be scalable. That’s the biggest challenge virtual machines that we all face, as well as adding features into those cloud pieces.
You want to have an entire living world. We have to do this in a way that’s easy to scale, and also not too expensive per user, especially in a free-to-play system. We have to think about compute power per user. How much is it going to cost to deploy a game where some fraction of the audience will pay you money?
Above: Scott Cronce
GamesBeat: Does anything interest you here at Siggraph that relates to games, that’s coming from another part of the industry? People always make that comparison about how the gamemakers have to reinvent their camera every few years or so. Things have been stable in the movie industry so long that it enables a more sophisticated level of creation. It isn’t held back by having to learn new technology.
Cronce: It’s true that the output for movies is at a much slower cadence. But I think the bigger thing is that whatever we end up doing has to be in real-time. In the movies, because of the amount of time they have to go back and redo things, you could hand-touch every single frame if you needed to and make it look perfect. We have to ship a game that runs in real-time, all the time. That’s a much bigger difference. Then it’s an added complexity that our devices change on a regular basis.
GamesBeat: Ubisoft opened up a motion-pictures division. It seems to suggest that the technologies could be converging.
Cronce: With the tools we’ve had in gen 3, definitely. You see a lot of the same artists move back and forth between movies and games. The animation tools that they use are identical. What happens is that a lot of the base models are pretty much the same. Ours, by the time we’re running them in-game, are much lower in resolution, with fewer polygons, but the tools we use at that base level are the same. It’s just that the pipelines for getting those models into a movie and getting them into a game are different.
GamesBeat: There are still some things coming out of left field for games, like the Oculus Rift. It’s gotten everyone excited about virtual reality again. Is that something you guys look at as well?
Cronce: Definitely. The first game we did with VR was with a VX-1 helmet. We did a couple of flight sims with it. The difference back then is that they were so expensive. In the ‘90s you had a $700 or $800 helmet with a 320 resolution. It wasn’t very compelling. What we’re seeing now is the hope that these companies will come out with a compelling VR system that’s relatively inexpensive and high-resolution. Right now we don’t know whether it will be like the other dozen or so that have come in the past, or whether it’ll actually be a breakthrough that people want to use.
Above: Oculus Rift
GamesBeat: It seems like your due diligence on tech must be busier than ever, with all kinds of things coming out now. You have GameStick, Ouya, Oculus Rift, lots of Android things.
Cronce: Every day something new pops up. It’s more around what you decide not to support. We see even more of these Android consoles in Asia than what we see in the news in the U.S. The question is, which one will actually spend the money to become a platform? That’s more than just commodity hardware and software. It means building the platform from a developer point of view, a consumer point of view, and a marketing point of view.
It’s very exciting. Some of these are exciting just for developers to play around with, even if they don’t believe they’ll become the next big thing.