Nvidia CEO’s 7-year journey to make the Project Shield portable gaming device (exclusive interview)

Jen-Hsun Huang has thought about making Project Shield, the portable Android gaming system that Nvidia unveiled this week at the Consumer Electronics Show, for the past seven years. But the chief executive of the world’s biggest graphics chip maker had to pull a lot of things together first before deciding to make it — like figuring out how to make a gaming console.

Nvidia designs chips such as the Tegra 4 inside the Project Shield, but it’s never gone directly into the consumer market with its own game system. (It did go out on its own into the retail market before with its 3DVision 3D glasses in the past). Nvidia has also worked to establish a cloud gaming network dubbed the Nvidia GeForce Gaming Grid, which provides the underlying foundation for delivering games to the Shield device, which includes a 5-inch screen atop a game controller. Then it designed a new kind of processing for its latest graphics chips. And in the past year, it put the finishing touches on its Tegra 4 mobile processor, which has four computing cores and 72 graphics cores.

It would be wrong to think that Shield is just another Android game controller. It has a whole ecosystem built around it and high-end components for hardcore gamers. In short, Project Shield is a massive effort that required the work of thousands of Nvidia employees. Now, it’s finally ready, and Huang showed Project Shield to us in an interview at the Nvidia booth at CES. (See our hands-on vide0). Here’s an edited transcript of our exclusive interview.

GamesBeat: How heavy is it? A pound or so?

Jen-Hsun Huang: Here you go. It’s under a pound. A bit under a pound.

GamesBeat: It feels like a regular Xbox controller. 

Huang: Exactly. That’s ideal. We haven’t tuned any of the knobs on this, or the buttons, but ideally it should respond just like one. The moment you pick it up, you should recognize it as a controller.

There are two things that we wanted to do. We wanted to make sure that for anybody who picks it up, and in the moment that they do, they’re instantly familiar with it. All the controls are exactly where they expected. Secondly, because it’s pure Android, you know exactly how to use it. Because this is pure Android. If you’re an Android user — and this is targeting Android users — the moment that you turn it on, your whole life shows up. All your music is there. All your movies are there. All the Tegra Zone games you’ve already bought on your phone are there.

GamesBeat: What are some of the reactions you’ve heard so far? 

Huang: I would say that the No. 1 reaction is, “Wow, it’s a lot smaller than I expected.”

GamesBeat: What design decisions can you talk about? You could probably have just had a tablet sitting on top of something that you could attach, taking the controller away. I think I’ve seen other designs like that out there already. Is there a reason not to do that?

Huang: First of all, a tablet is too heavy. It’s too top-heavy. Second, notice that this isn’t a tablet. It’s just a display. All of the electronics, the batteries, are in here. The balance of the system wants to be in your hand. Even a phone is too top-heavy. Worse, you don’t want to drain the battery life of your phone. Your phone is used for communication. The benefit of using Android is that all of the content on your tablet or your phone automatically shows up here. The instant familiarity of the system is a real plus. The connection to your open system, with all of your digital content, is a plus.

The design decisions started with this. It has to be a wonderful gaming device. The controller can’t be sacrificed. We started with the palms of your hands, how everything fit into your hand — the bumpers, the triggers, the buttons. They have to be in the appropriate places. Then we fit all of the technology around it. That’s No. 1. No. 2, it has to be as small as possible, to the point where the volume of this device is approximately a game controller. The reason for that is because if you’re willing to carry a game controller in your bag, then from a volume perspective, you want to carry this in your bag. The third thing, of course, is that the performance has to be amazing.

And then, the magic trick … as a device that is consistent with the way we expect to consume media in the future — wireless, cloud, streaming — those kinds of experiences have to be introduced. Streaming from your PC. We do streaming to your TV. In the future we’ll do streaming from Grid. Not only is this a wonderful game device, but it’s also gaming in a new way, the way that we expect to consume digital content.

GamesBeat: Are you filtering Android content in some ways so that it works with this?

Huang: No, not at all. This is pure Android.

GamesBeat: Some of those games are designed more for touch, though, as opposed to buttons. How does that convert?

Huang: The way we’ll do it is this. Everything will work. But for games that are designed for controllers and also optimized for Shield, we have the Tegra Zone. All of your games in here just work. This is your Android experience. Everything is there. This is your gaming experience. Tegra Zone pops up. You’re playing your games. If, all of a sudden, you got a text and you want to check it out, and if you want to go back to that game, it’s exactly like Android. The behavior is exactly the same if you’re connected to the PC.

GamesBeat: How much of the hard work focuses not just on this but also in the whole ecosystem around it? Like, for a console maker, they have to go round up all the game publishers and game developers to get behind a console some months or years ahead of time.  

Huang: We are perpetually doing that. You know that we work with game developers all over the world to enhance for GeForce or to optimize for Tegra. All of the optimizations that we made for Tegra are in Tegra Zone, and all of those games just showed up. For the last two years, all of the games that we’ve optimized for Tegra have already included controller support. We’ve been preparing for this day. All of those games, the 100 games or so, and the few hundred games in development right now, they’re all being optimized for this.

GamesBeat: How much of this resembles a console business and how much might be different or disruptive to it?

Huang: It’s completely different. That’s a really good question. We’re not trying to build a console. We’re trying to build an Android digital device, in the same way that Nexus 7 enjoys books and magazines and movies. This is an Android device for enjoying games. It’s part of your collection of Android devices. That’s how we think about this device. All of the economics associated with this device are exactly the same. We’re going to have to sell this device based on the value of this device, for people who are enjoying it and finding it useful.

The differences are this. The console, as you know, is proprietary, it’s closed, and it has a razor-and-blades business model. Our business model is open, it’s based on Android, it’s completely familiar to you, and the games range from free to free-to-play with virtual goods and also, of course, to premium games. This is the perfect platform for free-to-play. The free-to-play publisher wants to get their games on as many platforms as possible. Broad distribution is what they’re looking for. Broad distribution isn’t what the console guys are looking for. They’re looking for exclusivity. For us, this is just a fantastic thing we’re doing for that industry. That’s why free-to-play works great for PCs. Free-to-play is going to work great for Android, and if it works great for Android it’s going to work even better for Shield.

GamesBeat: What was going on in your heads during all of this time? People were saying that free-to-play was disrupting everything on Facebook, for instance, or iOS. They said that somebody had to bring this to the consoles and disrupt those $60-dollar console games, but nobody did it. Then things like Ouya started popping up. It seemed like it was taking startups to do something the established companies wouldn’t. You guys are an established company in different ways, partnering with some of those guys. What was your thinking when the opportunity for something like this came up?

Huang: If you talk to Julie at Ouya, she’ll tell you that her best partner in the world is Nvidia. We have hundreds of engineers working on her behalf to realize that game console. We love what they’re doing. But I frankly think that Ouya and that game console could be made by other companies aside from us. We can do it, but other companies can do it too. The reason why I built this device is because only we can build this device.

This device, Shield, has such incredible performance. It’s got the software stack that treats it like a server to stream to television. It treats it like a receiver for servers in the cloud and on your PC. The software that’s necessary to do that is so complicated. No company in the world is going to get it together. We’re going to have to do this. It’s the same technology that I was going to use for GRID anyway. The GRID technology goes into my PC. My PC now becomes a GRID server for this. I put that same technology in Shield. Now Shield becomes a GRID server for television. I’ve got all the technology that I can leverage. It’s very delicate and complicated technology, because it has to work, but it also has to work with very low latency.

GamesBeat: What came first here? Graphics chips for supercomputers, or this?

Huang: In the schematic that I drew, we have a supercomputer in the cloud streaming to PC, to a mobile device, and to a receiver on television. I put “receiver,” because it could be Ouya. It could be a dongle. It could be an Xbox. It could be a smart TV. It could be anything, right? It doesn’t matter. Receivers of all kinds. I’ve been working on that schematic for almost seven years.

The question is, how do I get it all connected in a way such that not only will the technology come together, but there’s a way to bring the whole service to the marketplace, the whole experience. That’s why we had to create GFE. The reason why this works is because Tegra Zone is here. That’s why we had to create Tegra Zone. Without Tegra Zone, how would I have the games? Without the games, why would this be special? All of this technology had to be built in pieces and taken to market. We had to allow them to be enjoyed in the marketplace individually. Then, finally, when the final piece comes together, everything else clicks in. It’s been quite a journey.

GamesBeat: With GRID, it looks like that strategy is a broad strategy where you could have lots of partners. In this business, it almost seems like you compete with some of those guys. I wonder how you navigate that. 

Huang: I don’t think anybody’s going to make a mobile device like this. I think this is a little extreme. The R&D budget necessary to create something like this is pretty significant.

GamesBeat: So a simple game controller attached to an Android phone is not going to do what you just talked about here.

Huang: It’s a very different experience. It emulates or approximates the experience. We’ll work with all of them. We’ll help them in fact. The reason for that is because the more devices are out there, the more Tegra Zone will become valuable. The more Tegra Zone becomes valuable, the more devices will be out there. We don’t mind that this is the center of the ecosystem. It doesn’t have to be the only device in the ecosystem. If other people would like to build Android devices for gaming, that’s fine. We can’t stop anybody. But this is a device, a game store in Tegra Zone, and an integrated ecosystem that quite frankly I don’t think anybody else in the world can do.

GamesBeat: You make someone like Google happy doing this, I think. 

Huang: They should love it.

GamesBeat: But Microsoft and Sony, maybe… [laughs] They may have to get used to this kind of future.

Huang: Well, a console is a dedicated gaming experience. It’s a very specific experience. It’s beneficial in its own ways. The platform stays constant. It’s just less messy. They’re perfect experiences made possible because of the nature of the business model. I don’t think that’s going away.

We’re not trying to be that. We’re going to try to be something very different. If anything, this is just an extension of your Android device family. I now have a smartphone. I now have a seven-inch tablet. I have a 10-inch Android tablet. I have a 10-inch Android transformable. And now I have a sexy little Android device for playing games. People are making Android TV pucks. There’s Android stuff all over my house. This is just yet another Android device.

GamesBeat: For games, are we going to see 4K content materializing because something like this exists? 

Huang: We will definitely be able to generate 4K content from games sooner than 4K video content becomes available to consumers. I can synthesize the pixels.

GamesBeat: Is that what you’d call scaling up…? 

Huang: I can upscale it, but I’d probably just synthesize it. GeForce renders to 4K all day long, right? On 25″ by 16″.=

GamesBeat: So it’ll be more like a good PC game display. 

Huang: Exactly. I’m going to stream 4K right to this thing and HDMI it right to the TV.

GamesBeat: Valve are very interesting partners here, too. They like openness. Is that a reason for you guys to tie up here?  

Huang: Because we’re open and because they’re open, we are tied by definition. Notice that we instantly demonstrated Steam running on this thing. They didn’t have to lift a finger. We instantly put Steam on television. That’s the beautiful thing about it. That’s the whole point of openness. All of my available digital content is instantly available on this device. Here’s the wonderful thing. Next year, Shield 2 comes. I buy it, I turn it on, and all of my Shield 1 stuff is there.

GamesBeat: That’s another question. How many models do you think you’ll need to have? 

Huang: We’ll make one model a year.

GamesBeat: Are they backward compatible?

Huang: By definition. Just like an Android device.

GamesBeat: Do you want to change around your screen size? Have multiple models that way?

Huang: The second generation, of course, I don’t know what it’s going to be yet, but obviously we’ll keep innovating and keep making it better. People are telling us how we can improve and we’re learning from that.

GamesBeat: It seems like there would be one challenge, in that you might have to teach people that this is more than a game controller. They’re used to paying $40 dollars for a nice game controller. They might not realize… 

Huang: For people that have seen it and read about it on the web, I think the reaction has been pretty overwhelmingly exciting. The reason for that is, we’re targeting Android gamers and PC gamers. They’re not six years old. They’re not grandmothers. We’re targeting our core market. They understand what this is. Our core market are open-platform gamers. They want to be able to do free-to-play. They’re the guys that buy brand-new GeForce cards every year at $300 dollars. They’re the guys buying GeForce cards at $1,000 dollars. These are people who love gadgets. They get it.

GamesBeat: Backing up a little, how did you get past the point of making your own hardware and selling your own hardware? In the same way that Microsoft was a software company until they decided to do the Xbox, or Intel at some point made systems…  

Huang: We never once thought like that. We always think like this. First of all, “Is this something that the world would benefit from if we were to build it?” Number two, “Is this something that we should go off and build?” Then we ask ourselves, “Is this something that we can build better than anybody else in the world?” You ask yourself, “Who would build this?”

It turns out that Sony’s not going to build this. Nintendo is not going to build this. It doesn’t make sense for Dell to build this. This is something that we believe could make a real contribution to the world. We believe that this is something we could build like nobody else could. This is something that none of our existing customers really make sense to build. So if it’s something that has to be done, and we’re the best in the world to do it, and we’re actually the only person to do it, then of course we’re happy to do it.

GamesBeat: Did this make sense, too, because you were not deeply involved in a particular console project at this time as well?  

Huang: We started this project long before any console project. We’ve been working on this whole system now for five to seven years. It’s not related to that. I would still love to have built any of the game consoles, so long as two conditions would have happened — that the economics were right, and that it wouldn’t have distracted the company too much. These game consoles are very consuming. Every console is a radically different computer — different operating system, different system logic, different architecture, different system software. Everything is brand new. There’s no leverage whatsoever, except for the core technology and the expertise.

It’s very disruptive to a company. It’s very dangerous. Every single time I’ve worked on a game console, I’ve almost lost my life. [laughs] You’ve been watching me a long time. I think the reason for that is because they’re just radically different computers. It takes a lot of energy to build a computer. If I could do it, I would have enjoyed doing it. We just couldn’t do it. This is a leverage of our existing capabilities. Everything here is exactly what we were building anyways. By building this, it contributes to our existing strategy. This will help GeForce. This will give GeForce users yet another way to extend and enjoy their game experience. This will help Tegra. Every one of these games that are optimized and whatnot for Shield are going to show up in Tegra Zone. Tegra Zone helps all of the Tegra users. Everything here is amplified by the work that I do. Not a game console. Once I’m done with that, if the economics are terrible, there’s no recurring benefit.

GamesBeat: How much of this are you going to do yourself? When you describe GeForce GRID, people wonder, “Is Amazon going to help you with some part of that? They’ve already got lots of infrastructure.” Are contract manufacturers going to make this? Are you going to find those partners out there to help you in this business? 

Huang: Same thing. We should only work on the pieces of technology and the pieces of capability that nobody else is doing. We’re working with great partners to build this device. With respect to the cloud, cloud game streaming is going to come from a lot of different places. It’s going to come from hosted infrastructures like Amazon. It’s going to come from cable companies who have their own headends and their own data centers, because they want to stream the video game directly over a channel to you. They want to give you another premium channel to subscribe to. You have ESPN and you have HBO and here comes another channel called GRID or something. Then there’s also telcos who want to stream it as a game service over the 4G network.

There’s a lot of different ways that people are going to try to deliver this content and this service to their end users. Sometimes it makes sense for Amazon to host it for us or host it with us. Sometimes it makes sense for them to host it themselves. Other times, you’re going to find that cable providers or telcos are going to host it themselves.

GamesBeat: Is there some part of the infrastructure that’s not so great here? I would think of the wireless part of all this being, in some ways, not quite mature. I don’t know if Bluetooth Smart has to be used in this in some way. Or you’re using a cable because wi-fi is just not good enough, especially with something like 4K. How do you get stuff from Steam to the TV? 

Huang: All of those things are exactly the same as it is for video. The way to think about it is this. If Netflix doesn’t run, there’s no hope for GRID to run. There are many environments where Netflix doesn’t run at all. If Netflix can’t deliver decent service, then it’s going to be hard for us to deliver decent service. So we have all the same constraints. But if you’re sitting at Starbucks, my guess is that you’re going to rock and roll pretty well. If you’re sitting at an airport, you’re going to get a pretty good shot. If you’re sitting in a hotel room, you’re in pretty good shape. If we can’t figure out a way to deliver all 720p, they can always scale down to 480p, just like Netflix does. We understand the connection and we’ll adapt just like video streaming.

GamesBeat: I think some people who watched the demos were skeptical of how powerful Tegra 4 might be and whether it’s really approaching the high-end PCs in some ways. Where should the expectation be about how good the graphics will look, both on this screen and on a big screen?

Huang: Well, you saw Hawken. That was ported from a PC. Hawken is pretty stunning. The capability of the GPU is surely there. If we were to run the same game as Tegra 3 was running at the same framerate, the power is about half Tegra 3. However, you could deliver two or three times the performance at about the same power as Tegra 3. That’s the benefit of 28nm. That’s the benefit of better design and so on and so forth. It depends on what you’re trying to do. If I’m delivering the exact same experience as Tegra 3, I will deliver it at lower power. Most of the Tegra Zone games today, I’ll play them with lower power on Tegra 4 than I do on Tegra 3. But if it’s an extremely optimized and amped-up game like Hawken, we’ll run many times the performance of Tegra 3 with about the same or slightly higher power.

GamesBeat: You guys haven’t quite said anything about pricing yet.

Huang: We haven’t decided yet. Obviously it’s going to have to be priced for the value that it delivers. It’s going to be priced like an Android device. That’s the way to think about it.

GamesBeat: The timing here looks good. There are no consoles arriving in the next quarter or two. On the other hand, did you have to wait for anything to get this shipped?

Huang: The most important thing is Tegra 4. We are basically the first device on Tegra 4. We’re running at the speed of Tegra 4. The moment the chips come out of the fab, it’s going to drop like that 3D animation. It’s going to go right into another production line and come out the back end. That’s the beautiful thing about this. The customer who buys Shield is going to get the state of the art in everything.

GamesBeat: But it’s probably good that you’re not going to be selling up against an Xbox 720 or whatever.

Huang: Yeah, I guess at some level that’s right. At another level I just don’t think it’s exactly the same. This is targeted at people who are Android users and PC gamers. This is a device that helps them enjoy that better. If you say, on the other hand, that when Xbox comes out, everybody stops buying everything else in the world — tablets and keyboards and monitors and graphics cards — we don’t generally experience that. They’re related, but they’re not exactly the same. My sense is that we’re targeting a completely different audience. We’re targeting a completely different value proposition and a completely different business model. It’s not designed to be a game console. It’s a gaming device.

GamesBeat: I just came from my press Q&A with Qualcomm CEO Paul Jacobs, and he said he’s going to beat you guys on graphics.

Huang: [chuckles] Oh, is that right? Uh-huh.

GamesBeat: He can beat Tegra 4, he says.

Huang: Pretty brash words. We’ll see, I guess.