Microsoft games exec details how Project Natal was born

Microsoft threw one of the biggest curve balls at the E3 video game conference on Monday as it announced a new way to control games through your speech, gestures, and your full body, code-named Project Natal. You can control an Xbox 360 video game through your own motion. It goes farther than the Nintendo Wii because it uses a 3-D depth camera to capture not just the movements of a handheld controller but the movement of your whole body. In a one-on-one interview, I asked Shane Kim, corporate vice president for strategy and business development at Microsoft’s game division, to explain how Microsoft developed the technology. I also got an update on all of Microsoft’s game plans.

VentureBeat: How would you score the E3 press conferences of the competing console makers?

Shane Kim: I think we won. It’s not just me saying that. It’s pretty universal. We had something very special yesterday, starting with the games portfolio. We had a lot of content for a normal E3 briefing. But Project Natal doesn’t come along every year, and it is very difficult to match.

VB: Sony’s own gesture-control system uses a wand in your hand and a camera system. It’s very different from what you showed.

SK: That was interesting. Project Natal has nothing to do with Sony or Nintendo. It has everything to do with breaking down barriers and getting to the mass market, where controllers are barriers and they’re intimidating. It’s awkward for some people to learn to use a controller. This is really about unleashing a new category of controller-free gaming and entertainment. We didn’t want to do something that would be derivative of what Nintendo did with the Wii. That’s been a nice innovation for them but this goes much farther. We’re talking about full-body recognition. It can tell when someone is standing or sitting down. It can tell when something moves in front of you or behind you.

VB: And you can play this multiplayer?

SK: It’s multiplayer as well. This is not tracking points the way that the Wii controller does or Sony’s new controller does. It’s tracking your full body movement. That alone is very different. It’s also got voice recognition. It has a multi-array microphone so that it can recognize different voices in a room among the different players. It also recognizes faces.

VB: How long have you been working on this?

SK: A long time. It’s complex. At Microsoft Research, we’ve had a lot of work going on for a long time. We’ve done a lot of work in natural user interfaces. Voice recognition is one of them. That’s why we have been able to deliver development kits for it this week.

VB: What does Natal mean?

SK: Natal is a city in Brazil, which is where one of our key development team members if from. Natal also means birth. We thought it was appropriate for what we are doing. This is the birth of the next-generation of home entertainment.

VB: How much more are you saying about it?

SK: We aren’t talking about pricing, timing, or actual products yet.

VB: What technology does it use?

SK: It uses an RGB camera (image sensor), a 3-D depth camera (which determines how far away an object is from the camera), and a multi-array microphone.

VB: The depth cameras are a key technology?

SK: To me, the magic is more software. You’re talking about an extraordinary amount of data that has to be processed in real time. You saw the latency was very good yesterday. You can parse voices, recognize faces. It’s complex hardware, even more sophisticated software, and simplified for developers to use it immediately. All of that so that consumers can have the most simplistic and easy experience to get into gaming.

VB: There were leaks about which camera maker you were buying, 3DV Systems or Prime Sense.

SK: None of those rumors did justice to what we were actually doing with Project Natal. People expected a controller you put in your hands.

VB: Can a competitor get by with just an off-the-shelf 3-D depth camera and do the same thing?

SK: The companies are doing a lot of great work with the cameras. But the magic is in the software. It’s a combination of partners and our own software.

VB: The timing for Sony’s product is the spring of 2010. What is yours?

SK: We are saying it’s not going to be 2009. We have put the technology in the developers’ hands. But the spring of 2010 is very aggressive. Sony has made other promises before. We’ll see if this one comes true. Not only do you have to get the technology right, you have to have third parties supporting it and great first-party software. We showed demos yesterday and we have a lot of work being done at Microsoft Game Studios that we haven’t shown yet.

VB: That leaves a lot of wiggle room for Nintendo to respond.

SK: I don’t expect them or Sony to respond. It’s about a controller-free gaming experience. Nintendo went with the Wii MotionPlus to increase the fidelity of what they do. It’s not going to be easy to compete with Project Natal.

VB: So Natal is a huge investment?

SK: It’s substantial. Xbox Live and Natal are fundamental parts of our identity going forward. We will lead in online and lead in natural user interfaces.

VB: Some people are very curious about the patents in the gesture control technology. Is there freedom to innovate here, or do you have to be very aware of the patents out there?

SK: You have to be very aware. We want to ensure that we have great intellectual property protection. You have to have a strong legal approach, and this is not easy stuff. It has to be all buttoned up, legally. We have had a very concerted focus on this.

VB: What’s the strategy for the game division in terms of what you will do inside the company and what will be done by third parties?

SK: Microsoft has always used first party (internal) developers to showcase technology and new features on the platform, like high-end graphics on the Xbox 360, Xbox Live, or Project Natal. We just acquired Big Park a few weeks ago. You can see their new game, Joy Ride. It’s going to be an innovative game on Xbox Live, taking advantage of the avatar system. It has a business model where it’s free to play and you can pay to upgrade your car. We are going to do with Project Natal the same thing we did before. We will take the leadership position.

VB: I’ve been writing a lot about the flood of free games hitting the market. Bobby Kotick (chief executive of Activision) remarked how good prices for games are holding up in the recession and the current cycle. Could it be that console games are ripe for someone to come in and undercut them?

SK: There are different segments of customers. There are the fans of Halo, Forza, and other titles that command a premium price. They deliver an amazing, professionally developed, full-blown experience that customers are willing to pay for. There are some customers who want more casual, bite-size experiences. They want a free-to-play model. You need the downloadable episode in Grand Theft Auto to extend the console game experience or a song you can download for Rock Band. You have a lot of different kinds of options. We enable all the business models.

VB: Does the recession make you want to re-prioritize?

SK: We still have the lowest-price console. No one announced price cuts. That has benefited us greatly. We are the only console up 30 percent for the first four months of 2009. The economy is affecting the industry, but we are a pocket of success. Since we launched Netflix and the New Xbox Experience, we have been adding half a million new members a month for Xbox Live.

VB: Do customers have less appetite for buying these games with add-ons like the guitars for $120 or more?

SK: Consumers are trying to be judicious about the money they spend on all things, including entertainment. But people want to be entertained. The box office is stronger than a long time. If we offer good experiences, people are willing to pay for it. We have enduring value. It’s not like a two-hour movie. We have replayability and many more hours of entertainment.

VB: Can you think about postponing the next-generation console launch if you introduce something big in mid-cycle like Project Natal?

SK: We firmly believe that the Xbox 360 has a life cycle through 2015 (10 years after the launch). Project Natal is a great innovation. It will work with every Xbox 360 sold. It’s not about pushing more pixels on the screen. It’s about how to break down barriers that stop people from playing games.

VB: The console is becoming a good home entertainment system, beyond games. But Sony talked about integrating Facebook today.

SK: They are limited to doing it on the PSP. No one is bringing social networking to life the way we are. We are starting from a base where we have a real social network on Xbox Live. That number keeps growing quickly. We are building off a base of active members, Netflix, Facebook, Twitter, instant-on 1080p. We are building things on top of Xbox Live that are tough to compete with.

VB: So it’s getting you new customers?

SK: No question. The number of new households using it with multiple sign-on is growing. My wife uses it a lot more because of Netflix. This is a significant piece of our profits and overall business model.

VB: Do you still have holes to fill in the strategy?

SK: We have more work to do making it a next-generation social network. We want to get more content to offer. We are building a strong foundation. Sky Broadcasting is delivering live TV over Xbox Live. That’s very compelling.

VB: What’s the size of the game team?

SK: It’s a big team.

VB: You cut some studios like Flight Simulator and Ensemble Studios.

SK: We also acquired Big Park Studios. I don’t know how constant the ebb and flow will be. It’s not unidirectional. We made some hard decisions in the past. We made good decisions. We are still committed to first party. The Big Park acquisition should demonstrate that to folks. We were positioning for things like Natal and I don’t think the talent at Ensemble or Flight Simulator were necessarily the right studios for that.

VB: And you gave up on massively multiplayer online games a long time ago?

SK: There are lots of things we can do without chasing the MMO space. If you look at Big Park’s Joy Ride, it’s an online virtual world that is like a casual MMO. It’s an advanced, casual game. So is One vs. 100. It doesn’t have to be World of Warcraft.

VB: How do you manage the number of games? The iPhone has more than 10,000 games.

SK: The difference in cost between creating an Xbox 360 game and an iPhone game is pretty significant. You aren’t going to see 10,000 games with us. We want a wide breadth of games that appeal to a wide group of people. We have more than 1,000 games on the Xbox 360. But we do have to stand for quality as well. Finding the content on the AppStore is not very easy. We want to manage a quality experience, and that’s part of why we redesigned the interface for the Xbox 360 dashboard as the New Xbox Experience.

VB: So you managed to get another big game coming in 2010 from Bungie, Halo Reach.

SK: The Bungie guys love Halo. There has never been any question about that. They want to keep telling the Halo story. Our partnership works well. Our hardcore fans will have a great time with Halo ODST this year. And Halo Reach comes next year.

VB: Will Halo Reach follow the story of the first novel? (The Fall of Reach).

SK: You’ll have to wait and see. There is an opportunity to tell the prequel to Halo. People are interested now they see how the trilogy of Halo ended. Now they can see how it began. The Halo universe is very rich and there are a lot of stories to tell.

[Photo credit: www.joystiq.com]