Oculus cofounder describes CES demo — and what comes next for virtual reality (interview)

Nate Mitchell has been at Oculus VR from its humble beginnings as a virtual reality goggles startup to its $2 billion acquisition by Facebook last year.

A year ago, he came to the 2014 International CES with Oculus Rift creator Palmer Luckey to show the Crystal Cove demo with the development kit 2 (DK2) in a tiny room. This year, Mitchell and company returned with a giant booth where the public could view the latest Crescent Bay demo in huge numbers.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":1639565,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,mobile,","session":"D"}']

We caught up with Mitchell at the 2015 International CES, the big tech trade show in Las Vegas last week after viewing the Crescent Bay demo, which transported me to another world and let me turn around in 360 degrees to view it. Mitchell spelled out the details of Crescent Bay and said there’s a team of 250 engineers working on a consumer version of the Oculus Rift headset and a full system for using it in games and other apps. Mitchell said there is a chance that the consumer version will ship in 2015, but he made no promises.

Here’s an edited transcript of our conversation, which was jointly conducted with writer Chris Morris.

GamesBeat: What’s new at the show for Oculus?

Nate Mitchell: We’ve taken an early build of the Oculus audio software development kit (SDK) and gone back to all of the Oculus Connect demos and added spatial 3D audio. You’re going to be able to — hopefully, though this is still early — pinpoint sounds above, below, and out in front of you in full 3D. That’s pretty awesome. Audio is this major amplifier for VR presence. That’s what we’re chasing on the PC side, presence and comfort. It makes a big difference.

We’re pushing through the Crescent Bay demo set here. We’ll walk you through the same demos we showed at Oculus Connect, give or take. We’ve revamped and polished a few things, added a few things. It’s technically not the same, but relatively speaking, it’s the same. The first set of demos was developed in-house by one of our studios in Seattle. We went back and enhanced it, revamped it, added spatialized audio, improved presence. You’re seeing a director’s cut or something like that. There are a couple of new experiences in there too. If you saw it at Oculus Connect, you’d see some new stuff. That’s what we were going for.

The other big thing we’re talking about is Samsung Gear VR. Gear VR Innovator Edition launched in December. We’re excited to get that out to developers and enthusiasts. We don’t have any major announcements about Gear VR other than we’re working on commerce and internationalization for early this year. The plan is to roll out Gear VR in additional countries soon, and then follow that up with commerce in the U.S., followed by commerce rollout to additional countries as they come online. That’s the big push right now internally. We have a lot of developers holding content back waiting for commerce to come online, because they don’t want to give away their games, movies, and experiences for free. That’s a big undertaking for us. It’s something we haven’t done before. We’re close. We’re getting very close.

GamesBeat: I saw a couple of new games coming out, like Temple Run. Some of them are still going forward.

Mitchell: There’s a mix. Some people just want to be there early. Some games would be free otherwise, and they’ll add in-app purchases later. Some games are releasing demos. And then some people are saying, “No, we’ll just wait. Tell us when commerce is ready and we’ll put it out there.”

[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":1639565,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,mobile,","session":"D"}']

It’s one reason we call it Innovator Edition. We don’t have that story yet. The software experience is still coming together in terms of UI and UX for moving between content and discovering new content. It’s getting a lot better. We’re continuing to iterate pretty rapidly.

GamesBeat: Do you have a timetable for that?

Mitchell: We haven’t announced anything. It’s early this year. It’s not December.

GamesBeat: What about whatever comes after Crescent Bay?

[aditude-amp id="medium2" targeting='{"env":"staging","page_type":"article","post_id":1639565,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,mobile,","session":"D"}']

Mitchell: We haven’t announced anything yet. What comes after Crescent Bay is most likely going to be the consumer Rift. We are really all hands-on with that, working on shipping as soon as we can.

We get a lot of questions. Is Crescent Bay going to be a dev kit? Why can’t we buy one? We don’t want to derail the team and go down a path where we’re trying to productize Crescent Bay, because that takes us away from shipping the consumer version. We want to get the consumer version out as soon as we can. We’re working on it.

GamesBeat: What have you been doing as far as acquisitions and other parts of the system?

Mitchell: We just acquired Nimble VR and 13th Lab. They are two computer vision companies doing unique work. I can’t go into too much about what they’re working on. One of the things we’ve been transparent about is that it’s pretty likely that their technology won’t make it in for consumer version 1 (CV1). It could never be used even for CV2 or CV3. But we have brought them on to do some stuff, both in Oculus research and in product engineering. Hopefully we’ll see some fruits of their labors in the coming months and years.

[aditude-amp id="medium3" targeting='{"env":"staging","page_type":"article","post_id":1639565,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,mobile,","session":"D"}']

GamesBeat: You’re still not showing any input systems, though.

Mitchell: That’s one of the big missing pieces, yeah. One of the things we talked about at Oculus Connect is that input needs to be there for great consumer VR. It’s still the big missing component, the elephant in the room. We want to deliver great input solutions for CV1. We think that’s important, especially for it to be a real consumer device. We haven’t made any announcements quite yet, but it’s something we’re working on.

GamesBeat: That Unreal 4 Showdown demo in Crescent Bay looked good, even if it wasn’t interactive.

Mitchell: In our defense, it’s the tiniest bit interactive. The aliens will track you, for example. They respond to your head position. That’s pretty neat. The T-Rex looks at you and moves a little bit. But you’re right.

[aditude-amp id="medium4" targeting='{"env":"staging","page_type":"article","post_id":1639565,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,mobile,","session":"D"}']

GamesBeat: Tell us where things stand for you these days. It’s been a while since we last caught up with the company. Where are you in the production cycle? Are you getting close to a point where you can talk about a consumer date?

Mitchell: We’re always getting closer. [Laughs] One thing we talked about at Oculus Connect is that we’re aligning on the consumer specifications. What you see with Crescent Bay is that we’re getting–There’s a number of components to Crescent Bay that we’re happy with, that we think can go in the consumer version. Yesterday someone asked if we think the resolution and screen technology in Crescent Bay is good enough for the consumer Rift. I do think it is. Crescent Bay as a holistic system crosses that threshold where you’re delivering comfort and presence that is that magical, holy grail of VR. It is going to get better.

One of the things we’ve said — it’s always as good as what you see. It’s not better. But now that you have 360-degree tracking, a wide tracking volume, integrated audio, substantially better ergonomics, we’re getting close. The ergonomics are much, much better than version two. Like any feature prototype, part of it is experimentation, production experimentation. Do we like the strap system? How well does that hold up? Putting thousands of people through it at a show is a good way to test our assumptions. We’re always testing internally. It’s good to run it through its paces.

GamesBeat: As you’re starting to see sales coming in for the Gear VR, and some of your competitors are starting to roll out, are you starting to get a sense of where things will be with price? Or is that something that you’ve known for a while?

[aditude-amp id="medium5" targeting='{"env":"staging","page_type":"article","post_id":1639565,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,mobile,","session":"D"}']

Mitchell: We’ve known the price for a while. We’re still targeting that $300-$400 price point. DK2 was $350. We want to be in that same ballpark for the consumer Rift. I can say that it’s challenging, especially when you start to talk about — OK, what does input look like? How does that integrate into the system?

A good consumer VR system is going to cost — look at Gear VR. All up, it’s about $1,000 if you go for it without a phone carrier subsidy, and a $200 piece of hardware. With Rift, these are expensive units — DK2 is $350 — but for a high-end PC add $800, $900. You’re even higher than Gear VR. That’s the ballpark we’re going to be in for these first units, whether it’s mobile or PC. You’ll be above $1,000 all up for a great system, a great experience. The headset itself will be $300 or $400.

GamesBeat: Are you close to locking system specs down?

Mitchell: In terms of the mid-spec machine, I was talking about that last night. We’re not close to locking them down. We’re close in the sense that we have a pretty good idea of what we think the system specs should be. But we haven’t gone out to developers and said, “Here are the min specs we’re targeting for CV1.” That’s a good benchmark, a good inflection point. When we’re telling developers, that’s when it’s locked down. We’re not quite there.

But with the Crescent Bay demos, these are running on high-end Nvidia 980s. You can expect it to be in that ballpark to start and we’ll see where it goes from there. It could be a bit lower or higher. But if you extrapolate out on the Nvidia line, we might be in that same range. We try to be up front about what systems we’re running on today. Pretty much every show we’ve shown up with the highest-end graphics card we can find, because we’re pushing so many frames through it. 90Hz at that resolution in stereo is hard.

GamesBeat: How is your content pipeline coming along at this point?

Mitchell: It’s coming along well. It’s one of the biggest challenges we have. You have this problem where the audience doesn’t have a good sense of the content and the content has no audience. It’s something we continue to invest in. Jason Rubin we brought around in the E3 timeframe to lead our studios effort. He’s doing a lot of internal studio development and external publishing deals.

Gear VR is a great litmus test. You buy Gear VR and drop into the Oculus store, you can see there’s a diverse array of content. We have some big names in there, some indies in there. We have different types of experiences. It’s not all just action and 360-degree experiences.

There’s a huge amount of content waiting for commerce, as I say. That’s one of the things missing from Innovator Edition at launch. All the content on Gear VR is free. On PC and on mobile, aligned with all that, developers need a way to distribute their content, make money, and be successful. We’re committed to that. We want Oculus to be the best VR platform around. As that stuff comes online, you’ll see more and more companies coming into the space. Gear VR has done well. It’s sold out. We’re moving units there. As CV1 comes online, you’ll see more and more developers jump in.

GamesBeat: Do you have hard numbers for Gear VR?

Mitchell: No. Samsung products are Samsung numbers. Samsung manufactures it, so it’s their product.

GamesBeat: Will you support Razer’s announcement of Open-Source Virtual Reality?

Mitchell: We are not supporting that. That’s not an official stance. The first I heard about it is yesterday morning. I think I’d seen OSVR previously, but I didn’t know anything about Razer announcing a hacker development kit. It was all news to me. I sent it to Palmer and said, “Hey, have you seen this?” “Nope.”

Overall, in terms of the specification, that’s the right way to go long term. In the short term, you need multiple VR headsets out there that are mature as far as feature set before that kind of thing is really warranted. To give an example that’s resonated in the past, we had different SDKs for mobile and PC internally. We don’t use the same specifications for our own systems internally. That’s because they’re so different. If you talk to Palmer, he’ll say that mobile is more like Vita, PC is more like PlayStation. We don’t have the same SDK for both. They’re different systems with different challenges and different features.

Rather than have anyone slow down and try to align inside of Oculus, we have to go. We have to innovate. We want to keep moving. Spending a bunch of time trying to align–That’s true across the entire industry. I don’t know how they’re going to build an open specification for so many different input devices either. It seems like a hornet’s nest to me.

In the long term, when the time is right for a specification, we’ll be there. But we’re still far away. We need a couple great headsets, a couple great input devices out there. You want to see Morpheus and the Rift and Gear VR come out. You want them to mature. As they all start to align over the next few years, that’s when a specification makes sense. In the meantime developers are targeting the lowest common denominator. They’re not leveraging all the features of the high-end headsets. It becomes a bad experience for everyone. It’ll end up poisoning the VR well for people. That’s not to say OSVR is all those things. But we want developers targeting our SDK — not to own the platform, but to leverage everything and do the integration right. That’s the only way you’ll be able to develop a great experience.

GamesBeat: As other headsets are hitting the market these days from other companies, does that start to dilute the market? You were and are the big name in VR, but now smaller players are rushing to market and getting out there first.

Mitchell: All the people rushing to market first does — it can poison the well. No one has poisoned it yet. I haven’t seen the OSVR hacker development kit. I have seen plenty of bad VR headsets. What we’ve shown is that it’s incredibly hard to deliver a great experience. We’ve been working on Gear VR for more than a year with John Carmack, one of the best engineers of all time. We have a huge team of people working on Gear. On PC we have a huge team working there too. We have 250 of the best VR engineers in the world eating, sleeping, breathing, creating VR. We’ve been iterating on the Rift for more than two and a half years and Gear VR for more than a year. So I think we’ve proven that it’s hard to do this right. We’re committed to nailing it.

What everyone else is showing is that it’s easy to do poorly. It’s easy to build a simple headset. I’ll say, Cardboard is not done poorly, but Cardboard shows how easy it is to build a simple VR headset. You take a piece of cardboard, drop a phone in, you have a VR headset. Is it the experience we want to deliver? No. It’s a cool toy. I think Google will do some neat stuff with Cardboard now that they’re doing an SDK and stuff like that.

Some of the other groups — “Oh, we have a VR headset now! High-resolution display! Works with Xbox!” — you should be wary. I’m hoping it doesn’t poison the well overall. Hopefully we’ll get a chance to try more of these headsets here at CES, because–I didn’t know what to expect, but it really is the year of VR here at CES. People are joking that it’s like 3D TVs a few years ago. VR is everywhere.

GamesBeat: I’m not sure that’s a comparison you want.

Mitchell: Exactly. We’re very up front and transparent. We don’t think we’re quite there, in Innovator Edition or Crescent Bay. We have a ways to go before it’s consumer-ready.

GamesBeat: With Crescent Bay, are there things people pointed out as shortcomings? There’s a bit of the screen-door effect, still.

Mitchell: Definitely. We are transparent about all that stuff. The headset is fragile. We’ve had some audio components break off. The tracking volume could be better. The tracking could be more precise. I could tear it apart because I know what we’re working on for the consumer Rift. But overall it’s not perfect. That’s why it’s a prototype.

Another testament to that whole situation is that Sony still hasn’t announced whether Morpheus is going to ship. I assume that’s because they’re working on it. Just like us, they know consumer VR is not quite there. We’re getting close, but we’re not quite there. That’s why you’re seeing some of the people who’ve been in it the longest still holding back and not rushing to market.

GamesBeat: Is there a chance that we will see a consumer unit by the end of the year, or is that too aggressive a schedule?

Mitchell: There is a chance. But that’s not to confirm anything. The general life cycle for product development on a normal hardware product can be something like — You can turn a product around in 90 days. We don’t turn a product that quickly. We’re a startup that struggles sometimes. We’re still learning. VR is still new.

Assuming we knew what it wanted to look like, though, we could have something built out in China in 90 days. You can get these things rolling off the line in 90 days, but then you want to build stock. Then you’re producing them for a month, two months. Then you ship for a month or two over freight. Overall you might have six or eight months of getting a lot of units somewhere.

GamesBeat: Are you setting up manufacturing? What you have for the DK now is certainly not going to be enough when you go live.

Mitchell: We’ve been setting up manufacturing for a long time. DK1 was our first foray with a CM. With DK2 we ramped in a big way and did something a lot more complex. Since then we’ve been doing a lot on the manufacturing side to set up for the consumer run.

It’s fascinating, because I come from a software background. The way you can leverage multiple manufacturers for different components to be competitive and to be backstops for each other — If you have one CM where Apple shows up and says, “Here’s a bag of money, build for us,” we have another CM in reserve, in ramp. We’ve been setting all of this up since the beginning of the company, really. We’re in a great spot. A lot of that is the talent we’ve brought on here. We brought on Raoul who was at Java and Monster. We brought on Adrian Wong who was at Google Glass. These guys have been setting up an awesome NPI and manufacturing program for us to be ready when it’s time to ship the consumer Rift, or start going mass production. We’re going to be ready. We can’t be flat-footed on that type of thing.

GamesBeat: What’s the pattern you see in the latest things people are working on? I see a lot of these 360-degree videos.

Mitchell: It’s this blue-sky mixed bag. We’re seeing people do 360 video a lot. Palmer was on a panel yesterday with the CEO of Jaunt and some of the other filmmakers messing with 360 video. 360 video is still super hard. It’s not something we’ve invested a huge amount into, aside from building our own applications for playback. We’re seeing a lot of game development still. It’s awesome to see all the indies in the space dive in. We’re seeing bigger names drop in, especially with Gear VR. Uber Entertainment and companies like that.

Every morning we’re always talking about the new weird demo, whatever it is — some students at a hackathon building a game where you defuse a bomb in the Rift, or a high-end game on Gear VR that we’re testing for the first time to go into the app store. There’s always new content. It’s always cool to see what people are doing.

That’s one of the best things about VR in general. It’s a new medium. Everything is new. Just like the Crescent Bay you went through, we’re trying to showcase a couple of the types of experiences you can have. Some are more cinematic. Some are more about storytelling. Some are more like a SimCity-style game. Some have you dealing with an NPC in a space. We’re trying to show a diverse set of the things you can do in VR, and they’re all new. Again, that’s one of the cool things about VR. It’s fresh. We don’t know exactly what the killer app is going to be. I do think gaming is hopefully going to be the killer app, because I’m a big gamer.

GamesBeat: What do you think about the different variations on VR? Things like sitting at a desk or standing up and moving all over the place.

Mitchell: We’re still targeting a seated experience for CV1. This demo is standing up to demonstrate presence. When you stand up it’s an amplifier for presence, much like audio or low persistence. When you stand up and you’re free in the space, for whatever reason, your brain–I think it’s because rather than being rooted in this chair, feeling stable and safe, when you stand up your brain is able to suspend disbelief a lot more quickly, believing that you’ve been teleported to someplace entirely new. In this demo we want to show how powerful presence can be, so we’ll have you standing.

For the consumer Rift we want people sitting down. There are all sorts of liability concerns. We are putting you in a different world, so the last thing we want you to do is not only be blindfolded, but also running through an open field in VR and crashing into walls. It’s a recipe for disaster. People keep asking what people are going to do there. They’ll do their own thing and set up their own space. I think people are going to do all kinds of stuff. People do crazy stuff with their own computers all the time.

One other simple thing, from a product and user experience standpoint–You look at something like Kinect or Wii, we saw that it’s hard to find enough space in your home to have an experience like that. People were like, “Oh, let’s play Wii” and they’d start moving furniture around and pushing the TV back. We don’t want VR to have to have a space like that. We want you to be able to experience VR at your desk. Almost every home has a computer. Mount the camera, be comfortable, be safe, and have a great experience. Getting that right is the path to great consumer VR. We’ll see where we go from there.

GamesBeat: Does that work against you? Most of the demos I’ve had had me standing.

Mitchell: It works against us, but only a little bit. You don’t necessarily want to stand up all the time when you’re going through a long-play VR experience. When you drop into Elite Dangerous, sitting and playing Elite Dangerous is a great experience. People are coming in and saying they play it four hours every day. You don’t want to be standing or walking around two to four hours while you play a game like that. There’s going to be experiences where you do want to be like that, but those are going to be fewer, I think. When you start to get into a standing experience, it gets more niche, especially early on. Long term, who knows? We may all have rooms like this in our house. We might do this interview in VR, 10 years from now. But we want to start with the seated experience.

GamesBeat: Do you think you can get rid of the wire?

Mitchell: You can, but it’s expensive. There are latency challenges. It’s not quite there. We’re pushing so much bandwidth over the wire that it’s not practical to send it over wireless today.

GamesBeat: You’re at 90 Hz right now. Is 120 Hz the target for later on?

Mitchell: We’d like to be higher. 90 is a good threshold for consumer. This runs at 60 Hz. Some people do perceive flicker. It’s an issue. With DK2, we were at 76 Hz, which is a good threshold. A lot of people can’t perceive any flicker at 76. When you get to 90, that’s when it starts to be — Almost no one can perceive it. We’d love to be at 120. 90 is a good sweet spot in terms of bandwidth over the wire. If we jumped up to 120, we would need a faster GPU. That’s one of the fastest GPUs on the market that’s not supercomputer craziness. That’s a normal, “I’m a gamer, I want the fastest GPU.”

GamesBeat: Do you see bundling with video cards when you come out, or will you do a stand-alone product?

Mitchell: I’d love to see an experience where you can buy the Rift along with some sort of machine specification that you know is going to work well. That’s something we talk about a lot internally. That’s something you and I were alluding to before. I got an email this morning from someone who said, “My son’s interested in building games for the Rift. What system should he buy to develop?” It’s a question we get all the time and there’s not a good answer. That’s something we have to overcome for the consumer version.

Whether that’s just, “Hey, you need this graphics card” and some people will bundle it — you go to Newegg to get the Rift and the recommended graphics card — or if we have a partner or multiple partners who sell an Oculus system, there are so many ways to tackle it. We don’t have an official answer yet, but it’s something we’re looking at. We want to make it easy to get into VR.

GamesBeat: What are you showing in the booth? Is it the same as what you have here?

Mitchell: Yeah, the same thing. VR channel surfing. It’s a big booth, isn’t it? It’s the public debut of Crescent Bay. We wanted to go big at CES and give people a chance to see it, because so few people got to see it at Oculus Connect. A thousand developers that we hand-picked does not make for a big splash.

GamesBeat: Do you feel like 3D audio is done at this point, or do you still have different variations of it, different things to improve?

Mitchell: Definitely not done. This is an early build of 3D audio. I don’t know how well the spatialization worked for you guys, but audio is a big amplifier for presence. We’ve been working on the audio SDK. It’s early days. The spatialization does make a big difference. When you hear that pipe burst above you or hear the bird chirping and you can turn your head while the sound stays perfectly spatialized, that has to be there. It’s relatively important for that to be there for a powerful sense of presence.

GamesBeat: This one has it all attached. Is that the preferred method for audio, or could you make this work with a third-party headset?

Mitchell: You can use a third-party headset. We want to provide the integrated audio solution, sort of like the computer situation we were talking about. You buy the headset, put it on, and you have a great experience. The other nice thing about integrated audio is that, from a sound design perspective, it’s really powerful to know, as a sound engineer, what headphones the audience is going to be using. When you’re a sound designer for games right now, it’s kind of a nightmare. They’re probably going to be playing it out of laptop speakers, so who cares about audio? But once you know they’re going to be using this type of headphones, it’s awesome, the tuning and magic you can do.

All that said, people are zealots about their favorite audio. We all probably have different favorite brands. Those are removable — not on Crescent Bay, technically, but for the consumer Rift we want to make them so they can be removed and you can plug in your own headphones. It’s a great driver that we’re using in Crescent Bay. We’re not saying which one it is, but it’s still a relatively cheaper driver. If you have some Sennheisers or Bose headphones they’ll be higher quality. We want you to be able to use your own headphones. We’re not trying to force audio on anyone.