Sony’s virtual reality chief aims to create magical Morpheus experiences

LOS ANGELES — Richard Marks is the director of Sony PlayStation’s Magic Labs research division, and he’s responsible for dreaming up Sony’s Project Morpheus virtual reality headset. At the Electronic Entertainment Expo this week, he talked with us 1-on-1 about the growing body of virtual reality applications.

Variety of content is going to be key to wowing the crowds — and competing rival Oculus VR on PCs — once Morpheus ships next year. And Marks said he is encouraged by the progress that Sony is making, both with hardware technology and the array of games and demos that are available for the headset.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":1753553,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,","session":"C"}']

At E3, Sony showed off demos such as a scene from The Getaway, a crime game where you have to drive an escape van and elude attacks on motorcycles and cars that are chasing you. It also had puzzle games, a precision shooter, and a group social experience dubbed The Playroom. We talked with Marks about all of them.

Here’s an edited transcript of our conversation.

 

GamesBeat: What advances have you made in the last year?

Richard Marks: Huge advances, especially in the display. The panel is 120 frames per second, OLED, low persistence, low latency. There’s additional tracking markers that make the tracking better. Wider field of view on the display, too, 100 degrees roughly for each eye.

GamesBeat: Does all the computing still happen in the PlayStation 4?

Marks: All the graphics computing does. Right now some of the 3D audio is done in a breakout box, but that’s part of the thing that makes it so the HDMI can go to the headset and the TV.

GamesBeat: It’ll be a wired solution?

[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":1753553,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,","session":"C"}']

Marks: Right. It’s a 1080p signal at 120 frames per second. That’s hard to send over wireless.

GamesBeat: You still use it with headphones?

Marks: Yeah. Currently there’s a headphone jack, so you can use any headphones you want.

GamesBeat: Is there a camera peripheral that still works with the Move controller?

[aditude-amp id="medium2" targeting='{"env":"staging","page_type":"article","post_id":1753553,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,","session":"C"}']

Marks: It requires the regular PlayStation camera for PS4 to work. It works with the existing Dual Shock 4 and Move controllers for tracking.

GamesBeat: What are you learning so far from the kinds of games that are coming in?

Marks: It’s neat to see such a wide spread of ideas. What’s surprising is how many different things are fun. There’s not just one thing you want to do. This is a game that you could almost see playing without VR, but it’s fun to play in VR. You can feel the 3D of it and get a new perspective. People like a lot of interaction and different kinds of interaction, whether it’s driving a tank or picking things up with their hands or shooting a gun in a first-person shooter.

GamesBeat: It’s kind of fun to just load a clip in the machine gun. I never had to think about that before.

[aditude-amp id="medium3" targeting='{"env":"staging","page_type":"article","post_id":1753553,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,","session":"C"}']

Marks: One thing that’s surprising to a lot of people is how much fun it is to do things with two hands separately. Picking something up and doing something with the other hand to it—humans are really good with that. If you have to paint with just one hand and then move your body around it, it’s really hard, but if you can hold the object in the other hand and paint, it’s much more effective. You can move the paintbrush or move the object quickly.

GamesBeat: It’s interesting science. What sort of science are you applying to this? The perception thing was, if you’re at 60 or 30 frames per second, you get dizzy, but when you take it up to 120 it gets better.

Marks: The 120 does make it so you can refresh the screen more often and change the content. That makes you believe the world is real. It also makes your system feel better about it. But really it’s the low persistence that’s key. The way the brain works is kind of crazy. It’s better to show something for a very short time, a couple of milliseconds, and then show black after that. Your brain will fill in the black time in between with what it thinks would be the right answer. If you show the same thing for a long time, like eight or 16 milliseconds, your brain knows it should have changed a bit and it gets upset. With 120, we can update more often to do that.

GamesBeat: The mismatch is the problem. If your brain expects something and then get something else, it’s confused and gets dizzy.

[aditude-amp id="medium4" targeting='{"env":"staging","page_type":"article","post_id":1753553,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,","session":"C"}']

Marks: We’ve found that 120 feels really good. A lot of people are doing 90. They’ve found that that’s high enough. But for us, we have most of our games on PS4 now at 60 frames per second. With the re-projection we can easily convert them to 120. Some games will do native 120. That’s the best. That feels awesome. But it feels quite good going from 60 to 120, having it convert for you.

GamesBeat: What remains to be done? Giving developers more time to complete games?

Marks: We’re going through the manufacturing process now, making it so we can mass-produce them. It’s pretty close to feature complete, I’d say. There’s not going to be a major change before we launch the product.

GamesBeat: How long are developers shooting for? I think people expect the experiences to be shorter than a 20-hour game.

[aditude-amp id="medium5" targeting='{"env":"staging","page_type":"article","post_id":1753553,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,games,","session":"C"}']

Marks: It’s funny, because just like all games, it’s different for each different kind of game you’d want to do. Some of the character-driven games are fun to do multiple times, because the characters in the game react differently. That’s pretty interesting to see. The predecessor to the game you tried, The Getaway, there’s another segment of that game we showed at GDC. They call it The Heist. You’re being interrogated by a guy. Depending on what you do he changes his behavior a bit. If you look at something, he’ll look at it and comment on it. Even if the experience itself isn’t super long, there’s a lot of replay value in VR.

It’s hard right now, when we’re only showing these very short experiences. It’s hard to get people to try VR, so we want to have that kind of thing here. But we’ve been in VR a lot longer than that as developers. Some people will make some long experiences.

GamesBeat: The gun peripheral, what do you take from that? Is that starting to prove more precise or more necessary for the shooters?

Marks: It’s using existing peripherals. They’ve made a prototype out of existing peripherals because those things are already supported for tracking. What we’ll do with the final product, we haven’t decided. We’re just evaluating the experience part of having a gun, what that feels like. It’s an evaluation step right now. It feels pretty good to have one of those.

GamesBeat: The horsepower required to do this, is the PS4 pretty efficient? Is it just fine for your purposes?

Marks: It’s a left-eye right-eye thing when you’re rendering for VR, but developers are pretty familiar with how to do that. The tech of it is well-understood already. People are surprised sometimes that the PS4 can generate such a good graphical quality for VR, but it’s because every single one is the same. The developer can target this platform and tune their graphics to just the right level to look good on PS4.

GamesBeat: If you have an only-in-VR experience, what do you think that could be more like? You can do a shooter without VR. You could a puzzle game like this without VR. What do you think is going to be something that can only be done in VR?

Marks: All of the genres that already exist translate into VR. But something like The Getaway, where you have a huge level of interaction with things around you, that’s hard to do any other way. We tried to do some of that with the Move on PS3, but doing it through the window of a TV set isn’t the right way to do that, when you want to feel like you’re grabbing things and picking things up.

We’ve done a lot of physics demos with VR – picking things up, moving them, stacking them. That feels great. There’s no other way to do that that’s as good as in VR.

GamesBeat: So 3D visualization, if you need to look from different points of view to do something or solve something?

Marks: The thing that’s different in VR—the sense of scale matters in VR. You see these huge things and they feel huge. Even if you’re just playing a space sim game, you get a feeling like the Battlestar’s huge. You don’t get that feeling looking at a television set.

The other big thing with VR is the space. Having something feel like it’s close by to you. Some of the character interaction—when you feel like somebody is next to you in VR, it’s a lot different. Multiplayer things—rigs is a multiplayer game over the network. We already have multiplayer games, of course, and you have audio chat, but you don’t really feel like a person’s there next to you when you do that in a regular game. In VR it feels like they’re there next to you. That’s a powerful feeling. The sense of immersion is the key with VR, that feeling like everything around you is more real.

As far as the second screen, we let developers choose. If they want, they can have a completely different image on the main TV. The player could see something different. We happen to have two TVs here, but really it would just be this and this that you’d have in a game. This is just for showing, to make people understand. The game developer can decide to have an asymmetric experience where people are playing with you. They might have a top view of what you’re doing.

GamesBeat: Eye-tracking, is there some of that in there?

Marks: No, in the current things we’re trying there is no eye tracking. When you point your head, turn your head to look at things, somebody may react in the game, or the game might react knowing where you point your head. We’ve done experiments with eye tracking and we’ll continue to look into that. It’s an interesting potential future option.

GamesBeat: Does AR look interesting to you in different ways?

Marks: AR is challenging. It’s pretty different. It’s hard to get as wide a field of view. You can’t do the same things you can do in VR. You don’t have control over the display as much. I’ve always been interested in AR personally. It’s a really cool area. But I don’t think we can offer the same level of experiences with that. That’s why VR is our solution right now.