Magic Leap’s Graeme Devine on how to design for mixed reality and the impossible

Above: Graeme Devine wants to make super-compelling mixed reality applications.

Image Credit: Michael O'Donnell/VentureBeat

GamesBeat: How would you see the difference between AR and VR?

Devine: VR takes you somewhere else. It transports me and puts me in some other place. AR presents information on top of the world. Mixed reality adds to the world, adds photons to the world.

Why would I want photons added to the world? What are the applications that I actually want to add on to the world? I think of a five-mile application. A five-mile application is, I’m five miles away from my house. I’ve forgotten my smartphone. Do I go back to my house and get my smartphone? Yeah, I’ll do that, because my smartphone makes me better at what I do. There’s games on it.

What if I’m five miles away from my house and I’ve forgotten my smartwatch? Do I go back and get it? Well, not right now. There’s no game I want to play. There’s no compelling application I need. I’m 50 feet away from my house, will I go back get it? Nope. So what is our game, our application that we have in mixed reality that makes you turn around after five miles? That is the key application we need to be thinking about. That’s why you wind up at the store for four days in the rain. We’re thinking a lot about that application, that experience is so compelling that you’ll turn around for it.

GamesBeat: Tim Sweeney had a very interesting comment about augmented reality. He said that when it’s perfected in the next 10 years, we won’t need displays anymore. No smartphones, no TVs. You’ll have your 40-foot screen right in front of your eyes.

Devine: We talk a lot about that, about a world without atoms. Unless you actually need those atoms, we can make photons work. That sets off a weird round of, well, what needs to be real in my world? My coffee, my coffee cup. Other things – displays, my work, the people in my environment, the experiences I have in the evening, the entertainment I watch, the sports I engage with, the content I go through – I don’t generally interact with the atoms of them. We’ll be changing how you perceive reality.

GamesBeat: Mixed reality versus VR — it almost seems like it’s easier to make games in VR. Maybe you’re more used to or familiar with it. Augmented reality seems like it’d be useful for a lot of non-game applications. Do you foresee that kind of world coming, where maybe AR is more useful for non-game purposes?

Devine: For a while it was very hard to think about what a game could actually be in augmented reality. We started with doing games on tabletops, placing things on tabletops and making the game there. That’s awesome, because now there are games I really like on tabletops. Dungeons & Dragons is cool and fantastic, all those things.

Then we moved on to putting games in your house. You start playing a game using wooden blocks. You spell things with letters on them and look at those words and learn how to actually interact with the world. Then things move in your house. You start to see lights and hear sounds going off in a room. Eventually you go and look in that room and see what’s there. In front of you is a ghost, a ghost standing in your house, in the middle of your room. It points directly at you and directly beyond you. You look behind yourself and there’s a body on the floor. Then you look back and the ghost is gone, but in your ear you hear, “Please help me.”

What you realize is, “I’m having a story in my house. What I’m going to be talking about at work tomorrow is all the haunting going on in my own environment. My gosh, I’m not going to be able to sleep at night. This thing might still be there.” That kind of experience is something that can’t be had in VR, because VR takes you somewhere else. We’re giving you experiences that happen inside environments you’re already very familiar with. That’s incredible. It’s not been done before. That kind of design and narrative is incredibly difficult. It’s the hardest thing I’ve ever worked on, but it is so awesome to work on that problem.

Above: Graeme Devine, the chief creative officer at Magic Leap, at GamesBeat 2015.

Image Credit: Michael O'Donnell/VentureBeat

GamesBeat: Who has the hard job when it comes to making something convincing – the mixed-reality game designer or the virtual-reality game designer?

Devine: We both have very hard jobs. I’m already focused on my space. I’m focused on making things appear in rooms and making that compelling. I can’t wait to drive spaceships around in VR, but it’s not something I’m thinking about. I don’t think about that problem space.

GamesBeat: Have you figured out the part about how everyone has a different house?

Devine: Learning to see what is in your house, getting the idea of where spaces are at on the floor, where the walls are at, where objects are—The idea of the ghost is to help you with that. You’ll talk to artificial intelligence people and they’ll say, “We can work it all out. We can work out where everything is at in the house.” But I can also just have a game character ask, “Hey, where is a good place to put this game? Is that a floor? Is that the best wall to put the screen on?”

People love to interact with characters. They love to interact. We don’t need to know that much about the house because we can have someone virtually asking you there, “Hey, is this the right thing to do?”

GamesBeat: How much time for experimentation do we have here? There are different lines. CastAR is a different version of mixed reality. People are going to be experimenting with different platforms. How much time do we have for experimentation before companies start sending out commercial products?

Devine: We think a lot about three things that we do at Magic Leap. We have a team that just looks at Lego bricks, looks at small experiences. What’s music like? What’s it like to touch the world? What’s it like to go and do these small things in Lego bricks, so that we can learn what games are like in mixed reality? We think about that a lot. Those Lego bricks will be helpful to everyone.

Then there’s something we call the incubator lab. It’s in the final week of our internal process of asking everyone across the company, “What is the most awesome thing you could do in mixed reality? Tell us what it is and you’ll have a chance to go prototype it.” We’ve gathered 250 ideas from across the company. 22 of them we asked to go and make better presentations. Eight of them were put together with a team to go and prototype them. The incubator lab is something we work with across the whole company, because we have rocket scientists and Neal Stephensons.

Then we look at what is a large triple-A game in what we’re doing. We take a chance. We try to make something. It might be wrong. It might be bad. But we have a pipeline, moving from the art department to the animation department to programming and so on. What is it like to run on a device that moves around? What’s it like to place something on the table? We think about experimenting in that group quite a bit.

GamesBeat: I’m sure everyone in the audience is thinking this, so I’ll ask: What are you doing up until February, and when is your product coming?

Devine: We’re working day and night, I’ll tell you that. We work around the clock. I’ve never seen a harder-working set of people on the planet. All I can tell you is, it’s very exciting.