The tantalizing promise of augmented reality games

Ellsworth: Pokemon Go is an interesting moment in time, where it shows this gap we have in entertainment that needed to be filled. I saw families getting together and playing this. Couples were walking around the street looking for Pokemon. It’s amazing, the technology we have that can unlock that. All these various sensors that have been commoditized by cell phones are enabling that. Even five years ago we didn’t have enough technology to do it. It’s all here now and we can start to have these experiences.

GB: With toys, we’re seeing things like Lego AR. You can snap the QR code on the box with your camera and then you can see the characters tied to that toy come to life on your screen.

Beliaeff: That’s another widespread but not necessarily labeled use of AR, in advertising. If you look at companies like Vuforia, which is an AR middleware provider, their top clients are all in advertising. It’s becoming more mainstream. People are recognizing what it is and they can put one and one together.

If you look at what’s going on in phones, we’re soon going to have stereoscopic cameras in phones and things like that. That’s also stuff that’s going to help AR move forward. Once you can start adding depth, you’re not going to need markers anymore.

Ellsworth: Having a more seamless experience is going to be essential for the adoption of AR. We’ll start seeing more platforms doing it in a smoother way. Instead of having to install applications to do this, it’ll be part of the OS and part of the hardware itself. You don’t have to disrupt your normal day to have an AR experience. It’ll just happen automatically.

Above: VR Fund’s AR industry landscape.

Image Credit: VR Fund/Tipatat Chennavasin

GB: When it comes to games, it seems like the mad rush has been straight into VR instead of AR. Jeri, at Valve they did have that sort of choice point, and they chose to do Steam VR and get behind the HTC Vive. Your project was spun out, basically. Why did you choose AR as something to do with gaming?

Ellsworth: VR is amazing. I absolutely love it. But usually the experiences I have with VR — I pull out my Vive, set it all up, move all the furniture out of the way in the living room, move my main computer over, and then I have the experience for 20 minutes. Then I take it all down and pack it away and wait a month for the next cool experience.

The thing that excites me about AR is that it’s going to be a daily experience. When all of the technology is there so we don’t have to disrupt our lives to experience it, then it’s going to cover a much broader demographic than just hardcore gamers like me.

Beliaeff: If you look at the evolution of video games, they’re trying to control everything a person is experiencing. They want you to look at the television or the PC monitor in front of you and completely immerse you. VR is the next generation, where it’s giving you that room-scale experience, but still totally taking over and wanting to dominate what it is you’re taking in.

AR allows you to mix reality, where you blend what’s going on in the room or wherever you are, plus now there’s all this stuff that wasn’t there. That’s one of the things Pokemon Go did well, that experiential part. If you saw those first pictures in Central Park, you got this experiential blend of the video game portion of it and the reality around you.

Ellsworth: That’s important. VR is inherently isolating. AR, you can have your friends and family around and share an experience that’s blended with your world. You have that comfortable safe place that they’re a part of.

GB: Magic Leap is a big presence in the AR discussion. They’ve probably raised half the money that’s gone into AR and VR in the past year, that one company alone. I wonder what the presence of that means for AR’s future. It seems like their goal is different from where you guys are going. They want to create imagery in your AR view that’s indistinguishable from real life.

Beliaeff: I’m always a big fan of investment. The rising tide is going to raise all ships. It’s why I’m a big fan of Pokemon, a big fan of everything Jeri and her team are doing. The more things raise awareness and popularize this technology, the better it’s going to be for all of us. It’s an emerging tech. A lot of is figuring out what’s going to work and what isn’t. As you look at any technology over time, there will be periods of consolidation. Players will drop out and new platforms will appear. But the great thing about AR in particular, when you’re thinking about it versus VR, is it doesn’t necessarily require an HMD and dedicated hardware. A lot of it can just work with a cell phone or a tablet.

You’re going to get a lot of innovation out of what they do, particularly as they get closer to publicizing what they do. Outside of the sort of people in this room and in the games press, I don’t think most people know what Magic Leap is. Their unveiling, hopefully, will open a lot of eyes and get people excited about what we’re doing.

Ellsworth: I totally agree with that. This market is going to be a constant evolution. To have that walk-around AR experience is going to be a difficult thing to do, and to do it all in one chunk, it’ll be hard to stick that landing. We’re trying to focus on a tabletop experience in the comfort of your home where you feel OK putting some glasses on. We’re figuring out what those experiences are going to be, not pulling out a crystal ball and deciding how people are going to interact out in the real world. Even looking at my house today, how do I fit a game into my house? There’s clutter on the floor. There’s an old beer bottle on the coffee table. There’s a lot to learn about how we can start to blend these different experiences into the world.

It has to come in stages. We’re staging our company to nail the table first, then lift people up from the table and let them move around their home. Eventually we’ll let them leave their home. But we’re focusing on things we know there’s a hunger for, like this communal aspect, making the table the center of your home again. Communication over distance is going to be very important for AR. Collaborating in a holographic space, having direct voice communication. All these things will be figured out in the next two to five years, and that will shape what walk-around AR is going to be like.

Above: ODG augmented reality smartglasses at CES 2017.

Image Credit: Dean Takahashi

GB: Why did you decide to focus on the tabletop at first?

Ellsworth: Again, it’s difficult to know how to handle all these different interactions. If your hardware is thousands of dollars, too, it’s going to be difficult to find mass-market adoption. By constraining ourselves to the tabletop experience, we can nail that. Everything you experience there is going to be great, and from there we can springboard to people moving around the room or interacting with stuff in the home. That gives us time for sensors and technology to shrink, and for customers to get used to wearing glasses. It’s evolution. Customers are going to rebel against new technology that they have to take a big step into.

GB: How have you seen the technology advance? There some interesting companies here, like ODG. We’re seeing AR glasses already that are around four ounces. The larger ones are six ounces. I can remember times when these things were well over a pound. How fast is the technology moving when it comes to weight and cost?

Beliaeff: It’s going very fast. When you talk about investment, certainly Magic Leap has garnered the most, but ODG just closed a big round. If you look at AR investment over the last two or three years, it’s more than a billion dollars in total. That’s what you’re seeing.

We got into AR seriously in 2014. We released our first project, the Mission Drone, last year, and now we have the second one, Nitro Boost, for this fall. There’s stuff we can do with a car that we couldn’t do with a drone, and it’s only been 12 months. The level of acceleration is far more than what we’re used to. You were talking about Moore’s Law before. Right now I think we’re exceeding that.

For the companies that are doing dedicated HMDs, the hardware is accelerating very fast. But then the software is also accelerating. When you get that combination of hardware and software acceleration, the innovation, the speed of tech advancing, is glorious.

GB: Mike Abrash, formerly of Valve, now at Oculus, their chief scientist, he made this interesting observation about Moore’s Law. Everyone always thinks that Moore’s Law is inevitable, that it’s going to happen no matter what. He says that’s not the case. When you break it down, it’s really the work of very talented people figuring out how to do something that otherwise would be impossible. Advancements don’t automatically happen. It seems sometimes that some companies can either over-promise or get into trouble because their expectation is that Moore’s Law will automatically happen without enough going into the engineering side.

Ellsworth: Having been in engineering for many years, you see these pressures build up in an industry that force change to happen. We’re at an interesting point right now where you can’t really add any more pixels to my iPhone and make it any better, for me at least. You can’t make the screen any curvier. The audio fidelity is already great.