Above: The first CastAR prototype created by Jeri Ellsworth while at Valve
GamesBeat: What was your background leading to this?
Ellsworth: I’m an electrical engineer. I’ve done a lot of toy designs and produced custom ASICs (application specific integrated circuits), which allows me to make really low-cost devices. That was attractive to them as far as hiring me. I could help them create products.
Johnson: My first professional video game was in April of 1990, so I’ve been doing programming, lead programming, technology programming, all sorts of stuff. At Valve I did the graphics technology for DOTA. I was one of the three people that started the Linux revolution at Valve.
GamesBeat: With SteamOS, or before that?
AI Weekly
The must-read newsletter for AI and Big Data industry written by Khari Johnson, Kyle Wiggers, and Seth Colaner.
Included with VentureBeat Insider and VentureBeat VIP memberships.
Johnson: This was the initial effort to get games up and running, to get the OpenGL drivers running, and to get Steam ported. The SteamOS thing was at the very beginning when I left. Doing all that simultaneously, I got to know Jeri and the project she was working on. She didn’t have any software support, so I worked in my spare time to create this prototyping environment, so we could create samples and ideas and see what they looked like in AR. Figuring out what games work and what doesn’t, how to handle input, how to present data to the users, all that sort of thing. We started working together and all this spawned from there.
Above: Left4Dead
Ellsworth: I knew we were on to something big when people started coming to the lab and hanging out for three, four hours at a time, playing with our prototypes. Rick had put together a really simple multiplayer game – which he hates, because it had a lot of bugs – but it used the wands. Multiple people could sit around the table with a flat piece of reflector, and it had a scrolling maze that would move around depending on where you moved your character through with the wand.
The fun part of the gameplay, what was so addictive, was that you could grief the other players. There were zombies and other stuff running around this maze, and you could get the other player to run into zombies by doing things. But you could also look at each other across the table and kind of tell what they were doing, because they’d be moving their wands in different ways. You could counter what they were doing if you could tell what they were up to, even if they were five meters away in this virtual maze.
Those prototypes were ridiculously heavy, not even close to the miniaturization we have now. It was just giant, these big things that sat on your head. That was the first prototype, there. It was just to prove the concept. We shrunk it a bit and got it down, but it was still big. But we knew we were on to something, because all these different game experiences were super addicting. People were disrupting us, because they’d come down wanting to play.
Johnson: Jeri picked up these horrible sounds from Left 4 Dead, the zombie deaths. Every now and then there was a bug in the sound system, where the sound would take five times as long to play. It was this low-pitched AAAAAAGH! Here I am at two in the morning, I’m trying to do pixel shader coding, and people are talking and making noise while these crazy sounds play. Finally I’d have to kick them out.
Above: Left4Dead
Ellsworth: We were working late into the night, because everyone was so excited about this. People would come down about six or seven o’clock when the rest of Valve was getting done, and they’d be there all evening. We’d all go out to dinner and come back, and they’d be like, “Hey, can we do another round of Zombie Maze?”
GamesBeat: Was this kind of self-assembled? You would just grab people, or people would gravitate to your project.
Johnson: In this case, it was a little more difficult to attract people. Introducing hardware to a software company — you can’t put out hardware and patch it. The iterations of going from the giant head crab to something that’s actually comfortable, it’s a long road. Sometimes it’s hard to convey that it takes time for these things to iterate.
We found ourselves, mostly, with other hardware people developing it. We were constantly inviting people down to take a look at it.
GamesBeat: Did you have to bring them in, or were they already at Valve? Did you have to hire new people?
Ellsworth: I recruited most of the hardware team. I was brought on to do that.
GamesBeat: For this project, or for this broader laboratory operation?
Ellsworth: The whole thing. We were all helping each other out on different projects.
Johnson: Historically, for the most part, this has just been Jeri and I, up until the completion of the Kickstarter. Now we have some capital to hire people who can help grow the business and alleviate some of the effort we’re doing.
Ellsworth: We’re almost two years on it. It was actually a very short time at Valve. We exited Valve in February of last year and it was still the big prototypes.
Above: This is the Oculus Rift development kit, which is compatible with Valve’s just announced SteamVR.
GamesBeat: Was that part of a strategic decision Valve made, to do VR as opposed to augmented reality?
Ellsworth: It’s complicated. It wasn’t necessarily the project at all. It was just a changing direction inside the company. Fortunately, Gabe [Newell, Valve’s managing director] just let us have it all. It wasn’t exactly easy to get it past the lawyers, but we were able to negotiate, free and clear, all the technology. Part of the tech was developed at Valve, but a lot of the novel stuff was floating around in our heads just before we left. In fact, we’ve developed a lot of the tracking technology, a lot of the image stabilization, all this stuff, outside of Valve. We’ve been working to protect our IP ever since.
Johnson: We have a full-time person who’s working to protect what we’ve already done, and also the progression into the future. Patents are a necessary evil, unfortunately.
Ellsworth: We see the big picture of where this can go. We know the trends in the way you can do image processing and do tracking and image stabilization. We’re just taking baby steps into it now, but we’re developing the IP to go forward, so that we can do the ultimate in walkaround AR. Surface-free.
Johnson: Essentially what you see in movies. The processing power and the hardware power is still a decade away, though. In order for us to be successful, we’re taking logical steps in order to get there, but we don’t want to bite off too much to chew. What we’re doing now is finding a good way to blend the physical world with the virtual world, by using the wand and the RFID grid. Those are our first steps.
We have concepts of how we want to progress the technology going forward. As we continue with the company, we’ll progress that way.
GamesBeat: How did you come up with a business in the meantime? Can you get something out more quickly that will satisfy people’s curiosity while you’re still working toward this 10-year horizon?
Ellsworth: Just the early experiments, how engaged people were with the simplest demos — it’s a whole new way to interact with digital media. Part of our customer base found us, just as we showed our early demos at Maker Faire. We were stunned.
We chose Maker Faire because we thought we’d be sympathetic. It’s the crowd I run with. I’m an engineer, maker type. We showed up there with four workstations and we set up several very simplistic game experiences, without announcements, really. The first couple of people dribbled into the booth, took a look at it, and they didn’t want to put the system down. Then they’d go off and come back with five of their friends. Within 30 or 40 minutes of the opening door, we had a line outside our booth 45 minutes to an hour-long.