Augmented reality takes pop-up books to the next level (interview)

Augmented reality, combining real world views and digital information, is not particularly new, but it is still considered in its infancy. The technology has been used in a variety of ways, from GPS systems to fitness apps, though entertainment seems to be the most popular.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":380386,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,games,media,","session":"C"}']

GamesBeat had a chance to sit down with Scott Jochim, president of Digital Tech Frontier and Popar Toys, to discuss his company’s line of augmented reality children’s books, which are paving the way for affordable household 3D experiences.

GamesBeat: Can you elaborate on what your company does?

Scott Jochim: The background of Digital Tech Frontier, my firm, is that for the last 15 years we’ve been working in the entertainment, training, and educational realms of technology. Finding different ways to not only influence, but educate and entertain through technology. Human interface devices have been a very important part of our technology — anything that involves both emotion and intellect, and if you can have fun with it, that helps as well.

Our products have included everything from NASCAR simulators for General Motors and Toyota to cancer therapy simulators, to help patients get through some different pain modalities, such as dialysis or burn treatment centers or even anxiety through chemotherapy. Our products have done everything from 3D stereoscopic imaging to natural, surprisingly enough, aromatherapy technology, allowing scent to be dispensed for different ride films or entertainment venues, so people see, hear, smell, and feel the experience.

In the last five years we’ve been concentrating on a product the Virtual Reality Development Lab, allowing virtual worlds created by students and teachers and be shared outside and inside the classroom, creating an engaging way to take information and package it for an environmental and spatial technology forum. These technologies have created what our current technology infrastructure is, which is the Popar Toys lines of products. We feel that a convergence of a virtual environment and a real environment, that augmented reality, the transference of being able to place objects or experiences from the real world in the virtual environment, and vice versa, is a very important aspect of where we’re converging to with technology.

GamesBeat: Can you give me some specifics of what products Popar Toys will be debuting and selling?

Jochim: Well, we figured we’d want to come out with four products to debut into the market. Trying to cover a lot of different individuals. It’s hard, as a child at heart, coming out with one core product. The first product we came out with was called Planets 3D. Myself, I wanted to be an astronaut, until Virgin Galactic gets off the ground I don’t have tickets yet, but my hope is there one day. Rich Branson, if he could send me a ticket, that’d be awesome. Are you familiar with that?

GamesBeat: Yes.

[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":380386,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,games,media,","session":"C"}']

Jochim: Yeah. Hopefully you’ll get a ticket, maybe you will, you can write a report on it and tell me how cool it is.

GamesBeat: Journalists get a lot of free perks but I don’t know if I’d take it. I have a hard enough time flying from San Francisco to Las Vegas, so I think going up in the atmosphere would be kinda rough.

Jochim: Yeah, it could be rough, but it’s all worth the patch, man, the patch that says you’re an astronaut, it’d be very cool.

GamesBeat: I think when I’m like 85 and really old and have beaten Skyrim and everything, I’d maybe give it a go.

[aditude-amp id="medium2" targeting='{"env":"staging","page_type":"article","post_id":380386,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,games,media,","session":"C"}']

Jochim: Fair enough, fair enough. I’m sure my employees would not want to hear that I want to go to space, they’d probably want their paychecks before I went there.

So we decided to create Planets 3D, giving every kid the ability to be the explorer for the first time. If I read a current book, or trading cards or something from NASA or about a space explorer or even Stephen Hawking, you flip through the pages, you see some pictures, some text, maybe you could kick on the Discovery Channel on TV and listen to William Shatner talk about space and the final frontier. But our books do something completely different, you’ll actually open up the pages of the book and the earth will rise out of the book and spin around you. If you bring an object into the book, we call it an “I-paddle,” I for interactive, and bring that close to the earth, you’ll actually fly above the earth into the ISS space station and you’ll see that traveling above the earth’s atmosphere.

Every page of the book that you open up, something different will occur. On Mars you’ll see the Spirit rover traveling over the Martian landscape. You can explore that landscape or the Mars atmosphere. There’s even a page that’s called “Into Orbit,” as you open the page you see a space shuttle launch off the page and into your world. Allowing yourself to not only read it and see it, but become the process as well. That’s an important aspect of our books, we feel that you can read about the book, you can see the book come alive, but now you can actually become the astronaut as you’re reading it. As I’m watching this space shuttle launch off the pages of the book, I can look at myself in the book and see myself as an astronaut. There’s a helmet on my head.

GamesBeat: What technology, in addition to the books, is required to have these experiences?

[aditude-amp id="medium3" targeting='{"env":"staging","page_type":"article","post_id":380386,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,games,media,","session":"C"}']

Jochim: A laptop computer or desktop computer, Mac or PC, and a webcam. That’s it.

GamesBeat: I was watching a video that you sent along to me, it showed a few children playing with the books. I was wondering, in what ways is this really different from… Especially on the videogame side and especially on iPhones, there’s a lot of augmented reality that came onto the scene about two years ago, and over the past two years has blown up a little bit. The iPhone, PlayStation Vita, they all have augmented reality programs and games, the space has been explored reasonably well so far. So are you familiar with those, and what is really the unique thing you have over those?

Jochim: Well, I think it’s interface, and the ability to become what you’re doing. A mobile phone, for instance, or a Vita, which I’m very excited about, you are using it, you’re looking through into the looking-glass universe. Looking at cards, exploring an environment. When you’re using a Mac or a PC, the environment is being looked at with you, you are exploring the environment together. The ability that our books or our trading cards allows is, as you’re reading about an astronaut, you can look up at the screen and see yourself as an astronaut. You have a bubble over your head. Or you’re reading about the praying mantis, and you are a praying mantis as you’re reading about it in the book. The ability to change into those costumes, be able to see yourself as a swashbuckling pirate while you’re reading an adventure story, is possible with our book. You’re looking at the universe through those other devices and exploring it, while this device is showing you in that universe. It’s giving a reflection, rather than a binoculars approach to augmented reality. You have an interface such as a book, you can use that book to then control that space shuttle. Or if you’re a girl reading about princesses, your dad’s reading to you about princesses, and you’re like, Daddy, I want to be the pink princess. Daddy can change her into the pink princess as he’s reading through the book. How awesome is that? You’re becoming those characters you’re reading about, you’re exploring them, you’re able to play with those individuals and those attitudes inside the book, becoming a little bit of a drama-based or theater-based experience as well. Which is translating into a lot of different aspects. Mobile aspects, don’t get me wrong, are going to be incorporated into our books, we’re looking at releasing all of our books on the Mac and PC at one time, and we’re going to be working with mobile to create games, as you’ve seen before, inside the books, such as… The construction book will have a real-world brick-breaker. Our Planets 3D book will have a basketball game in it. You’ll be able to use our books and our cards as those interface devices to create those video game as well. We see the video game aspect being a very interactive, hands-on experience in each one of the books, but not the main aspect of it. What we want to do, realistically, is when you open up a book, you want to become that experience you’re reading about. Reading it, seeing it, and becoming it.

GamesBeat: I was actually intrigued by one of the very first things you said, you mentioned smelling, when you’re having these experiences. I was wondering if you could give me some examples of how that applies here?

[aditude-amp id="medium4" targeting='{"env":"staging","page_type":"article","post_id":380386,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,games,media,","session":"C"}']

Interview continues on the next page.

GamesBeat: I was actually intrigued by one of the very first things you said, you mentioned smelling, when you’re having these experiences. I was wondering if you could give me some examples of how that applies here?

[aditude-amp id="medium5" targeting='{"env":"staging","page_type":"article","post_id":380386,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,games,media,","session":"C"}']

Jochim: A few years ago, we were contacted, surprisingly enough, by the Cancer Therapy Centers in New Rochelle, New York. They wanted to build a pod that allowed patients to be able to see, hear, and relax. What I asked them about is, how they could incorporate some different sensations beyond that. Our background was with experiential technologies, and we wanted to incorporate smell within a relaxation center for the Cancer Therapy Centers. And so we incorporated some aroma modalities, a synthetic aroma dispensing system that was completely hygienic, hypoallergenic, and FDA approved, that allowed dispensing of smells. While you saw in 3D, you heard in a stereoscopic 360-degree surround sound environment, you felt as if you were laying in a neutral body position, when you fly up into space your body goes into this position. You’ve got the sensation of flying or floating or relaxing. The scent system that we added to it is actually very detailed, in the sense that it releases small chemicals throughout the nozzle that’s near the nose, and because it’s a heavy chemical that doesn’t stick to the skin or to the clothing, you can just smell it for a few milliseconds and then it goes away. It allows us to dispense a multitude of smells in a matter of a few seconds, so you could smell anything from lavender to orange blossoms to salt water. We worked very closely with some aromatherapists and positive visualization therapists to create a very relaxing 25-minute session. If you were getting chemo or having dialysis done, you would put on a head-mounted display using stereoscopic imaging, you’d have headphones on, and then aromas would come in while you’re laying down in the chair getting your treatments. We found out that… What created the movement for us toward experiential technologies was that we found, through all of our studies, 100 percent of the patients felt 80 percent better while using our technology when getting treatment. That was amazing to us, that we could create such a simple mechanism, sight, sound, and smell, that would relax people, lower anxiety while they were getting these treatments that were very harsh on their bodies. It’s easy to say, but your mind controls a lot of your body. This is an influence on that process. We utilized that technology and took it back into entertainment as well. For a tour with Lego… Everyone’s familiar with Legos growing up, we were the first company ever to bring Legos to life, we animated Legos back in the late 90s and brought them to life in a stereoscopic adventure, with sight, sound, and smell, called Deep Sea Adventure. We traveled around the US to all the Toys R Uses and Zany Brainys and Targets across the US showing kids that Legos can come to life. As most of the kids now know, Lego is not just a physical object anymore, it’s a digital object. That was due to our company.

GamesBeat: The concept of sight, sound, and smell, will all three of those be factoring into the upcoming Popar augmented reality books?

Jochim: Probably not the smell system. A little expensive, you’re looking at a dispensing system that’s in the thousands of dollars, while these books will be retailing between 20 and 27 dollars.

GamesBeat: Do you see a fuller experience where players and people are experiencing… Not just your products, but video games and movies, do you eventually see those as experiences that will use more of the senses? Smell and things like that factoring into the home experience?

Jochim: Being a professional experiential artist, there’s a difference between a utility activity and a novelty activity. Very quickly we can harp upon the idea that smell was very engaging, it was very emotional for people to smell salt water taffy or fresh-baked cookies, it brings up the sensation of being at home. The problem with smell technology, and we realized this very quickly, is that everyone has a different emotional reaction to smell. Tires screeching, everyone will back up a little bit and worry that there’s an accident, it becomes anxiety. But not everyone likes the smell of cookies. You would think everyone loves it. Visually, we can show somebody fresh-baked cookies, they’re not going to have the same emotional reaction as they would smelling it. Smelling it triggers a deeper emotional reaction than vision does. So what we found out very quickly is, smell technology, albeit very influential and emotional, is much more of a novelty experiential technique than it would be a utility. For instance, stereoscopic imaging is gaining a tremendous amount of ground these days, it’s been in video games for years, been in movie theaters before that, specialty motion rides. We’re framing that as 3D comes home. 3D becomes involved in the process. That is catching up, people are starting to realize the value of depth perception or environmental and spatial technology within video games. But even further, with the Kinect and with augmented reality, people are utilizing their bodies and their natural movements as an interface to become natural for video games. That’s where we really feel augmented reality and our video games will go. Imagine playing the next Counter-Strike and it’s actually in your office. Imagine playing BrickBreaker, which everyone loves to play, and you’re playing it on the wall of your home, the bricks of your home are coming down. These are where augmented reality and video games and our technology are starting to converge, creating a displacement of reality, and a virtual environment where we can do things that normally we couldn’t do in a real world. Allowing us to play a little bit more. Imagine playing an augmented reality of… Remember the old-school game called Punch Buggy?

GamesBeat: Uh, no.

Jochim: Okay, you see a Volkswagen, you hit somebody on the shoulder and say “Punch Buggy.” It’s an old game we used to play on road trips, you probably called it something different.

GamesBeat: Oh, Slug Bug yellow…

Jochim: Slug Bug, yeah. So imagine an augmented reality Slug Bug game. Say you’re traveling, you’ve got your brother in the back seat, and the whole idea is to use your phone and see who can capture the Slug Bug. But you’re doing it virtually. Your buddy’s not riding with you in the car, you’re playing a virtual game, your camera’s out the window a Bug passes, you see yourself on the other side of the phone, smack, one point. You can then start to create some of these social games that are created in your environment, as if someone was actually there. A lot of the augmented reality games that we have seen and entertained at this point have been very simplistic. For instance, a zombie attack where zombies are coming down, or a car that’s racing across the ground. We believe, one of the groups that is going to be focusing and doing well is PlayStation, they’ve got some very unique games that are coming onto the market with their PlayStation Vita, with augmented reality, that we’re looking forward to seeing. Not everybody is going to be able to have the opportunity to have that device, but everyone’s going to have an opportunity, we think, in the next few years, to be able to use their mobile phone. We’re excited to be able to utilize those inside of our books and other environments as well.

GamesBeat: I noticed that your technology has been implemented in a number of school systems. I was wondering if you could elaborate on that.

Interview continues on the next page.

GamesBeat: I noticed that your technology has been implemented in a number of school systems. I was wondering if you could elaborate on that.

Jochim: Well, where Popar Toys came from is our interaction with school systems. Our augmented reality development lab is a lab created for the core curriculum standards here in the US. Allowing both teachers and students to use augmented reality in a multitude of different classrooms. We have a build-the-human-body experience, where kids have little paddles, one through eleven, for each part of the body, they can slowly build up the body, see the skeletal system, see the lymphatic system, see the muscular system, and at any point in time they can take out the muscular system from the body, examine it, learn more about it through visuals and text and oral. Now, all of this is augmented, which means they’re seeing themselves hold the skeleton in the palm of their hand. Bring another paddle, which translates into what we call the cheapest iPad 2 on the planet, because this one paddle displaces text, shows audio, shows 3D objects, and it changes upon the proximity of what paddle it’s next to. If this paddle is brought into the muscular system, it talks about the muscular system. If it’s brought into the skeleton, it talks about the skeleton. This allows kids, just like they’re doing in video games, to build the human body. They can take their own time, they can mess up, they’ve got a linear approach to seeing how it works, which means it’s not like a multimedia program that says it’s A to Z, but only goes A-B-C-D. Our program allows you to go A, D, B, C, allowing a kid who’s having some struggle to work through the problems himself. Interactivity is key to a kinesthetic learner. Our augmented reality development lab works in mathematics, social sciences, social studies, literature, arts, and foreign languages as well.

When you grew up, did you get a drum set when you were young?

GamesBeat: A drum set? No. I got video game consoles and Ninja Turtles.

Jochim: So did I. [laughs] I wanted a drum set, I never got one. To give you an example of how the AR deal works, we have an AR development implemented in the midwest. Actually we’ve got hundreds of them across the US, but this one in particular in the midwest, a group of kids said, “We want to be able to build something in augmented reality.” I said, “Okay, this isn’t very educational, but can it be done?” They wanted to build a drum kid. It sounds very fun, I gave the kids the tools to do it, within two days they’ve created an augmented reality drum kit. They showed me a video, it was pretty darn cool. They downloaded the objects, associated and recorded the sounds, before they knew it they had a drum kit in their classroom. This started to roll in the process, looking at our lab… Our lab is not just about a tool which we give to schools like a textbook. We’re giving kids and teachers the ability to create their own augmented reality. It’s not about us just shoveling a textbook experience to people, it’s allowing teachers to use their creativity and their imagination to excite and motivate students to do what they have in their head, as a commonality. Which is information. Kids, we’re finding, students are utilizing it as well. Both inside and outside the classroom. We’ve got kids creating a CPR t-shirt for Red Cross. We’ve got kids creating children’s books just like we are. And reading them to kindergarten kids in high school. We’ve got kids making energy-efficient modules to share with their parents to see how they can save energy. So our augmented reality development lab is not just about having individual pieces of software. It is similar to a Microsoft Excel or Microsoft Word, where we’re giving you tools and letting you write and create the formulas for whatever you want to share in augmented reality. That’s the key to where our ARDL lab is going. It’s not about the software, it’s about what you can do with the software, giving those tools to students to be able to utilize them as well.

GamesBeat: How have children, so far, reacted to the Popar books?

Jochim: I would say incredibly. We set up at a mall this Christmas, just to offer the books, and at first we were a little bit worried, because we set up right outside the Disney Store. What we found is a common occurrence, parents would ask their kids, “Do you want this, or another toy?” And they were talking about Disney. And as often as not they said they wanted the Popar book rather than the Disney toy. It shows the level of interactivity that we’ll have, something that’s not branded, but interactive, that will allow people to truly engage themselves. The kids love it, because not only can they read a book, they can see the book come alive. They can become the book. At a half or a third of the price of a video game. The parents like to buy it because, again, it’s a book. Or we’ve got trading cards as well.

GamesBeat: I also understand that you are looking into partnering with companies like Disney?

Jochim: Yes.

GamesBeat: So maybe those kids won’t have to make a choice down the road.

Jochim: They probably won’t have to, no, because I think licensing is a great opportunity to open up what these cards can be, beyond just our imagination. We can take a brand like Dora the Explorer… There are so many aspects in which we can create in this playset reality. It’s really the imagination that is going to be our limitation. If we give the right tools to the right people, it’s in their hands.

GamesBeat: You’re obviously an expert on the matter, where do you see augmented reality in a year or a couple years from now? How does the technology and the experience progress?

Jochim: I think people are going to realize how mainstream augmented reality is going to become. It’s been part of our daily lives for many years, we have it in front of GPS, we have it on the football field, we’re now starting to embrace it in video games and mobile devices. There are so many products coming out, even from Google, Microsoft, PlayStation. We’re going to realize how much we’re going to rely on augmented reality in our own reality. We rely, currently, on a virtual database, whether it be Google or computers every day. We’re always looking at our devices. The moment that we don’t have to look down at a device, and we’re looking out at the universe and the world around us, into the digital overlay, just like the Terminator, that information is placed in our reality. That’s what we’re realizing in the next few years, where we won’t be able to work without this digital overlay. Can you imagine a time when you didn’t google for information? Augmented reality is going to become the same way. Can you imagine a time when you just looked into the world and didn’t recognize all your friends from Facebook out there in the universe? You didn’t see the information about the ingredients in the McDonald’s hamburger that you’re eating? Or you didn’t see the coupon codes pop up around the Times Square you were looking around? This information is part of our universe now, but we have to access it by looking down. Augmented reality is going to become an overlay where it becomes part of our sunglasses, our contact lenses. Anything we wear as a device can then become augmented. And that information is there all around us. It’s just having the interface to be able to connect the virtual universe to our actual universe. We’re getting closer, the bridge is getting closer for us to convert both realities together. I’m excited for the day that we’ll have the information, not just in our phone, at our fingertips, but right in front of us, without having to touch anything.

GamesBeat: Just as long as it doesn’t all smell like fresh-baked cookies.

Jochim: [laughs] Just as long as maybe it doesn’t smell like a burning skidmark. But I love fresh-baked cookies myself.

GamesBeat: Alright, thanks for your time, Scott. Is there anything else you’d like to add?

Jochim: Thanks as well, I appreciate it. Again, we write everything from children’s books to… I’ve done video games all across the spectrum. I don’t know if you remember back in the 90s when computers didn’t have the processing speed nor the sound capabilities, we created a game called Cyber Knights. This was back when we could only do two sounds at a time, this was before the Sound Blaster 16. I was creating video games for Windows 98. So technology for me has taken a multitude of different leaps. We used technology for Nokia back in early 2000 that was gesture recognition. Very similar to the Kinect. It was just to access information and create your virtual buttons to work with. They wanted a technology to talk about their 3G networks. Now, of course, we’ve gone way beyond 3G networks at this point. But it’s not just about the technology. Good games stand the test of time, good interfaces stand the test of time, and ultimately good experiences stand the test of time. Think about that. Video games are a core of my beginnings, part of my daily life. It’s about making that experience more real. And more accessible to more people.