Osmo Monster lets kids interact with cartoon characters on iPad

Kids love television cartoons, but they can’t actually interact with the characters in them. Osmo is changing that with the launch of Osmo Monster, a “mixed reality” experience where children use an iPad to interact with a monster and bring their physical drawings to life.

The product, which introduces kids to Mo the Monster, is the latest in a series of award-winning titles that use the iPad to teach children how to think. Pramod Sharma, the chief executive of Osmo in Palo Alto, Calif., told GamesBeat that it took three years to create Osmo Monster. (Sharma is speaking on augmented reality in a panel on Monday at our GamesBeat 2016 event).

“This is our most engineered product ever,” he said. “It’s hard to figure out what these characters do when they come to life.”

The product is an interdisciplinary effort of former Google veterans, a former Stanford human-computer interaction academic, and ex-Disney animators. With Osmo Monster, Mo tells the kids what to do, like what to draw on a physical whiteboard. The iPad camera sees what the child has drawn and translates it into the scene on the iPad display.

So Mo can ask a child to draw a stick. The child does so, and then taps on the screen. Then the program imports the stick into the animated scene on the iPad. Mo can then interact with the stick. It’s a magical experience that can make turn a cartoon-watching experience into an interactive, mixed reality lesson.

“We did this because we don’t want technology to turn children into passive, digital zombies, just staring at the phone,” Sharma said. “With Mo, you can put the iPad in front of you and make him come alive.”

Mo the Monster is a furry orange creature, designed to interact with children ages 4 to 10. Mo encourages kids to co-create animated activities using the Osmo system and its accompanying Creative Set. When Mo requests various objects, kids respond to him by creating real-world drawings that Osmo’s reflective artificial intelligence technology incorporates into Mo’s digital world on an iPad. I watched Sharma draw a picture of himself, which was then imported into the cartoon scene. Then Mo started dancing with Sharma’s character.

“If Walt Disney was a 5 year old today, we think he would create his first cartoon with Osmo Monster,” said Sharma. “Creativity is critical to early learning and development, as is building motor skills through drawing. We believe that technology shouldn’t just be for math and science kids; it can inspire and empower creative children as well.”

Monster has been in development since the company’s inception in 2013, but the technical and creative hurdles proved to be much more challenging than anticipated, requiring more than three years and over 20 iterations to bring the product to fruition. Sharma said the team was inspired by thinking about how to take a character from something like Sesame Street and then make that character come alive. Every time you interact with Mo, it will ask you different questions.

Along the way, Osmo brought in Zac Pavlov, an animation engineer from Disney, and Jesse Cirimele, a Stanford Ph.D. in Human-Computer Interaction (HCI), to build the technology behind Monster.

Now, Osmo is leveraging Cirimele’s background in HCI to pioneer “Human-Character Interaction,” a platform that combines machine learning, artificial intelligence, and computer vision with the magic of an animated character to deliver the first mixed reality game that creates fully animated experiences.

“Each technical hurdle the team solved for Monster ultimately improved the entire Osmo system,” said Sharma. “Mo is just the beginning of what’s possible. Our Human-Character Interaction technology will enable children to hang out with their favorite characters and do fun, creative activities together.”

Monster builds on Osmo’s existing creativity games, such as Newton and Masterpiece, which have been remastered with updated content for the Osmo Creative Set, also released today. Available for $50 at playosmo.com and Amazon, the Creative Set includes the Osmo Creative Board, six Yoobi colorful chalk markers and a fuzzy pencil case, and the Monster, Newton, and Masterpiece apps.

Both the Creative Board and the chalk markers have been specially calibrated for Osmo’s computer vision technology. Osmo has gone from an idea of two parents — Sharma and Jerome Scholler — to being a mixed reality game system that was named a best invention by Time Magazine, landing in Apple Stores and in more than 15,000 schools across the world.

Monster launches with one experience, “Mo’s Magic Show,” and the company plans to release more experiences this fall as free updates to the Monster App. Each experience has numerous activities, and kids can make unlimited drawings, so each one can be played over and over again.

Sharma said the company has 40 employees and is now able to produce things in parallel. The Osmo Monster and Creative Set is the third product launch in the past year, in addition to Osmo Coding and Osmo Numbers.