Microsoft is rolling out a whole host of speech recognition features for its Windows Phone 7 devices and Kinect-enabled Xbox 360 consoles in a move to make the gaming experience more immersive. The new features are part of what the company refers to as the “natural user interface” (NUI).
The company acquired speech recognition software developer Tellme in 2007. But there wasn’t whole lot of interest in the speech recognition sector until Apple stunned the tech community by buying Siri last year — and doing nothing with it for more than a year. With the release of Microsoft’s next mobile operating system update Mango and the next version of its Xbox 360 operating system, Microsoft is finally putting that technology to use.
“In the living room, fundamentally, you don’t want something on your lap,” Ilya Bukshteyn, Tellme senior director for sales and marketing, told VentureBeat. “We want the Xbox 360 to be an application hub for the living room and move the technology out of the way.”
There were already a number of speech features built into the Xbox 360 dashboard, but Microsoft’s NUI adds additional search features and makes it more accurate. When you say something to the Kinect, it automatically uploads that clip up to remote servers run by Microsoft that determines what you are saying and then issues a command back to the Xbox 360. The recognition is constantly improving as Microsoft’s cloud servers gather more and more data.
“Right about now it’s about 80 or 90 percent accurate,” Bukshteyn said. “But that’s improving on a daily basis.”
Giving players new ways to control the game is one way to make the gaming experience more immersive. Microsoft has tried to market its Kinect motion controller as an “experience” by giving Xbox 360 owners ways to control other parts of their Xbox 360 with motion controls — like the menus in the console’s dashboard access popular applications like Hulu and Netflix. Getting the features to catch on requires rebuilding games, even hardcore games like first-person shooters, with those same Kinect features.
Adding voice controls gives gamers an array of new options, like new ways to control their squads in the role-playing game Mass Effect 3. While still controlling the series’ protagonist Commander Shepherd with a regular controller, players can verbally tell their squad mates to advance on enemies and use special skills to disable them while Shepherd goes in for the kill.
I had a chance to test out the new speech features recently to see first-hand how they changed the gaming experience. The voice commands are pretty accurate, and I only ran into a few issues when the Kinect didn’t understand what I was saying. Gestures were also enabled at the same time, so I could wave my hand to move menus around and say something when I wanted to quickly jump to a new part of the menu. It doesn’t feel like you have to do any extra work to use the NUI commands and they are pretty intuitive — it’s like adding an extra button to a traditional game controller that should have always been there.
Developers can tap into the Xbox 360 speech recognition features as soon as they become available, but they will have to wait a little longer to get the kind of immersion that Microsoft applications enjoy on Windows Phone 7 devices. At first, developers will have to make users launch the app before they start using speech commands (though they can launch the application with a speech command).
For example, you couldn’t just say “Tweet ‘Hey, this music is awesome.'” Instead you would first have to go to Microsoft’s Me service or jump into the Twitter application, then dictate your tweet. Those kinds of features are in the pipeline, Bukshteyn said, but there currently isn’t a timeline for implementing them.
You can catch a few quick hands-on presentations of both the Xbox 360 and Windows Phone 7 NUI in action below.