Microsoft brings Cortana to the Xbox One in latest dashboard makeover

Microsoft is announcing today that it will soon bring its Cortana voice-recognition system from Windows to the Xbox One. It is also tightening the integration between Xbox One and Windows 10 and creating a fresh makeover for the Xbox One dashboard.

The integration is all in the name of unifying the technologies behind both Windows 10 and the Xbox One, and improving the experience for console users by adding the battle-tested recognition capabilities that Cortana has picked up from tons of usage on both mobile phones and the PC.

Cortana is expected to be a big improvement because it will support better natural language recognition, said Albert Penello, senior director of product management and planning at Microsoft, in an interview with GamesBeat. Microsoft will sunset the existing voice commands for Kinect on the Xbox One in favor of Cortana. Cortana will still use the microphone for Kinect, but it will now have access to all of the machine learning improvements that make the natural commands work so much better with Cortana. Cortana will also work with consoles that don’t have connect, where the user has a microphone-headset instead.

Above: Xbox One’s dashboard update now includes Facebook friend finder.

Image Credit: Microsoft

You start out by saying, “Hey Cortana, Xbox on.” The light indicates that the system has heard you.

Today, talking to Kinect is a little like talking to your dog. You have to issue a very specific command to get it to recognize what you mean. Penello demonstrated how Cortana works in a demo.

He said, “Hey Cortana, I want to play Killer Instinct.”

Cortana was smart enough to call up three different possible versions of Killer Instinct that are available. Speaking with the voice of Cortana from the Halo video game series, Cortana presented the options.

“The third one,” Penello said.

The language is much more conversational. If you only had one version of the game installed, Cortana would go directly to that option.

You could also say something like, “Invite Ralph to my party.” Cortana seeks out the real name associated with an Xbox Live gamertag and send the invitation to join the party on the Xbox One. It sends the invite so that Ralph can join a multiplayer game.

If the current system doesn’t understand you, it doesn’t do anything. Cortana will always try to take some kind of action. If you ask, “Hey Cortana, what is there to eat around here.” She will answer with a web search that shows restaurants nearby. You can’t do that today. But Cortana knows how to do web searches, and it knows your location.

As Cortana improves on the Xbox One, those improvements will get rolled back to the PC.

You will also soon see some cosmetic changes Microsoft is making to the Xbox One dashboard. In the past, you had scroll down to get to “My games and apps.” Now you can simply see it on the right side of the screen. And for those with large libraries, you can scroll vertically, rather than horizontally, to find your games. You can see the progress of your installs, the speed of your current downloads.

Above: Xbox One’s dashboard update. Now it’s easier to find games.

Image Credit: Microsoft

The store will change too where you will see a lot more apps available from the Windows app store, which is possible because of Microsoft’s integration of the Universal Windows Platform. UWP has been rolling out gradually, but it has also drawn some criticism. Tim Sweeney, CEO of Epic Games, said at our GamesBeat Summit even in early May that UWP could result in a much more closed platform for game and app developers.

Penello echoed what other Microsoft executives have said: Windows, the Windows app store, and UWP are open, and Microsoft is going to make features available that expand the openness over time. Developers can deploy UWP apps on the console or Windows 10 and deploy them to the store. Those apps will be accessible to both the console and the PC. The result is more apps should be available to console owners than it the past. Games will still be submitted through the filter of ID@Xbox, so that Microsoft can screen the games as needed.

The store has some new features. The Windows store had strikethrough pricing, where an old price is visible next to a new price. If you own a game like Rocket League, you can look at that game and all of its accompanying downloadable content (DLC) will be nested and visible.

“We get some nice benefits by having the stores merge,” Penello said.

The Windows store wasn’t really designed for blockbuster Triple-A game launches. Now app makers can do things like preorders, bundles, and DLC orders in the Windows store for Triple-A games.

Above: Cortana makes it easier to send Xbox party invites.

Image Credit: Microsoft

On the PC side, Penello showed the Xbox app running on Windows 10. You can see your friends activity feed, the same one as you see on the console. You can purchase Windows and Xbox games through the app. You can stream games from the console and view them on the PC.

“We’ve been iterating a lot on this app over time,” Penello said. “This has been focused on Xbox Live and UWP games.”

But people on the PC are playing a lot of non-UWP games such as Overwatch from Blizzard and other Win32 games, which aren’t part of Xbox Live. You can see a game like Age of Empires 2, a Win32 game purchased on Steam. The app allows for game DVR and party chat in any game. So you can capture video of your gameplay of Age of Empires 2. And you can see game details on the top 1,000 or so PC games.

“Xbox Live is aware of the presence of Win32 and Steam games,” Penello said. “If I am on the console and see what you are doing, I can see you are playing Age of Empires 2. That allows you to keep in touch with your gaming community even when you are not playing a Xbox Live game.”

This is the first major update for Xbox One since last November. Cortana will debut in the U.S., United Kingdom, France, Italy, Germany, and Spain first.

You can edit the game DVR clips on the PC and upload them back to the Xbox Live service. You can make title changes or edit length. You can chop a video or add title screens, and then push it back to the console or PC for others to view. You can do this for both PC games and Xbox Live games. You can find friends who are Facebook friends and are also Xbox Live gamers. You can add those friends on Xbox Live now quite easily.

With productivity apps, you can now add Xbox Live friends to the Windows people app. That means that you can add Xbox contacts and email or message those Xbox Live friends directly from Office.

“This app is getting more robust all the time,” Penello said. “Our goal is to have you running the app all the time.”