6 things we recently learned about the Google Assistant

The computing world has been shaped by massive changes that tend to come about every 10 years, said Google CEO Sundar Pichai today at a major press event where Google Home, Google WiFi and the Daydream VR headset made their debut.

The personal computer, the web, and mobile devices led massive change in decades past. In the decade to come, artificial intelligence will lead the way, he suggested, and Google Assistant will be your guide.

“We envision Assistant as a two-way conversation, a natural dialogue between our users and Google to help them get things done in the real world,” Pichai told an audience in San Francisco and the more than 300,000 watching on YouTube. “We want to build each user his or her own individual Google.”

Bits of information about Google Assistant have trickled out ever since it debuted at Google I/O in May, but now we know a lot more about what it can do, and how it could someday become a killer bot.

Here’s what we learned today about Google’s answer to Siri, Cortana, and Alexa.

1) Google Assistant will have its own bot ecosystem

Called Actions on Google, the platform will launch in early December. A software development kit (SDK) to put Google Assistant into devices beyond Google Home or Pixel phones will be available next year.

Developers who created bots or virtual assistants using API.ai will be able to bring their bots into the Google Assistant world. This is a quick turnaround, since API.ai was acquired just two weeks ago. Launched in 2014, API.ai has been used by more than 60,000 developers in the past, surpassing Facebook Messenger and Microsoft Bot Framework ecosystems made earlier this year.

A few dozen companies have already signed up to be some of the first services integrated into Google Assistant. These include Product Hunt, Food Network, and LinkedIn.

Above: Screenshot of initial partners with plans to make Google Assistant integrations

Image Credit: Google

2) On Pixel phones, you won’t have to ask Google Assistant to search for anything. It can automatically follow your conversations and do web searches

Say you’re chatting with a friend about your plans to grab dinner and see a concert this weekend. Ready to make reservations? On Pixel or Pixel XL, Google Assistant automatically follows your conversations and has already searched for reservations and tickets to the show. To see what Google Assistant has cooking for you, do a long press on the home button, and search results will scroll up from the bottom of the screen.

The feature goes a step further: Say you’re looking at a photo of Yellowstone National Park. Do a long press on the home button, and Google Assistant will already have searches based on your past search history, from what Google’s computer vision is able to “see” in an image.

This appears to be the same as the Google Now on Tap feature, which made its debut at Google I/O 2015 and is available on Android devices. The ability to recognize images was added to Google Now on Tap in June.

3) Google Assistant can do a lot

We’ve known for a while that Google Assistant will be able to find nearby events and search the 70 billion facts known to Google, but Assistant has a lot of other functions that became clear today.

Google Assistant can put events on your calendar, remember things for you, and make restaurant reservations. Several functions will be added when Actions on Google launches later this year.

Above: Example questions to ask Google Assistant shared during the Oct. 4, 2016 special presentation by Google in San Francisco

Image Credit: Google

The above list of questions was shared during Google’s keynote today to illustrate to the idea that once Actions on Google launches in December, Google Assistant won’t just help with making a grocery list, it will also teach you how to speak Spanish with Busuu, meditate with Headspace, and figure out what to watch with AndChill.

4) Google Assistant can’t make purchases, at least not yet 

Payments is an area where Alexa and Facebook Messenger appear to have a leg up on the competition. It may be the year of conversational commerce, but very few chat platforms allow you to make purchases without leaving the app.

Echo and Tap users can ask Alexa to order millions of items from the Amazon retail marketplace just by using their voice, and a beta began last month with partners like Stripe and PayPal to allow payment within bots on Facebook Messenger. With the ability to make reservations and order tickets for a concert, Google Assistant may facilitate plenty of commerce, but Google Assistant does not yet appear to have the ability to process purchases. It’s really hard to imagine Amazon and Facebook being the sole platforms to support purchase for much longer, as others will surely race to incorporate payments within their app.

5) You can use your voice to bring up photos based on name, date, or location

Using voice commands in the special presentation today, Google Assistant was able to display a photo album using Google Home, a TV, and Chromecast. This feature not need to be limited to a Chromecast. Google Assistant makes it easy to bring up photos in Allo, too, by saying something like, “Show me all my pictures from Berlin.”

Photos can be displayed based on name or by name and date. A similar function is available in Google Photos.

6) You can speak to Google Assistant anywhere in your home, through Google Home or other devices

Multiple Google Home devices will be able to communicate with each other so that you can speak with Google Assistant anywhere in your home. Whether through an Android watch, Pixel smartphone, or other devices, you will be able to talk with Google Assistant on the device that hears you best.

The embeddable SDK due out next year means that Google Assistant may also come to other devices, like cars or appliances, so you may truly be able to speak with it anywhere in future.