How Siri works on the new Apple TV

At long last, Siri is no longer confined to mobile devices and the Apple Watch. Now you can use Apple’s personal digital assistant with the redesigned Apple TV.

You can do more than just ask sarcastic questions and get sarcastic answers in this new implementation of Siri. Using speech recognition through the remote control for the Apple TV, you can search for videos available from multiple apps running on the set-top box.

“Siri will search iTunes, Netflix, Hulu, HBO, and Showtime, and we’ll be adding even more over time,” Apple executive Eddy Cue said during Apple’s big “Hey Siri” event in San Francisco today.

You can ask natural-language questions to find the content you’re looking for, but you can also ask Siri certain questions while you’re watching video, too. Jennifer Folds from the Apple TV team showed how you can use Siri with the redesigned set-top box.

Above: Jennifer Folds asks a question while watching video in the redesigned Apple TV.

Image Credit: Screenshot

Here are some things she asked in today’s demo:

• “Show that Modern Family episode with Edward Norton.” That brings up a Modern Family page with just the right episode selected.
• “What did she say?” That temporarily turns on captions for the most recent 15 seconds of video.
• “Just the ones with Sean Connery.” That filters down a page containing all James Bond films to only show the ones with the famous Bond actor. In fact, Folds said, in addition to being able to pull up people who appear in movies, you can also filter for directors, and dates.
• “What are some new movies that are good to watch with kids?” This shows content selected based on age ratings.
• “Who stars in this?” Now you can see the names and headshots of everyone in the cast of a TV show or movie.
• “How did the San Francisco Giants do yesterday?” It’s easy to look at sports scores with Siri on Apple TV.
• “What’s the weather like in Juneau, Alaska?” There’s a new weather section from Siri in Apple TV. You can swipe up to see more detailed weather information, and Siri automatically pauses video content if you do that.
• “Open Music.” That makes the Apple Music app pop up.

Above: Weather from Siri in Apple TV.

Image Credit: Screenshot

But Siri doesn’t let you control your connected devices by using Apple TV as a home automation hub, as some Apple observers expected.

Still, new capabilities do amount to substantial feature upgrades for the Apple TV, which first appeared in 2007.

The thing is, Microsoft’s Xbox already offers speech recognition with Kinect, and Google’s Android TV and Amazon’s Fire TV have speech recognition, too.

Google and Microsoft have poured extensive resources into developing the underlying technology for speech recognition in the past several years. Both of those tech giants employ researchers who work on a type of artificial intelligence called deep learning, which involves training artificial neural networks on vast supplies of data — like recordings of people speaking — and then throwing new data at them in order to receive inferences.

Last year Apple was reported to be hiring people in this field. And earlier this week Reuters reported on a more concerted effort at Apple to hire AI researchers.

Above: The cast of Paddington, courtesy of Siri on Apple TV.

Image Credit: Screenshot

With more brains working hard to improve the technology, Siri should be getting more accurate over time. And as more people use Siri, Apple gets more training data to make the personal digital assistant smarter, too.

Not that Siri’s IQ score has been stagnant all this time. In June Apple senior vice president of software engineering Craig Federighi said Siri’s word error rate had gone down by 40 percent.

Now that intelligence is packaged up for consumers to use with Siri on Apple TV. The question is whether this will help Apple sell a lot more Apple TVs in the months and years to come.