Facebook’s latest deep learning tech can quickly interpret text and video

Facebook chief technology officer Mike Schroepfer today showed off the newest capabilities of the company’s artificial intelligence systems. Not only can they mine images, but they can also recognize actions like sports in video, and they can answer questions about text, too.

“You can really get these systems to understand deep, minute differences,” Schroepfer said today at Facebook’s developer-oriented F8 conference in San Francisco.

Like Twitter, Baidu, Google, and other companies, Facebook has been adding on artificial-intelligence talent and developing systems for a type of AI called deep learning. The latest innovations from Facebook fall clearly in that category.

The advances in question answering were documented in an academic paper called “Memory Networks,” which you can read here. Facebook first talked about the research publicly in November.

Regarding video, Schroepfer demonstrated how Facebook can now identify hundreds of sports as they happen, even if they’re similar. So it can determine if a video is showing figure skating or roller skating, or whether it’s sledge hockey or roller hockey.

The video work follows advancements from a startup called Clarifai, which earlier this year expanded deep learning systems beyond image recognition. Now the startup’s technology can pick up on objects that appear in videos.

You can find clips from Schroepfer’s presentation here and here.