YouTube now uses deep learning to pick better video thumbnails

Google is back at it, using its artificial intelligence know-how to improve its products. Today the beneficiary is YouTube’s little video thumbnails. Google has begun to use a type of AI called deep learning to automatically create the best possible thumbnails when people upload videos.

Deep learning involves training systems called artificial neural networks on a large supply of labeled data — like images, text, or audio recordings — and then directing the systems to make predictions about new data.

In the case of the deep neural network (DNN) program now running at YouTube, Google is sampling frames from videos, scoring them, and generating thumbnails of those with the highest scores. And Google doesn’t need to worry about coming up with good-quality training data, because some users have already done the work of manually selecting thumbnails to use for their videos.

“Compared to the previous automatically generated thumbnails, the DNN-powered model is able to select frames with much better quality,” Weilong Yang of Google’s video content analysis team and Min-hsuan Tsai from the YouTube Creator team wrote in a Google Research blog post today. “In a human evaluation, the thumbnails produced by our new models are preferred to those from the previous thumbnailer in more than 65% of side-by-side ratings.”

Above: A comparison of video thumbnails from YouTube’s current and previous systems.

Image Credit: Google

Facebook, Snapchat, and Yahoo, among other companies, have also explored applications of deep learning on video. Startups like Clarifai and Dextro now extract information from video with deep learning for use in other companies’ applications.

Google uses deep learning across more and more products, including Google Translate, Google Voice, voice search, and Google Photos.