Google, Spotify, & Pandora bet a computer could generate a better playlist than you can

Google, Pandora, and Spotify haven’t exactly advertised it, but they are all working on using a type of artificial intelligence called “deep learning” to make a better music playlist for you.

All three have recently hired deep learning experts. This branch of A.I. involves training systems called “artificial neural networks” with terabytes of information derived from images, text, and other inputs. It then presents the systems with new information and receives inferences about it in response. Companies including Google, Baidu, and others have put deep learning to work for all sorts of purposes — advertising, speech recognition, image recognition, even data center optimization. A startup even intends to use deep learning to recognize patterns in medical images.

Now these companies are turning to music. A neural network for a music-streaming service could recognize patterns like chord progressions in music without needing music experts to direct machines to look for them. Then it could introduce a listener to a song, album, or artist in accord with their preferences.

Putting these complex systems into production won’t necessarily happen overnight. But look out: Once in place, deep learning could be the kind of technology that inspires listeners to stick around music-streaming services for years to come.

“It’s a really exciting area, and certainly, it’s of high interest to Pandora,” Pandora senior scientist Erik Schmidt told VentureBeat in an interview.

Send in the interns

The new wave of attention leads back to an academic paper that came out of Belgium’s Ghent University last year.

In the obscure “reservoir computing” section of the university’s electronics and information systems department, Ph.D. students Sander Dieleman and Aäron van den Oord collaborated with professor Benjamin Schrauwen to make convolutional neural networks (CNNs) pick up attributes of songs, rather than using them to observe features in images, as engineers have done for years.

The trio found that their model “produces sensible recommendations,” as they put it in the paper. What’s more, their experiments showed the system “significantly outperforming the traditional approach.”

The paper captured the imagination of academics who work with music and deep learning wonks as well. Microsoft researchers even cited the paper in a recent overview of the deep learning field.

The group’s work also hit a nerve at Spotify. Dieleman and van den Oord ended up meeting some of the company’s employees while presenting their paper at the prestigious Neural Information Processing Systems conference near Lake Tahoe in December 2013.

“They invited us to give a talk about our work at the Spotify offices, since we were already planning to stop over in New York on our way back home anyway,” Dieleman told VentureBeat in an email. “That’s how I eventually ended up there, so I suppose the paper was pretty instrumental.”

As for van den Oord, he ended up interning with Philippe Hamel, a software engineer in the music research group at Google.

Dieleman and van den Oord have since returned to university, but it’s clear the companies wanted to look closely at the method the academics devised.

A smarter breed of robot

Deep learning stands out from the recommendation systems in place at Spotify, which uses more traditional data analysis. And down the line, it could provide for improvements in key metrics.

Spotify currently recommends songs using technology from The Echo Nest, which Spotify ended up buying this year. The Echo Nest gathers data using two systems: analysis of text on the Internet about specific music, as well as acoustic analysis of the songs themselves. The latter entails directing machines to listen for certain qualities in songs, like tempo, volume, and key.

We “apply knowledge about music listening in general and music theory and try to model every step of the way musicians actually perceive first and then understand and analyze music,” said Tristan Jehan, a cofounder of The Echo Nest and now principal scientist at Spotify.

That system required lots of domain-specific input on the part of the people who built it. But the Belgians’ deep learning approach, while also complex, is completely different.

“Sander [Dieleman] is just taking a waveform and assumes we don’t know that stuff, but the machine can derive everything, more or less,” Jehan said. “That’s why it’s a very generic model, and it has a lot of potential.”

The idea is to predict what songs listeners might like, even when usage data isn’t available.

So Dieleman pointed the system at Spotify’s data set of songs — without data on which songs Spotify users like — and came up with several playlists based on song similarity. The blog post generated plenty of interest; it’s even caused researchers to start using some of Dieleman’s methods in their own work.

The implementation isn’t a perfect substitute for the current approach at Spotify. But Jehan believes it’s worth pursuing.