Google, Spotify, & Pandora bet a computer could generate a better playlist than you can

Head-bobbing robots everywhere

The interest goes beyond Spotify. Even Pandora, known for its use of humans in its process of finding songs to play, has been exploring the technique.

Pandora’s musicologists identify attributes in songs based on their knowledge of music. The end product is data that they can feed into complex algorithms — but it fundamentally depends on human beings. And people like Schmidt think that’s actually a great thing.

“I think to some degree that there’s always some superiority to the human ear,” Schmidt said. “We can get absolute ground truth by having musicologists label these things. There’s a level of accuracy, generally, that’s absolute when you have humans, and it’s something that we can really lean on.”

The human-generated data feeds into a system befitting a company with a $3.86 billion market cap. Pandora’s data centers retain an arsenal of more than 50 recommendation systems. “No one of these approaches is optimal for every station of every user of Pandora,” Schmidt said.

Above: Pandora’s team of music analysts.

Image Credit: Pandora

Proving a recommendation system’s advantages over those that have already had successful runs in production can be a challenge.

“If we move to something like deep learning, what does that deliver in terms of improving performance?” Schmidt said. “What we’ve seen is, in some cases, it does, but in many cases, we’ve been tremendously successful with simpler models.”

Meanwhile, deep learning has come in handy for a wide variety of purposes at Google, and employees certainly are investigating its applications in a music-streaming context. Doug Eck, a staff research scientist focusing on music technology at Google, wouldn’t say if deep learning were in place for, say, Google Play Music.

Still, he believes “passionately that deep learning represents a complete revolution, literally a revolution, in how machine learning is done,” he said.

The trouble is, deep learning on its own might do a good job of detecting similarities among songs, but maximizing outcomes might mean drawing on several kinds of data other than the raw content. Think upcoming events, or record labels — data that Google already has readily available in the Freebase database.

“What we see from these deep-learning models, including the best of best we’ve seen — very similar to what Aaron and Sander did — is there’s still a lot of noise out there in these models,” Eck said. Weeding out peculiar song recommendations becomes more important.

And so deep learning might not be a sort of drop-in replacement for music streaming. It can be another tool, and perhaps not only for determining which song to play next. Its capabilities could go beyond that.

“What I do see is that deep learning is allowing us to better understand music and allow us to actually better understand what music is,” Eck said. And that’s especially true when you consider its ability to analyze many seconds of music.

Above: At the Googleplex in Mountain View, Calif.

Image Credit: Jordan Novet/VentureBeat

From there, Eck said, Google could take what it can learn about music and “build better products, like a better streaming service.”

But even at Google, computational mechanisms must be judged by their effectiveness in terms of the most important measurements, like how much music users listen to. And making major strides by improving the experience so they listen, and do so longer, isn’t easy.

“This is a great challenge,” Eck said.

The result: You, too, become a music snob

Above all else, one thing deep learning could be useful for is exposing people to new music, or to music that is currently unpopular but which they might like if they only heard it.

Fundamentally, especially for those systems that depend on people to analyze music, introducing listeners to new music remains a challenge. Pandora, for instance, has had the luxury of time to enter more genres and accumulate rich data on each, but there is always new music, and thus there is always music that Pandora hasn’t accounted for.

Pandora needs “to figure out and find relevant tracks and be able to deliver them from the first spin,” Schmidt said. Which is why the company is experimenting with machine listening, including what Schmidt called “deep listening.”

And it’s not just about making things better for music fans. Consider the artists: Deep learning might just be able to help them, too.

“I think one of the struggles business-wide all music services have is all indie artists complain they are not paid and so on,” said Alexandre Passant, the cofounder and chief executive of music-discovery startup Seevl. “If you can find a way to surface new artists in algorithms, that’s a win-win solution for everyone.”

So deep learning amounts to one of those technologies that several companies could start to implement in the future in order to improve music streaming.

“In the future, we want to be ahead of everybody else when that stuff is ready, but I think it’s getting there, slowly,” Jehan said. “So we’re going to keep working on that stuff, for sure. It’s going to be tested, and it’s going to be incorporated as soon as results are good.”