Facebook details its company-wide machine learning platform, FBLearner Flow

Facebook today is talking for the first time about FBLearner Flow, a piece of software that manages machine learning models for employees throughout the social networking company.

It’s similar in some ways to cloud-based machine learning services like Microsoft’s Azure Machine Learning, or the open-source Airflow from Airbnb, but it’s optimized for Facebook’s own infrastructure and filled with algorithms developed by Facebook engineers, so that many other people inside the company can make use of them for their own models.

“More than 25 percent of the company are using it, in scenarios that we haven’t really envisioned before,” Hussein Mehanna, director of Facebook’s Core Machine Learning Group, told VentureBeat in an interview.

Facebook could go beyond just publishing academic papers on FBLearner Flow; the company may well end up open-sourcing it. Facebook engineers have discussed this type of technology with people at LinkedIn, Twitter, and Uber, and they’ve all been excited about it, Mehanna said.

It would be an open source option that has been proven to work at Facebook scale, and it wouldn’t cost money, unlike data science collaboration tools from the likes of Domino Data Lab and Yhat.

Facebook regularly builds and sometimes open-sources technology that’s perfect for its own needs, so it would not be a surprise to see that happen to FBLearner Flow, too. It’s just a question of whether Facebook wants to devote the engineering resources to decoupling the technology from Facebook’s proprietary systems.

Then again, it wouldn’t be surprising to see Facebook keep FBLearner Flow to itself, given how much more productive employees have become, thanks to the technology, since engineers started building the new system in late 2014. It could be a competitive advantage. The new system lets people see past experiments and results, and even combine two separate machine learning pipelines to come up with something more complex. And a graphical user interface allows non-engineers to use it — hence Mehanna’s 25 percent figure.

Perhaps in the future FBLearner Flow could become even more powerful if idle compute resources could be made to improve machine learning models, Mehanna said. So Facebook has been investing in an area known as AutoML, which essentially makes a bot go to work for Facebook’s own purposes. It could even be pointed at data to unearth the best possible machine learning model for it. Which, when you think about it, is a little meta and potentially very impactful.

For much more detail on FBLearner Flow, check out today’s blog post from Facebook software engineer Jeffrey Dunn.