Facebook’s data storage breakthrough: a huge box of Blu-ray discs

SAN JOSE, Calif. — When you think of Blu-ray, you think of movies on a disc. You don’t think of a data center. But Facebook executives saw beyond that minor detail.

Facebook has worked with a vendor to create a super-cheap, long-lasting, and downright cool storage vault that keeps files on Blu-ray discs. With 10,000 discs, each containing 100 GB, a single one of these cabinets has room for a petabyte — or 1,000 TB — of data.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":888827,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"big-data,cloud,enterprise,","session":"A"}']

It’s the sort of technology that befits a company of Facebook’s magnitude. It could help Facebook, but later on the model could be useful to large-scale cloud providers keen on cost-effectively storing companies’ data for the long haul.

That’s a bit far off, though. For now, Facebook is still planning to do production testing for the prototype in the third or fourth quarter of this year, Jason Taylor, Facebook’s director of hardware infrastructure, told VentureBeat in an interview.

“We do a lot of work with the data centers and with our data center techs to iterate on designs, and so they might have some ideas that could make it better, worse, whatever, Taylor said. “Depending on what changes we have and what we decide to do, if it’s minor tweaks, then it could be earlier. If there are more intense tweaks, it would be later.”

Regardless of when it gets deployed, the fact is, Facebook is on the leading edge when it comes to adding new tiers of storage infrastructure inside big data centers.

A few months ago, it switched on a “cold storage” facility at its first data center site, in Prineville, Ore., for storing pictures and other user media on hard disks, which are slower than the fast flash that much of Facebook’s freshest content depends on. Yesterday, Taylor and other Facebook executives talked about ongoing research into cold storage based on flash. Just one year ago, Facebook first hinted at experiments with Blu-ray storage, and today there’s a real prototype of technology that can be thought of as freezing-cold storage.

Few other companies make noise about their ongoing research in storage, much less share the designs with the rest of the world, and that’s what makes Facebook stand out with the Blu-ray vault.

The box is sort of like that six-CD changer in your old car — except it can hold tons more data. And the files on the discs will last a long, long time. Each disc is certified for 50 years of operation, although some might be capable of retaining information for 1,000 years, Taylor said. While it wouldn’t be the best choice for serving data rapidly in response to web queries, it would be a great option for data that needs to be around if copies of the same data on other storage gets ruined.

Inside the box, a robotic arm retrieves an individual Blu-ray disc and places it in a tray near the top of the unit, so a laser can read the data on the disc. Then the arm returns the disc to its hibernation spot. Because all disks except one are sitting still in the box at any one time, electricity use is tiny — about 200 watts.

[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":888827,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"big-data,cloud,enterprise,","session":"A"}']

“It’s like a light bulb, you know?” said Horst Schellong, president of the United States branch of HIT Archive Corp. His company collaborated with Facebook on the development of the prototype.

Storing data for the long-term on Blu-ray discs isn’t completely novel. Companies have been doing it for a few years now, ever since Blu-ray became a popular medium for delivering high-definition movies to consumers, Schellong said.

And before Blu-ray, there was storage on DVDs and CDs. Blu-ray is merely a more reliable medium for storage, and it can hold more than regular DVDs and CDs, Schellong said.

What makes Facebook’s appliance notable is its sheer scale, of holding a petabyte each. As Facebook executive Jay Parikh pointed out yesterday, seven of the boxes could hold all 4 million movies that have ever been made.

[aditude-amp id="medium2" targeting='{"env":"staging","page_type":"article","post_id":888827,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"big-data,cloud,enterprise,","session":"A"}']

Facebook could migrate data from its new cold storage hardware — pictures and other media from users of the social network — onto the even newer Blu-ray prototype. And because files can enjoy long lives on the Blu-ray discs, the prototype might be a good fit for data Facebook needs to hold onto in order to comply with legal regulations.

If Facebook finds success with the prototype, a few things could happen.

The technology could become part of Facebook’s open-source hardware initiative, the Open Compute Project, just as its other data center hardware has become open for others to check out and build to specifications.

Indeed, Schellong wants to see that happen. It would raise the profile of Blu-ray for archival purposes. Cloud providers might like the idea of buying the gear and offering Blu-ray-based very cold storage to their own customers, not least because it could be more cost-effective than other storage technologies, including tape.

[aditude-amp id="medium3" targeting='{"env":"staging","page_type":"article","post_id":888827,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"big-data,cloud,enterprise,","session":"A"}']

And if demand goes up, you’d better believe more suppliers will look at the specifications and draw up ways to do similar or better things for long-term storage of files.

All of that is speculation, of course. What we know now is that Facebook has a prototype of pretty impressive technology that can efficiently handle the output of millions and millions of people.