Paying for the petabyte future, one gigabyte at a time

Cloud storage is easy, but if you’ve got a lot of data, the costs can escalate quickly.

Case in point: DNAnexus, a startup that is storing DNA data so researchers can access and analyze it through a web interface, is using Amazon Web Services (AWS) to power its applications, and Amazon Simple Storage Service (S3) to store data.

The company now has half a petabyte of data stored in the cloud, at a cost of 11 to 12 cents per gigabyte per month. Additionally, Amazon charges 11 to 12 cents per gigabyte for data downloads. That adds up to a total cost of tens of thousands of dollars just for storage.

Like an old-school dime store hooking kids on nicotine with free cigarettes, Amazon makes it easy to get started.

“Amazon has made it very attractive to send data to them — they don’t charge for that. But they do charge when you want to get that data out of the cloud,” said Andreas Sundquist, the CEO and co-founder of DNAnexus.

Add up the costs, and Amazon looks like it’s about the same price as local storage.

“I can go to Fry’s and get a hard drive — and that’s what it costs to store data on Amazon for a month,” said Sundquist.

Despite the costs, it’s beneficial to have the data stored closely to the computing and web resources that Amazon offers. AWS has given DNAnexus flexibility and computing power it couldn’t have easily accessed otherwise.

Sundquist spoke on a panel discussion at CloudBeat 2011 today with vendors from four storage providers: Crossroads, Coraid, Scality and Whiptail.

The consensus of the panel: Initial costs, also known as capital expenditures (CapEx, or what you’d spend at Fry’s for that hard drive) are only part of the story. For large-scale enterprise storage, you’ve also got to account for network management, storage management, maintaining all those storage servers, ensuring that your storage area network is compatible with the rest of your applications, and so on. Therefore, the cost per gigabyte of raw storage is misleading.

“It’s a metric that doesn’t make any sense,” said Jerome Lecat, the CEO of Scality.

For that reason, it’s smarter to focus on the total cost of ownership per month, and in that light, Amazon doesn’t look like such a bad deal.

“Amazon is 70 percent cheaper than what legacy companies do, typically,” said Forrester analyst Andrew Reichman, who has studied the comparative costs of different storage solutions. “The biggest difference is you don’t need to pay for excess capacity. You only pay for what you do send to Amazon.”

For instance, if you have half a petabyte of data, you’d actually need to purchase 2 petabytes of storage capacity, in most cases, Reichman said. Cloud storage gives you the ability to pay for just what you use.

However, Amazon’s costs are higher than some of the alternatives. If you have the ability to build your own storage system in a datacenter, the companies on the panel are all able to offer you cost-effective alternatives so you can build your own cloud storage.

• Crossroads specializes in making tape storage work with modern storage systems, by essentially making the tape drive act like a USB drive. Because tape can be stored without requiring power — all you need is more cabinet space — it’s particularly economical for vast quantities of data. Crossroads storage can cost less than 1 cent per GB per month, or 1/10 the cost of Amazon, especially at the largest scale (exabytes of data).
• Coraid uses commodity hardware and Ethernet cabling to replace expensive FibreChannel and storage arrays. Using these pieces, it was able to sell the U.S. Department of Defense a $1 million storage contract with a “buy one petabyte, get the second petabyte free” deal. Coraid storage has a total cost of ownership of 5 cents per GB per month, or about half what Amazon charges.
• Scality offers software that makes storage networks work more efficiently, making it easier to add more capacity as needed — like “building an Amazon S3 at home,” Lecat said. Scality-based storage also costs about 5 cents per GB per month, all-in.
• Whiptail creates solid state storage-based systems that are more expensive than traditional spinning hard disks, but which are extremely fast. While it doesn’t charge by GB per month, its total cost of ownership works out to about 10 to 12 cents per GB per month for the largest scale installations.

Despite the advantages, it seems that DNAnexus will be sticking with the public cloud for now.

“The nice thing about Amazon is, I don’t have to think about what sort of storage device it is stored on. If I need to double my storage, I just do it,” said Sundquist.

Photo credit: Sean Ludwig/VentureBeat. Panelists, left to right: David Cerf, Crossroads; Kevin Brown, Coraid; Jerome Lecat, Scality; Dan Crain, Whiptail Technologies; Andreas Sundquist, DNAnexus; Andrew Reichman, Forrester Research.