How Joyent's cloud helps Wanelo analyze its big data

SAN FRANCISCO — As traffic swelled at online social store Wanelo, the relational database PostgreSQL became insufficient for logging and understanding what users were doing on the site, forcing the company to think about other options. Wanelo’s chief technology officer, Konstantin Gredeskoul, talked about the challenge on stage today at CloudBeat.

Data was coming in so fast that “at that time, you give up on trying to analyze the data,” he said, “because we’ll figure it out later. Let’s just start storing it. And that’s when we started hearing about Joyent’s Manta.”

Manta is an analysis tool for small businesses and was introduced by Infrastructure-as-a-Service (IaaS) provider Joyent in June. It represents a new direction for handling large swaths and a wide variety of data in public clouds. It performs the computational work right where the data sits — think of Amazon Web Services’ EC2 computation service sitting on top of the S3 object-storage service. The model speeds up jobs, enabling processing that’s closer to real time and allowing for novel types of computing in the cloud.

And it was a perfect fit for Wanelo, Gredeskoul said. “As soon as Manta became available (in beta), we started writing basic little scripts [to] extract and analyze data out of these files, instead of doing it in a single file. We were able to do it in parallel, across many, many nodes.”

With Manta, Wanelo was able to run hard calculations in five to seven seconds, according to a case study.

Manta supports the use of long-standing Unix commands for working with data. That makes sense if you listen to Bryan Cantrill, Joyent’s senior vice president of engineering.

“Unix dominates the cloud,” Cantrill said. “Unix won. And Unix won because it does allow you to construct these systems very quickly, very easily, in a very well understood way.”

That’s not all, Gredeskoul noted. It supports many languages popular among developers, including R, Ruby and Python. They’re “preinstalled for you,” he said. “If what you’re pushing to Manta is binary objects that R understands, you are welcome to write one or two lines to process R.”

Joyent received acclaim from some cloud commentators after announcing Manta and received recognition for it in Gartner’s latest major report on Infrastructure as a Service (IaaS) clouds. But Joyent hasn’t come out with major customer references for Manta since then. However, Cantrill said he knows many developers are using the service for large jobs and he expects it will take time for the model to take hold.

Yesterday, Joyent announced a significant executive shakeup with the departure of Joyent cofounder and chief technology officer Jason Hoffman. Joyent’s other cofounder, David Young, left last year.

As for Hoffman, he’s got some ideas about what to do next, he wrote in an email to VentureBeat:

I am just really wanting to get back into the health world and there’s currently an interesting intersection happening where mobile, m2m, wearables and ‘big data’ can potentially have large impacts human health. And all that together is really my wheelhouse.

Cantrill will take over the CTO responsibilities, he told VentureBeat.

Even with Hoffman moving on, the company has a nice innovation in Manta that other IaaS providers could copy if it becomes more popular.