Cloudera's Impala tool binds Hadoop with business intelligence apps

In traditional circles, Hadoop is viewed as a bright but unruly problem child.

Indeed, it is still in the nascent stages of development. However the scores of “big data” startups that leverage Hadoop will tell you that it is here to stay.

Cloudera, the venture-backed startup that ushered the mainstream deployment of Hadoop, has unveiled a new technology at the Hadoop World, the data-focused conference in New York.

Its new product, known as “Impala”, addresses many of the concerns that large enterprises still have about Hadoop, namely that it does not integrate well with traditional business intelligence applications.

“We have heard this criticism,” said Charles Zedlewski, Cloudera’s VP of Product in a phone interview with VentureBeat. “That’s why we decided to do something about it,” he said.

Impala enables its users to store vast volumes of unwieldy data and run queries in HBase, Hadoop’s NoSQL database. What’s interesting is that it is built to maximise speed: it runs on top of Hadoop storage, but speaks to SQL and works with pre-existing drivers.

“If you ask questions, you will get answers ten times faster,” Zedlewski explained.

These answers or “insights” can mitigate risk and inform strategy for a company’s sales, marketing and business development teams.

Travel booking site Expedia is one of the companies that is using Cloudera’s new product to manage four pedabytes of company data.

“We are able to work on one single platform for big data rather than many disparate systems for archiving, ETL and analytics,” said Jeff Prather, Director of Global Business Intelligence and Data Warehousing Platforms at Expedia.

“This evolution of Hadoop has enabled us to reduce our latency by 50 percent and produce a new real business insight service not previously viable,” said Prather in a statement.

Cloudera burst onto the scene four years ago. It was among the first to capitalize on the growing popularity of the open source Hadoop framework. (Fun fact: Hadoop was invented by independent programmer Doug Cutting, and shortly thereafter was brought to Yahoo. It was named after his son’s top elephant.)

“The reason why people are attracted to it [Hadoop] is because it’s scalable, extremely flexible, easy to get started with, and inexpensive compared to what else is out there,” said Zedlewski.

Cloudera’s founders used Hadoop for the purposes of data analysis and business intelligence. For developers, Hadoop is free to download under the Apache software license, but the company sells a broad spectrum of technical support services.

The Silicon Valley-based company has signed on business intelligence partners to test the product, including Tableau and QlikTech. (Learn more in our inaugural list of top “big data” companies). It will bring in revenue through this new product by charging customers for support services.

Cloudera has 400 employees and has over $76 million of venture funding in the bank. Cloudera’s chief scientist, Jeff Hammerbacher, conceived and led Facebook’s data team.