This new WibiData software could help retailers sell far more stuff online

WibiData, a San Francisco startup building tools atop the Hadoop open-source software for storing and processing large quantities of various types of data, wants to help retailers lay out their websites in the smartest possible way. So WibiData is going beyond personalization through machine learning; it will begin offering a way for retailers to quickly test out several machine learning models and then go with the ones that provide the best results.

The new feature, dubbed Experiments by Wibi, enhances the retail-specific software WibiData released last year. But the overall concept at the root of the new feature from the startup — which developers, data scientists, and even marketers can all use — could turn out to be the next big way to extract major value out of all the data that companies are stowing away.

“A large majority of our technology is directly applicable to other industries as well,” WibiData cofounder and chief executive Christophe Bisciglia told VentureBeat in an interview.

Sure, companies with distributions of Hadoop — including Cloudera, which Bisciglia also cofounded — could stand to make money by selling to retailers and other companies that want to run recommendation systems with Hadoop, where log data and other unstructured data can be kept. But WibiData, which partners with Cloudera, among others, has built interfaces above the infrastructure layer that strip away the tedium that would otherwise stop many non-technical people from making decisions on data on the fly.

Experiments by Wibi employs a scientific approach in the sense that for every experiment, there’s a control group to compare against multiple “candidate models” data scientists build to draw on data to personalize content, Bisciglia said. Retailers can then send some share of web traffic to each model until there’s enough data to make statistically significant decisions.

Then the data scientists can use their own tools to determine which model leads to the best results, whether it be revenue, engagement, or some other metric.

In some situations, retailers agree to keep specific products from appearing next to competing products to keep manufacturers happy. Or maybe retailers want to surface a particular product or category in recommendations. Or perhaps a marketer would like to try out an experiment based on a whim. In such cases, people without technical skills can set rules or override them, Bisciglia said.

Altogether, the new system from WibiData stands out for giving several types of people the power to take advantage of big data, and fast.

“Now you’ve got marketing and data scientists nimbly experimenting behind the scenes,” as Bisciglia put it.