More collaboration, more standards on deck for big data, Pivotal exec says

REDWOOD CITY, Calif. — Pivotal executive Scott Yara takes heart from companies laying out billions for big data and analytics technologies.

“I haven’t seen a healthier funding environment for these kinds of technologies ever, which is really neat to see,” Yara, Pivotal’s senior vice president of products, said in his keynote at VentureBeat’s DataBeat/Data Science Summit event today.

The glory days of big data are here, and we’re perhaps a decade into the first wave of the resurgence of data and analytics, said Yara, a founder of Greenplum, a big data analytics company that EMC acquired in 2010.

Yara is looking forward to the future of data science. He has a few predictions of trends that he expects to take hold in the near future:

• Strong standards should emerge. “The role of standards have kind of come and gone over the last 20 years,” Yara said, mentioning the IEEE and W3C as examples of governing bodies. Standards are needed, particularly as more companies are commercializing open-source technologies.Hadoop distribution vendors, hosted database providers, and even Yara’s own company deal in this sort of strategy. “Our most critical technology investments are all based on open-source technologies,” Yara said. Standards can help make sure that anyone can contribute to open-source projects, Yara said.
• Collaboration across firms, countries, and other groups will become more important across companies. That way, there will be less unnecessary duplication of efforts, with employees at many companies collecting data in silos.”We think collaboration is going to be this huge new opportunity for all of us around data,” he said. “There needs to be a common point to sort of collaborate around data.”At the same time, Yara said he’s interested in seeing how collaboration can get more popular without sacrificing privacy.
• Software will become even more critical. Pivotal executives have been vocal on how data and analytics can help developers build better applications, and Yara showed his belief in this during his keynote. Data science and applications are “really hitched together forever,” he said.Reworking applications based on insights that come from data is part of what Yara called “a closed loop of innovation” — just the sort of thing that resounds with enterprises that want to move as fast as startups. The faster companies cycle through application development, collection of data, and analysis of data, the faster innovation, disruption, and problem solving can happen, he said.

Overall, Yara foresees big data playing a larger role for role in business technology.

“It’s a very, very unsettling time for most enterprise IT organizations,” he said. “They know that they’re under huge change. … How do you innovate while maintaining cost controls actually, acquiring talent, know-how, and methodologies, where you can do all these things?”

Those are hard problems, but these are problems Pivotal is trying to solve.