Intel partners with Databricks to push adoption of the Spark data-processing engine

Intel today announced that it’s throwing support behind hot big-data startup Databricks, which provides support for the trendy Apache Spark open-source project for processing and analyzing large data sets.

Intel will also work with AMPLab at University of California, Berkeley, from which Spark and Databricks emerged.

Of course, Intel has an interest in making sure Spark, which quickly performs computations in memory, works well with Intel chips.

“With Databricks, we plan to demonstrate the efficiency of running Spark-based analytics on Intel Architecture-based platforms to datacenter owners using benchmarks and technology education,” Michael Greene, Intel’s vice president of Intel’s software and services group and its general manager of system technologies and optimization, wrote in a blog post on the news. He also spoke on the topic today at the Strata + Hadoop World Conference in San Jose, California.

But the bigger picture is that Spark is getting more traction from a big IT vendor as a serious system for handling big data, potentially as an alternative to the more widely deployed Hadoop open-source software.

Meanwhile, Intel last year demonstrated in a big way that it’s interested in seeing the growth of Hadoop. The chipmaker, which at one point had its own Hadoop distribution, invested hundreds of millions of dollars into leading Hadoop distribution vendor Cloudera. And Cloudera, in fact, has moved to support Spark.

Intel is not formally investing in Databricks, a Databricks spokeswoman told VentureBeat. But the partnership nevertheless is a good indicator for Databricks, which is at the forefront of the Spark world. Other startups embracing Spark include H2O and MemSQL.

Databricks last year took a $34 million funding round.