Hortonworks buys Onyara, the company behind Apache NiFi, which was developed at NSA

Hortonworks, a publicly traded company selling a commercial distribution of the Hadoop open-source big data software, announced today that it has acquired Onyara, an early-stage startup whose employees developed Apache NiFi, a piece of open-source software that was first used inside the National Security Agency (NSA).

Terms of the deal weren’t disclosed. The startup began in March and lists 10 employees on its website.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":1790755,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"big-data,business,dev,enterprise,","session":"A"}']

Apache NiFi (pronounced NIGH-figh) is based on Niagarafiles, a piece of software the NSA first created in 2006 to deliver sensor data to the right systems and keep track of what was happening to the data.

“[It] became a fairly important piece of technology,” Onyara cofounder and chief technology officer Joe Witt told VentureBeat in an interview. Then, last year, the NSA moved to release the software under an open-source license.

Hortonworks, which itself spun out of Yahoo, has previously acquired XA Secure and SequenceIQ. Now Hortonworks will be selling a new subscription based on the Apache NiFi software, under the name Hortonworks DataFlow. Companies can use DataFlow to collect data from many sources and send it into the Hortonworks Data Platform (HDP), which is based on Hadoop, for long-term storage. HDP enables companies to run queries on great supplies of historical data, while DataFlow will provide brief but up-to-the-minute information.

NiFi itself goes beyond messaging software, such as Apache Kafka, as well as enterprise service bus tools, such as IBM Integration Bus, said Witt, who is one of NiFi’s seven initial committers.

“Fundamentally, what we’re saying is the data broker needs to be more intelligent,” Witt said.

This isn’t the first time NSA technology has been commercialized. Sqrrl has been pushing a commercial distribution of the Apache Accumulo key-value store that the NSA originally developed.

In recent months, the company has found interest in the health care, financial services, and gaming industries, Witt said. Clearly, then, organizations other than the NSA were encountering the same problems that the NSA had.

“The problem space we’re going after is data flow,” Witt said. “That is inherently applicable to a lot of industries.”

[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":1790755,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"big-data,business,dev,enterprise,","session":"A"}']

Hortonworks went public in December.

Here’s a video of Witt giving a presentation of NiFi at OSCON 2015 last month: