Google launches Cloud Dataflow, Cloud Pub/Sub out of preview

Google today announced the general availability of two of its cloud services for handling big data: Cloud Dataflow and Cloud Pub/Sub.

Google first talked about both of those services at the Google I/O developer conference in June 2014. Cloud Dataflow, which can perform complex computations like extract-transform-load (ETL) on large quantities of data in batches or in streaming mode, became available in beta in April. Cloud Pub/Sub, a tool that can send and receive data to and from applications in the form of “messages,” came out in beta in March.

Google has thus had time to battle-test these services and make sure they’re ready for anything.

“A decade of internal innovation also stands behind today’s general availability of Google Cloud Pub/Sub,” Cloud Dataflow product manager Eric Schmidt and Cloud Pub/Sub product manager Rohit Khare wrote in a blog post on the news today. “Delivering over a trillion messages for our alpha and beta customers has helped tune our performance, refine our v1 API, and ensure a stable foundation for Cloud Dataflow’s streaming ingestion, Cloud Logging’s streaming export, Gmail’s Push API, and Cloud Platform customers streaming their own production workloads — at rates up to 1 million message operations per second.”

These services fit into the big data section of the Google Cloud Platform portfolio. And in the public cloud, Google is looking for ways to stand out from competitors, including Microsoft Azure and market leader Amazon Web Services. Articulating that these services are Google-scale — Pub/Sub’s use in an API for Gmail, for example — is certainly one way to stand out.

Google has also been pushing cloud prices lower and lower. But today, the emphasis is on performance and scale.