Microsoft launches Azure Data Lake Analytics with a new query language, U-SQL

Microsoft today introduced the launch of Azure Data Lake Analytics, a new cloud-based service for running queries on big data stored in the Microsoft’s growing public cloud. It uses a new Microsoft-built query language that draws from both the widely used SQL query language and Microsoft’s C# programming language.

Microsoft has also given a new name to the Azure Data Lake for storing big data. From now on, the tool will be called Azure Data Lake Store.

Azure Data Lake Analytics is a significant move for Microsoft, which at a high level is keen on building “the intelligent cloud platform.”

At the core of the new service is the new U-SQL query language. By incorporating elements of C#, Microsoft is taking advantage of its large base of programmers. That can help Microsoft Azure distinguish itself from other public cloud providers with data services, including market leader Amazon Web Services, as well as Google Cloud Platform and IBM SoftLayer.

T. K. “Ranga” Rengarajan, corporate vice president of Microsoft’s Data Platform, explained the logic behind U-SQL in a blog post today:

We know that many developers and data scientists struggle to be successful with big data using existing technologies and tools. Code-based solutions offer great power, but require significant investments to master, while SQL-based tools make it easy to get started but are difficult to extend. We’ve faced the same problems inside Microsoft and that’s why we introduced, U-SQL, a new query language that unifies the ease of use of SQL with the expressive power of C#.

The Azure Data Lake Store works with the Hadoop Distributed File System (HDFS), allowing for integration with third-party Hadoop distributions, and it works with Azure Active Directory, Rengarajan wrote.

Michael Rys, principal program manager for Microsoft Big Data, has a deep dive blog post on U-SQL here.

Above: Working with U-SQL in Microsoft’s Azure Data Lake.

Image Credit: Microsoft

Microsoft is also announcing that managed Linux-based Hadoop clusters are now generally available in the HDInsight cloud service, which is part of the more general Azure Data Lake.

And the Azure Data Lake Tools for Visual Studio now offers developers new debugging and authoring capabilities, with support for the Apache Hive data warehousing software.