U-SQL Language Reference
This is the reference documentation for the U-SQL language.
What is U-SQL?
U-SQL is the new big data query language of the Azure Data Lake Analytics service.
It evolved out of Microsoft's internal Big Data language called SCOPE and combines a familiar SQL-like declarative language with the extensibility and programmability provided by C# types and the C# expression language and big data processing concepts such as “schema on reads”, custom processors and reducers. It also provides the ability to query and combine data from a variety of data sources, including Azure Data Lake Storage, Azure Blob Storage, and Azure SQL DB, Azure SQL Data Warehouse, and SQL Server instances running in Azure VMs.
It is however not ANSI SQL. For starters, its keywords such as SELECT have to be in UPPERCASE. And its expression language inside SELECT clauses, WHERE predicates etc is C#. This for example means, that the comparison operations inside a predicate follow C# syntax (e.g., a == "foo"), and that the language uses C# null semantics which is 2-valued and not 3-valued as in ANSI SQL. To help SQL users to get familiar with U-SQL, a section is providing the mapping of some common SQL expressions and how to express them in U-SQL.
Overview of the Language Reference
The language reference is organized as follows:
This introduction provides a quick introduction and overview of the reference documentation.
Introduction of the U-SQL concepts
a. U-SQL Processing Model
b. U-SQL Script Structure
c. Files and File Sets as Inputs and Outputs
d. Rowset: The Processing Data Structure
e. U-SQL Identifiers
f. U-SQL Expressions
g. Metadata system and objects
h. Extending U-SQL Expressions with User-Code
Syntax and Semantics of the U-SQL Language
a. Data Types and Literals
b. Data Definition Language (DDL) Statements
c. U-SQL Query Language
d. Data Modification Language (DML) Statements
e. Outputting to files with U-SQL
f. Built-in Functions
- In order to see how the above concepts can be applied to solve certain scenarios, please refer to the following:
- For additional information please refer to:
- Release Notes
- U-SQL C# Developer’s Guide (under development)
- U-SQL Best Practices and Optimizations (under development)
- U-SQL Git Repo
- Azure Data Lake
- Get Started with Azure Data Lake Analytics using Azure portal