U-SQL Language Reference

  • Article

Introduction

This is the reference documentation for the U-SQL language.

What is U-SQL?

U-SQL is the new big data query language of the Azure Data Lake Analytics service.

It evolved out of Microsoft's internal Big Data language called SCOPE and combines a familiar SQL-like declarative language with the extensibility and programmability provided by C# types and the C# expression language and big data processing concepts such as “schema on reads”, custom processors and reducers. It also provides the ability to query and combine data from a variety of data sources, including Azure Data Lake Storage, Azure Blob Storage, and Azure SQL DB, Azure SQL Data Warehouse, and SQL Server instances running in Azure VMs.

It is however not ANSI SQL. For starters, its keywords such as SELECT have to be in UPPERCASE. And its expression language inside SELECT clauses, WHERE predicates etc is C#. This for example means, that the comparison operations inside a predicate follow C# syntax (e.g., a == "foo"), and that the language uses C# null semantics which is 2-valued and not 3-valued as in ANSI SQL. To help SQL users to get familiar with U-SQL, a section is providing the mapping of some common SQL expressions and how to express them in U-SQL.

Overview of the Language Reference

The language reference is organized as follows:

  1. This introduction provides a quick introduction and overview of the reference documentation.

  2. Introduction of the U-SQL concepts
    a. U-SQL Processing Model
    b. U-SQL Script Structure
    c. Files and File Sets as Inputs and Outputs
    d. Rowset: The Processing Data Structure
    e. U-SQL Identifiers
    f. U-SQL Expressions
    g. Metadata system and objects
    h. Extending U-SQL Expressions with User-Code

  3. Syntax and Semantics of the U-SQL Language
    a. Data Types and Literals
    b. Data Definition Language (DDL) Statements
    c. U-SQL Query Language
    d. Data Modification Language (DML) Statements
    e. Outputting to files with U-SQL
    f. Built-in Functions

  4. How to express common SQL expressions in U-SQL

If you are interested in seeing how you use the above concepts to solve certain scenarios please refer to the Language User Guide, tutorials as well as the U-SQL Programmability Guide.

See Also