Develop U-SQL scripts by using Data Lake Tools for Visual Studio

Learn how to use Visual Studio to create Azure Data Lake Analytics accounts, define jobs in U-SQL, and submit jobs to the Data Lake Analytics service. For more information about Data Lake Analytics, see Azure Data Lake Analytics overview.

Prerequisites

Install Azure Data Lake Tools for Visual Studio

Download and install Azure Data Lake Tools for Visual Studio from the Download Center. After installation, note that:

  • The Server Explorer > Azure node contains a Data Lake Analytics node.
  • The Tools menu has a Data Lake item.

Connect to an Azure Data Lake Analytics account

  1. Open Visual Studio.
  2. Open Server Explorer by selecting View > Server Explorer.
  3. Right-click Azure. Then select Connect to Microsoft Azure Subscription and follow the instructions.
  4. In Server Explorer, select Azure > Data Lake Analytics. You see a list of your Data Lake Analytics accounts.

Write your first U-SQL script

The following text is a simple U-SQL script. It defines a small dataset and writes that dataset to the default Data Lake Store as a file called /data.csv.

@a  = 
    SELECT * FROM 
        (VALUES
            ("Contoso", 1500.0),
            ("Woodgrove", 2700.0)
        ) AS 
              D( customer, amount );
OUTPUT @a
    TO "/data.csv"
    USING Outputters.Csv();

Submit a Data Lake Analytics job

  1. Select File > New > Project.

  2. Select the U-SQL Project type, and then click OK. Visual Studio creates a solution with a Script.usql file.

  3. Paste the previous script into the Script.usql window.

  4. In the upper-left corner of the Script.usql window, specify the Data Lake Analytics account.

    Submit U-SQL Visual Studio project

  5. In the upper-left corner of the Script.usql window, select Submit.

  6. Verify the Analytics Account, and then select Submit. Submission results are available in the Data Lake Tools for Visual Studio Results after the submission is complete.

    Submit U-SQL Visual Studio project

  7. To see the latest job status and refresh the screen, click Refresh. When the job succeeds, it shows the Job Graph, MetaData Operations, State History, and Diagnostics:

    U-SQL Visual Studio Data Lake Analytics job performance graph

    • Job Summary shows the summary of the job.
    • Job Details shows more specific information about the job, including the script, resources, and vertices.
    • Job Graph visualizes the progress of the job.
    • MetaData Operations shows all the actions that were taken on the U-SQL catalog.
    • Data shows all the inputs and outputs.
    • Diagnostics provides an advanced analysis for job execution and performance optimization.

To check job state

  1. In Server Explorer, select Azure > Data Lake Analytics.
  2. Expand the Data Lake Analytics account name.
  3. Double-click Jobs.
  4. Select the job that you previously submitted.

To see the output of a job

  1. In Server Explorer, browse to the job you submitted.
  2. Click the Data tab.
  3. In the Job Outputs tab, select the "/data.csv" file.

Next steps