Develop U-SQL scripts by using Data Lake Tools for Visual Studio
Learn how to use Visual Studio to create Azure Data Lake Analytics accounts, define jobs in U-SQL, and submit jobs to the Data Lake Analytics service. For more information about Data Lake Analytics, see Azure Data Lake Analytics overview.
- Visual Studio: All editions except Express are supported.
- Visual Studio 2017
- Visual Studio 2015
- Visual Studio 2013
- Microsoft Azure SDK for .NET version 2.7.1 or later. Install it by using the Web platform installer.
- A Data Lake Analytics account. To create an account, see Get Started with Azure Data Lake Analytics using Azure portal.
Install Azure Data Lake Tools for Visual Studio
Install Azure Data Lake Tools for Visual Studio 2017
Azure Data Lake Tools for Visual Studio is supported in Visual Studio 2017 15.3 or above. The tool is part of the Data storage and processing and Azure Development workloads in Visual Studio Installer. Enable either one of these two workloads as part of your Visual Studio installation.
Enable the Data storage and processing workload as shown:
Enable the Azure development workload as shown:
Install Azure Data Lake Tools for Visual Studio 2013 and 2015
Download and install Azure Data Lake Tools for Visual Studio from the Download Center. After installation, note that:
- The Server Explorer > Azure node contains a Data Lake Analytics node.
- The Tools menu has a Data Lake item.
Connect to an Azure Data Lake Analytics account
- Open Visual Studio.
- Open Server Explorer by selecting View > Server Explorer.
- Right-click Azure. Then select Connect to Microsoft Azure Subscription and follow the instructions.
- In Server Explorer, select Azure > Data Lake Analytics. You see a list of your Data Lake Analytics accounts.
Write your first U-SQL script
The following text is a simple U-SQL script. It defines a small dataset and writes that dataset to the default Data Lake Store as a file called
@a = SELECT * FROM (VALUES ("Contoso", 1500.0), ("Woodgrove", 2700.0) ) AS D( customer, amount ); OUTPUT @a TO "/data.csv" USING Outputters.Csv();
Submit a Data Lake Analytics job
Select File > New > Project.
Select the U-SQL Project type, and then click OK. Visual Studio creates a solution with a Script.usql file.
Paste the previous script into the Script.usql window.
In the upper-left corner of the Script.usql window, specify the Data Lake Analytics account.
In the upper-left corner of the Script.usql window, select Submit.
Verify the Analytics Account, and then select Submit. Submission results are available in the Data Lake Tools for Visual Studio Results after the submission is complete.
To see the latest job status and refresh the screen, click Refresh. When the job succeeds, it shows the Job Graph, MetaData Operations, State History, and Diagnostics:
- Job Summary shows the summary of the job.
- Job Details shows more specific information about the job, including the script, resources, and vertices.
- Job Graph visualizes the progress of the job.
- MetaData Operations shows all the actions that were taken on the U-SQL catalog.
- Data shows all the inputs and outputs.
- Diagnostics provides an advanced analysis for job execution and performance optimization.
To check job state
- In Server Explorer, select Azure > Data Lake Analytics.
- Expand the Data Lake Analytics account name.
- Double-click Jobs.
- Select the job that you previously submitted.
To see the output of a job
- In Server Explorer, browse to the job you submitted.
- Click the Data tab.
- In the Job Outputs tab, select the