Develop U-SQL scripts by using Data Lake Tools for Visual Studio
Azure Data Lake and Stream Analytics Tools include functionality related to two Azure services, Azure Data Lake Analytics and Azure Stream Analytics. For more information on the Azure Stream Analytics scenarios, see Azure Stream Analytics tools for Visual Studio.
This article describes how to use Visual Studio to create Azure Data Lake Analytics accounts, define jobs in U-SQL, and submit jobs to the Data Lake Analytics service. For more information about Data Lake Analytics, see Azure Data Lake Analytics overview.
Microsoft recommends you upgrade to Azure Data Lake Tools for Visual Studio version 2.3.3000.4 or later. The previous versions are no longer available for download and are now deprecated.
What do I need to do?
- Visual Studio: All editions except Express are supported.
- Visual Studio 2017
- Visual Studio 2015
- Visual Studio 2013
- Microsoft Azure SDK for .NET version 2.7.1 or later. Install it by using the Web platform installer.
- A Data Lake Analytics account. To create an account, see Get Started with Azure Data Lake Analytics using Azure portal.
Install Azure Data Lake Tools for Visual Studio
This tutorial requires that Data Lake Tools for Visual Studio is installed. Follow the installation instructions.
Connect to an Azure Data Lake Analytics account
Open Visual Studio.
Open Server Explorer by selecting View > Server Explorer.
Right-click Azure. Then select Connect to Microsoft Azure Subscription and follow the instructions.
In Server Explorer, select Azure > Data Lake Analytics. You see a list of your Data Lake Analytics accounts.
Write your first U-SQL script
The following text is a simple U-SQL script. It defines a small dataset and writes that dataset to the default Data Lake Store as a file called
USE DATABASE master; USE SCHEMA dbo; @a = SELECT * FROM (VALUES ("Contoso", 1500.0), ("Woodgrove", 2700.0) ) AS D( customer, amount ); OUTPUT @a TO "/data.csv" USING Outputters.Csv();
Submit a Data Lake Analytics job
Select File > New > Project.
Select the U-SQL Project type, and then click OK. Visual Studio creates a solution with a Script.usql file.
Paste the previous script into the Script.usql window.
In the upper-left corner of the Script.usql window, specify the Data Lake Analytics account.
In the upper-left corner of the Script.usql window, select Submit.
After the job submission, the Job view tab opens to show the job progress. To see the latest job status and refresh the screen, click Refresh.
- Job Summary shows the summary of the job.
- Job Graph visualizes the progress of the job.
- MetaData Operations shows all the actions that were taken on the U-SQL catalog.
- Data shows all the inputs and outputs.
- State History shows the timeline and state details.
- AU Analysis shows how many AUs were used in the job and explore simulations of different AUs allocation strategies.
- Diagnostics provides an advanced analysis for job execution and performance optimization.
Check job status
In Server Explorer, select Azure > Data Lake Analytics.
Expand the Data Lake Analytics account name.
Select the job that you previously submitted.
See the job output
In Server Explorer, browse to the job you submitted.
Click the Data tab.
In the Job Outputs tab, select the