Exploring Big Data: Course 6 – Processing Big Data with Azure Data Lake Analytics

  • Article

Big Data with Sam Lester

(This is course #6 of my review of the Microsoft Professional Program in Big Data)

Course #6 of 10 – Processing Big Data with Azure Data Lake Analytics

Overview: The “Processing Big Data with Azure Data Lake Analytics” course is comprised of four sections of videos and labs, where the lab documentation walks you through the creation of an Azure Data Lake Store, populating the store with data, creating database objects to store the data, and finally in querying the data to answer lab questions. The final challenge is similar in structure but does not provide the detailed code syntax, leaving that exercise to you to figure out. 

Time Spent / Level of Effort: This was one of the shorter courses I’ve taken through Edx, with the entire course taking only 3-4 hours. The four video and lab sections are very easy to follow, with step-by-step instructions to create the required U-SQL jobs to load data into the Azure Data Lake Store, create databases, schemas, and queries.

Course Highlight: I have intended to spend time learning more about U-SQL and this was a perfect course in that regard. I’ve been writing TSQL code and .NET code for many years, so U-SQL seems like a great subject for me to explore. It is interesting to see U-SQL use the C# data types and the ability to call C# methods on T-SQL objects, such as “SELECT Customer.ToUpper() FROM DB1”. I really like the idea of leveraging the power of both technologies over mountains of data within Azure Data Lakes and hope that I’ll get the chance to do more work with U-SQL soon.

Suggestions: The final lab challenge for this course was different than previous exams in the MPP in that it gave high-level instructions and left the details up to you to figure out. Fortunately, the steps in the final challenge are like those found in the first four lab exercises, so you’ve already created similar jobs that you can reference. To reuse the script from a previously created job (example: Create Table job from Lab 2), open the job and click on “Reuse Script”. This will open the script syntax in a new window and you can edit the query to reflect the task of the final challenge. By reusing the jobs created in the labs, the final challenge was much easier.

 

Reuse Script Azure Data Lake

 

Also, in the final challenge, I noticed the "Hint" button in the bottom right corner of the quiz questions. The hints do not affect your score, so use these if you need some extra suggestions to answer the questions.Edx.org Show Hint

If you have taken this course in the past or are going through it now, please leave a comment and share your experience.

Thanks,
Sam Lester (MSFT)