Analytics and visualization samples for Microsoft Academic Graph

Illustrates how to perform analytics and visualization for Microsoft Academic Graph using Data Lake Analytics (U-SQL) and Power BI.

Prerequisites

Complete these tasks before you begin this tutorial:

Gather the information that you need

Before you begin, you should have these items of information:

✔️ The name of your Azure Storage (AS) account containing MAG dataset from Get Microsoft Academic Graph on Azure storage.

✔️ The name of your Azure Data Lake Analytics (ADLA) service from Set up Azure Data Lake Analytics.

✔️ The name of your Azure Data Lake Storage (ADLS) from Set up Azure Data Lake Analytics.

✔️ The name of the container in your Azure Storage (AS) account containing MAG dataset.

Create database from MAG data

In prerequisite Set up Azure Data Lake Analytics, you added the Azure Storage (AS) created for MAG provision as a data source for the Azure Data Lake Analytics service (ADLA). In this section, you submit an ADLA job to create database from MAG data.

  1. In the Azure portal, go to the Azure Data Lake Analytics (ADLA) service that you created, and select Overview > New Job.

    Azure Data Lake Analytics - New job

  2. Copy code in samples/CreateDatabase.usql and paste into the code block.

  3. Provide a Job name and select Submit.

    Submit CreateFunctions job

  4. The job should finish successfully.

Sample projects

Getting started with sample projects

  1. Download or clone the samples repository
  2. Open the solution /src/AcademicAnalytics.sln
  3. For each tutorial there should be: A U-SQL script(.usql), a Power BI report (.pbix), a Power BI template (.pbit) and a README explaining the tutorial.
  4. Although each tutorial is different, running the U-SQL script as is and filling out the Power BI template using the same U-SQL parameters should give you a Power BI report with visualizations that match the Power BI report example included in the tutorial. Since the Microsoft Academic graph is contently improving, different graph versions may give you slightly different results.

Working with U-SQL scripts

  • How to run U-SQL scripts

    • Make sure you have selected your Data Lake account

      Select your Data Lake account in Visual Studio

    • Build the script first to validate syntax

      Build U-SQL script in Visual Studio

    • Submit your script to your Data Lake account

      Submit U-SQL script in Visual Studio

  • How to view U-SQL results in Azure portal

    Navigate to the overview tab and click data explorer

    Navigate to the output directory specified in the U-SQL script

Using Power BI

Important

Make sure U-SQL script finished successfully

  • Open up corresponding Power BI Template (.pbit) from file explorer (Visual studio doesn't recognize Power BI files)

  • Enter your ADL information and parameters corresponding to your scripts

    Load sample Power BI script template

  • Make sure the parameters cases are the same as your script and "click" to load

Resources