Access SQL Data Warehouse instances with Azure Databricks

Intermediate
Developer
Data Engineer
Data Scientist
Azure
Databricks

Learn how to access Azure SQL Data Warehouse from Azure Databricks by using the SQL Data Warehouse connector. This allows you to use Apache Spark with Azure Blob storage and PolyBase in SQL Data Warehouse to efficiently transfer large volumes of data between a Databricks cluster and a SQL Data Warehouse instance.

In this module, you will:

  • Access Azure SQL Data Warehouse from Azure Databricks by using the SQL Data Warehouse connector.
  • Use Apache Spark with Azure Blob storage and PolyBase in SQL Data Warehouse to efficiently transfer large volumes of data between a Databricks cluster and a SQL Data Warehouse instance.

Prerequisites

You'll need an Azure subscription. If you don't have an Azure subscription, create an account and add a subscription before you begin. The Azure free trial subscription type will not work with Databricks, but the exercises can be completed for free using the Databricks 14 day free trial offer on a pay-as-you-go subscription. Instructions for using the free trial offer are included in the exercises.