Connect to and manage dedicated SQL pools in Microsoft Purview

  • Article

This article outlines how to register dedicated SQL pools (formerly SQL DW), and how to authenticate and interact with dedicated SQL pools in Microsoft Purview. For more information about Microsoft Purview, read the introductory article.

Note

There are dedicated SQL pools (formerly SQL DW) and dedicated SQL pools in Azure Synapse workspace and you can refer to What is the difference between dedicated SQL pools (SQL DW) and dedicated SQL pools in Azure Synapse workspaces for the detail difference between the two. Currently Microsoft Purview provides separate data sources for dedicated SQL pools (formerly SQL DW) and dedicated SQL pools:

Supported capabilities

Metadata Extraction Full Scan Incremental Scan Scoped Scan Classification Labeling Access Policy Lineage Data Sharing Live view
Yes Yes Yes Yes Yes Yes No Limited* No No

* Lineage is supported if dataset is used as a source/sink in Data Factory or Synapse pipeline.

When scanning dedicated SQL pool (formerly SQL DW) source, Microsoft Purview supports extracting technical metadata including:

  • Server
  • Dedicated SQL pools
  • Schemas
  • Tables including columns
  • Views including columns

When setting up the scan, you can further scope it after providing the dedicated SQL pool name by selecting tables and views as needed.

Prerequisites

Register

This section describes how to register dedicated SQL pools in Microsoft Purview using the Microsoft Purview governance portal.

Authentication for registration

There are three ways to set up authentication:

  • System or user assigned managed identity (Recommended)

  • Service Principal

  • SQL authentication

    Note

    Only the server-level principal login (created by the provisioning process) or members of the loginmanager database role in the master database can create new logins. It takes about 15 minutes after granting permission, the Microsoft Purview account should have the appropriate permissions to be able to scan the resource(s).

System or user assigned managed identity to register

You can use either your Microsoft Purview system-assigned managed identity (SAMI), or a User-assigned managed identity (UAMI) to authenticate. Both options allow you to assign authentication directly to Microsoft Purview, like you would for any other user, group, or service principal. The Microsoft Purview SAMI is created automatically when the account is created. A UAMI is a resource that can be created independently, and to create one you can follow our user-assigned managed identity guide. Create a Microsoft Entra user in the dedicated SQL pool using your managed identity object name by following the prerequisites and tutorial on Create Microsoft Entra users using Microsoft Entra applications.

Example SQL syntax to create user and grant permission:

CREATE USER [PurviewManagedIdentity] FROM EXTERNAL PROVIDER
GO

EXEC sp_addrolemember 'db_datareader', [PurviewManagedIdentity]
GO

The authentication must have permission to get metadata for the database, schemas, and tables. It must also be able to query the tables to sample for classification. The recommendation is to assign db_datareader permission to the identity.

Service Principal to register

To use service principal authentication for scans, you can use an existing one or create a new one.

If you need to create a new Service Principal, follow these steps:

  1. Navigate to the Azure portal.
  2. Select Microsoft Entra ID from the left-hand side menu.
    1. Select App registrations.
  4. Select + New application registration.
  5. Enter a name for the application (the service principal name).
  6. Select Accounts in this organizational directory only.
  7. For Redirect URI, select Web and enter any URL you want; it doesn't have to be real or work.
  8. Then select Register.

It is required to get the Service Principal's application ID and secret:

  1. Navigate to your Service Principal in the Azure portal
  2. Copy the values the Application (client) ID from Overview and Client secret from Certificates & secrets.
  3. Navigate to your key vault
  4. Select Settings > Secrets
  5. Select + Generate/Import and enter the Name of your choice and Value as the Client secret from your Service Principal
  6. Select Create to complete
  7. If your key vault is not connected to Microsoft Purview yet, you will need to create a new key vault connection
  8. Finally, create a new credential using the Service Principal to set up your scan.
Granting the Service Principal access

In addition, you must also create a Microsoft Entra user in the dedicated pool by following the prerequisites and tutorial on Create Microsoft Entra users using Microsoft Entra applications. Example SQL syntax to create user and grant permission:

CREATE USER [ServicePrincipalName] FROM EXTERNAL PROVIDER
GO

ALTER ROLE db_datareader ADD MEMBER [ServicePrincipalName]
GO

Note

Microsoft Purview will need the Application (client) ID and the client secret in order to scan.

SQL authentication to register

You can follow the instructions in CREATE LOGIN to create a login for your dedicated SQL pool (formerly SQL DW) if you don't already have one.

When authentication method selected is SQL Authentication, you need to get your password and store in the key vault:

  1. Get the password for your SQL login
  2. Navigate to your key vault
  3. Select Settings > Secrets
  4. Select + Generate/Import and enter the Name and Value as the password for your SQL login
  5. Select Create to complete
  6. If your key vault is not connected to Microsoft Purview yet, you will need to create a new key vault connection
  7. Finally, create a new credential using the key to set up your scan.

Steps to register

To register a new SQL dedicated pool in Microsoft Purview, complete the following steps:

  1. Open the Microsoft Purview governance portal by:

  2. Select Data Map on the left navigation.

  3. Select Register

  4. On Register sources, select Azure Dedicated SQL Pool (formerly SQL DW).

  5. Select Continue

On the Register sources screen, complete the following steps:

  1. Enter a Name that the data source will be listed with in the Catalog.
  2. Choose your Azure subscription to filter down dedicated SQL pools.
  3. Select your dedicated SQL pool.
  4. Select a collection from the list.
  5. Select Register to register the data source.

Scan

Follow the steps below to scan dedicated SQL pools to automatically identify assets and classify your data. For more information about scanning in general, see our introduction to scans and ingestion

Create and run scan

To create and run a new scan, complete the following steps:

  1. Select the Data Map tab on the left pane in the Microsoft Purview governance portal.

  2. Select the SQL dedicated pool source that you registered.

  3. Select New scan

  4. Choose either the Azure integration runtime if your source is publicly accessible, a managed virtual network integration runtime if using a managed virtual network, or a self-hosted integration runtime if your source is in a private virtual network. For more information about which integration runtime to use, see the choose the right integration runtime configuration article.

  5. Select the credential to connect to your data source.

    Set up scan

  6. You can scope your scan to specific tables by choosing the appropriate items in the list.

    Scope your scan

  7. Then select a scan rule set. You can choose between the system default, existing custom rule sets, or create a new rule set inline.

    Scan rule set

  8. Choose your scan trigger. You can set up a schedule or run the scan once.

    trigger

  9. Review your scan and select Save and run.

View your scans and scan runs

To view existing scans:

  1. Go to the Microsoft Purview portal. On the left pane, select Data map.
  2. Select the data source. You can view a list of existing scans on that data source under Recent scans, or you can view all scans on the Scans tab.
  3. Select the scan that has results you want to view. The pane shows you all the previous scan runs, along with the status and metrics for each scan run.
  4. Select the run ID to check the scan run details.

Manage your scans

To edit, cancel, or delete a scan:

  1. Go to the Microsoft Purview portal. On the left pane, select Data Map.

  2. Select the data source. You can view a list of existing scans on that data source under Recent scans, or you can view all scans on the Scans tab.

  3. Select the scan that you want to manage. You can then:

    • Edit the scan by selecting Edit scan.
    • Cancel an in-progress scan by selecting Cancel scan run.
    • Delete your scan by selecting Delete scan.

Note

  • Deleting your scan does not delete catalog assets created from previous scans.

Next steps

Now that you have registered your source, follow the below guides to learn more about Microsoft Purview and your data.