Register and scan Azure Data Explorer
This article outlines how to register an Azure Data Explorer account in Azure Purview and set up a scan.
Azure Data Explorer supports full and incremental scans to capture the metadata and schema. Scans also classify the data automatically based on system and custom classification rules.
- Before registering data sources, create an Azure Purview account. For more information on creating a Purview account, see Quickstart: Create an Azure Purview account.
- You need to be an Azure Purview Data Source Admin
Setting up authentication for a scan
There is only one way to set up authentication for Azure data explorer:
- Service Principal
To use service principal authentication for scans, you can use an existing one or create a new one.
If you have to create a new Service Principal, please follow these steps:
- Navigate to the Azure portal.
- Select Azure Active Directory from the left-hand side menu.
- Select App registrations.
- Select + New application registration.
- Enter a name for the application (the service principal name).
- Select Accounts in this organizational directory only.
- For Redirect URI select Web and enter any URL you want; it doesn't have to be real or work.
- Then select Register.
It is required to get the Service Principal's application ID and secret:
- Navigate to your Service Principal in the Azure portal
- Copy the values the Application (client) ID from Overview and Client secret from Certificates & secrets.
- Navigate to your key vault
- Select Settings > Secrets
- Select + Generate/Import and enter the Name of your choice and Value as the Client secret from your Service Principal
- Select Create to complete
- If your key vault is not connected to Purview yet, you will need to create a new key vault connection
- Finally, create a new credential using the Service Principal to setup your scan
Granting the Service Principal access to your Azure data explorer instance
Navigate to the Azure portal. Then navigate to your Azure data explorer instance.
Add the service principal to the AllDatabasesViewer role in the Permissions tab, as shown in the following screenshot.
Register an Azure Data Explorer account
To register a new Azure Data Explorer (Kusto) account in your data catalog, do the following:
- Navigate to your Purview account
- Select Sources on the left navigation
- Select Register
- On Register sources, select Azure Data Explorer
- Select Continue
On the Register sources (Azure Data Explorer (Kusto)) screen, do the following:
- Enter a Name that the data source will be listed with in the Catalog.
- Choose how you want to point to your desired storage account:
- Select From Azure subscription, select the appropriate subscription from the Azure subscription drop down box and the appropriate cluster from the Cluster drop down box.
- Or, you can select Enter manually and enter a service endpoint (URL).
- Finish to register the data source.
Creating and running a scan
The steps and screenshots shown below illustrate the general process for managing scans across different data source types. Your options may differ slightly depending on the types of data sources that you are working with.
To create and run a new scan, do the following:
Navigate to the Sources
Select the data source that you registered.
Select + New scan
Select the credential to connect to your data source.
You can scope your scan to specific parts of the data source such as folders, collections or schemas by checking the appropriate items in the list.
The select a scan rule set for you scan. You can choose between the system default, the existing custom ones or create a new one inline.
Choose your scan trigger. You can set up a schedule or run the scan once.
Review your scan and select Save and run.
Viewing your scans and scan runs
To view existing scans, do the following:
Navigate to the management center. Select Data sources under the Sources and scanning section.
Select the desired data source. You will see a list of existing scans on that data source.
Select the scan whose results you are interested to view.
This page will show you all of the previous scan runs along with metrics and status for each scan run. It will also display whether your scan was scheduled or manual, how many assets had classifications applied, how many total assets were discovered, the start and end time of the scan, and the total scan duration.
Manage your scans - edit, delete, or cancel
To manage or delete a scan, do the following:
Navigate to the management center. Select Data sources under the Sources and scanning section then select on the desired data source.
Select the scan you would like to manage. You can edit the scan by selecting Edit.
You can delete your scan by selecting Delete.