How to create and manage critical data (Preview)

Not all data elements have the same importance or sensitivity, and dedicating resources to manage the quality of all data indiscriminately can be impractical and costly. Critical data elements (CDEs) are a logical grouping of important columns across tables in your data sources that you can use to focus your efforts.

These columns should be critical pieces of information that are necessary for decision making and so need to be governed with the highest care. For example: A "Customer ID" critical data element can map "CustID" from one table and "CID" from another table into the same logical container. Users can match this value across data assets to make connections, and when data producers create a new asset they can use this element as a blueprint to provide quality information in the correct format.

Critical data elements are created within business domains and can have data quality rules and policies set to manage these important pieces of information.

By creating CDEs, organizations can allocate resources strategically, focusing governance effort on areas that have the most significant impact on the business.

Prerequisites

• You must be using the new data catalog experience.
• To create critical data elements, the user must have data steward permissions.
• To access data assets in the data map, they need at least data reader permissions on the collection where the asset is housed. For more information about data map permissions, see the data map permissions article.

Access critical data

Critical data is defined in business domains. To see critical data in your data catalog:

1. Open the Data Catalog in the Microsoft Purview portal.
2. Select the Data management drop-down.
3. Select Business domains
4. You see a list of all the business domains you have access to based on your permissions. Select the business domain where you want to see your critical data.
5. Under the Details tab select Critical data elements.
6. Here you can see the list of your critical data elements, and select any one to view its details.

Critical data element details

On the details page for your critical data element, you'll find:

• Columns - all data columns mapped from your assets to this critical data element. You can add columns.
• Associated data products - an automatically generated list of data products that have assets associated with your critical data element.

  Note

  Currently, the list of associated data products is limited to data products in the current business domain.

• Data quality - Critical data elements allow you to attach data quality rules, to monitor and manage your critical data. For instructions, see the steps later in the article.
• Manage policies - Critical data elements allow you to attach access policies, to protect your critical information. For instructions, see the steps later in the article.

Create critical data

Note

To create a critical data you need data steward permissions.

1. Open the Data Catalog in the Microsoft Purview Portal.
2. Select the Data management drop-down.
3. Select Business domains
4. You see a list of all the business domains you have access to based on your permissions. Select the business domain where you want to add critical data.
5. Under the Details tab, select Critical data elements.
6. Select the + New.
7. Select the Create dropdown. Select Create.
8. Give your critical data a unique name, and description, owner, and expected data type.
9. Select Create.
10. You see your critical data in the list of critical data elements.

Note

Other users can't see this critical data as a filter in the data product search until its status is set to Published.

Add columns

Once you create your critical data, you can add data columns to map critical columns from your assets to this logical concept.

1. Select your critical data element from the list of critical data elements in your business domain.
2. Select the + Add column button.
3. Search for your data sources using keywords and filters, and select a data source.
4. In that data source, select any relevant column.
5. Select Add.
6. Now you should be able to see all your columns mapped to your critical data element.
7. Repeat for any number of data sources and columns.

Below the columns you've added, you'll also now see the list of data products that are associated with the assets you've added to your critical data element.

Remove column

To remove a column from a critical data element, select the ellipsis ... button in the row where the column is listed, and select Remove.

If you can't find the column you want to remove, select the View all columns button to see the full list of columns.

Manage critical data

Note

To edit a critical data you need the data steward role.

1. On the critical data's details page, select Edit.
2. Here you're able to edit name, description, owner, and expected data type.
3. Select Save to save your changes.

To update the status of your critical data element, select the Status button to change between: Draft, Published, and Expired.

Manage access policies

To manage access policies for critical data, you need data steward permissions.

1. On your business domain's page, select your critical data elements.
2. Select a critical data element.
3. Select Manage policies.
4. From the policy configuration window, you're able to create and manage access policies for your critical data elements.
5. For more information about setting up access policies, see the documentation about managing access policies.

These policies will be applied to all columns and associated data products in your critical data element.

Data quality

Critical data elements allow you to attach data quality rules. Attaching a rule will automatically apply it to all the columns listed in your critical data element, so the data can be properly managed.

1. In the Microsoft Purview Portal, select your critical data element from the list of elements in your business domain.
2. Select the manage Data quality button.
3. A list of all rules will be displayed and under Status, any rules listed as On will be applied to your critical data element and its associated columns.
4. To add or remove these rules from your critical data element, toggle the Status switch, or delete the rule.
5. You can also create a new rule directly from the menu by selecting New rule.

Data quality rules will be applied to all columns and associated data products in your critical data element. For more information, see the data quality for critical data elements article.