Data Profiling enhancements including better visualizations and more enhanced capabilities
Important
This content is archived and is not being updated. For the latest documentation, see Microsoft Power Platform product documentation. For the latest release plans, see Dynamics 365 and Microsoft Power Platform release plans.
| Enabled for | Public preview | General availability |
|---|---|---|
| Admins, makers, or analysts, automatically | - |  Oct 14, 2019 |
Feature details
Power Query continues to lead innovation in the area of smart data preparation by taking advantage of and creating products based on the strong Microsoft-wide investments in artificial intelligence and other research efforts around data preparation. Over the last 18 months, we've added several Power Query features to enable customers to transform their data in smarter ways, including:
- Example Data Extraction: Enables customers to extract data from HTML pages or from any table within the Power Query Editor, simply by entering sample output values that they want to extract, even selecting from a set of suggested values that Power Query automatically detected based on common transformation patterns. Power Query's AI algorithms can then infer the user intent and the optimal combination of data transformations that's needed to go from input data to the desired output specified by the user.
- Fuzzy Merge: Merges tables by using fuzzy matching algorithms (the Jaccard index) to determine matching rows across tables. These fuzzy matching algorithms are the result of several years of research at Microsoft, and they have been released in multiple products, including Microsoft Excel and Microsoft SQL Server, in addition to Power Query.
- Data Profiling: Supports over 300 different data transformations, allowing users to filter outlier values, remove duplicates, remove or replace errors, and so on; however, recent investments in Data Profiling within the Power Query Editor have made it even easier for customers to realize that their data has such issues, so that they know that they need to apply the necessary data transformations to fix them.
- Mapping to the Common Data Model entities schema: Allows Power Query Online customers to map arbitrary tables from any data source to a target entity schema that's defined as part of the Common Data Model specification. After an entity mapping is defined, downstream processing of that data becomes more powerful, because it can operate over this data at the semantic level instead of only the data level.
Going forward, Power Query's investments will further expand these capabilities.
Here is an overview of the next wave of smart data preparation capabilities that will be delivered to Power Query customers:
- Enhancements to Data Profiling
- Support for new AI Insights, including Cognitive Services and Azure Machine Learning models
- Query Diagnostics