Azure Data Lake and Entity Store

  • 15 minutes

Microsoft Azure Data Lake Storage is a no-limits data lake that enables organizations to power intelligent action. It removes the complexities of ingesting and storing all your data while making it faster to get operational with batch, streaming, and interactive analytics.

Azure Data Lake Storage enables you to have cloud storage that is less expensive than the cloud storages that relational databases provide. This data includes business data that is traditionally stored in business systems and data warehouses, in addition to device and sensor data, such as signals from devices. In addition, Data Lake Storage supports a range of tools and programming languages that enable large amounts of data to be reported on, queried, and transformed.

Finance and operations apps can use Data Lake Storage for AI and analytics capabilities, allowing you to take advantage of the cost effectiveness and strengths of the Azure Data Lake Storage technology.

Entity Store is available in Azure Data Lake Storage so that you can enable advanced analytics and AI, such as reasoning over data, across finance and operations apps and other data in your Azure Data Lake Storage. Finance and operations apps provide straightforward access to denormalized transactional data in a near real-time basis.

Entity Store schemas are available in Azure Data Lake Storage so that you can directly access denormalized data for reporting and analytics. Aggregate measurements are stored in Data Lake Storage as Common Data Model folders that enable rich data mash-up and analytical scenarios. For example, aggregate measurement data can be attached as Microsoft Power BI reference data flows and used for reporting with Power BI. Power users can create new data mashups and reports in Power BI Desktop.

Follow these steps to make Entity Store available as a data lake:

  1. Enable automated Entity Store refresh before you can enable data lake integration by going to System administration > Setup > Entity Store.

  2. The Entity Store page includes all Aggregate measures in the left pane. Select one of the records.

  3. Change the Automatic refresh enabled toggle to Yes.

    Screenshot of the Automatic refresh enabled toggle set to Yes.

    Important

    This action is not reversable. When you switch to automated Entity Store refresh, you can't revert to the old user interface experience.

  4. Specify the refresh Recurrence. You can select Every hour, Twice per day, Once per day, or Once per week.

    Note

    Administrators can refresh aggregate measures on demand by selecting the aggregate measure to refresh and then selecting Refresh in the Action Pane.

Follow these steps to be able to work in the Entity Store as a data lake:

  1. Go to System administration > System setup > System parameters.

  2. Select the Data connections tab in the left pane.

    Screenshot of the System parameters page, showing the Data connections area.

  3. Change the Enable Data Lake integration toggle to Yes.

  4. If you want to update Data Lake Storage in frequent small batches as data is updated in the system, change the Trickle update Data Lake toggle to Yes.

  5. Enter the following information:

    • Application ID – Application ID of the Microsoft Entra ID application.
    • Application Secret – Application key (secret) for the Microsoft Entra ID application.
    • DNS name – DNS name of the Key Vault.
    • Secret name – Name of the secret that you added to Key Vault together with connection string information.

  6. When you have entered all information, select Test Azure Key Vault and Test Azure Storage to validate that Dynamics 365 can access the provided configuration information.

Entity Store data should now be populated in the storage location that you provided, not in the relational Entity Store database.

For more information, see Make Entity store available as a Data Lake.