One of the biggest challenges that organizations face today, is that it can be difficult to understand who your customers are and what is important to them. Customer data can be scattered across multiple data silos. For example, the transaction system that is used by your organizations retail website might operate independently from the point-of-sale system used at your retail stores. Additionally, your organizations support and call centers might use a separate application designed for call center scenarios. With customer data scattered across different systems, how can you gain a clear picture of who that customer is?

Audience Insights, part of Dynamics 365 Customer Insights helps your organization unlock insights to build a deeper understanding of your customers. Audience Insights ingests data from all of your different data sources and unifies them into a single customer profile. From within the customer profile, you can track data across different demographics, and identify trends based on key data you want to track.

Dynamics 365 Customer Insights customer profile.

Gaining a 360-degree view of your customers starts by ingesting the specific data that you need to work with across all your organizations different data silos.

These sources might include:

  • Transactional data such as point-of-sale systems

  • Observational data such as product testing application

  • Behavioral sources such as Customer Service, Sales, etc., applications

  • Any data source where customer-related data is stored

Extract, transform, load (ETL)

Before applications that work with data from different sources, such as Audience Insights can use it. The data needs to be collected and refined into something that can be consumed by the application. This process is critical to ensuring that the data is used appropriately. This process is referred to as ETL, which stands for Extract, Transform, Load.

As the name states, there are three steps that make up the ETL process and enable data to be integrated from source to destination.

  • Data extraction: Extracts raw data from a data source such as legacy systems, cloud environments, CRM/ERP applications, data warehouses, etc.

  • Data transformation: Improves data quality and accessibility through processes such as data cleansing, standardizations, sorting, etc.

  • Data loading: Loads the data into its new location.

Depending on the data ingestion method that you're going to be using, ETL might be done as part of the ingestion process, or prior to ingesting the data. As we examine the different ingestion methods available, we'll look at when ETL should be done.

Ingesting data

Data ingested into Audience Insights is stored in a data set. A data set is essentially a table that contains the data you want to use. It might be profile-related data coming from a CRM application, or purchases coming from a PoS system. Data sets are stored in data sources, which are organizational units that help organize your data sets and make it easier to group/find them. For example, you might define a data source called "eCommerce" to ingest data from an ecommerce application. The ecommerce application will likely contain both customer profile and customer purchase data. A data set exist for each of them in the data source.

Depending on the volume of data you want to ingest, and where the data is located, you'll need to decide which ingestion method to use.

Audience Insights provides three options to choose from:

  • Import Data - Used when you want to connect to data such as Microsoft Dataverse, Azure Blobs, OData sources, etc. For more information. see Power Query Connectors.

  • Connect to a Common Data Model Folder - Used when you want to connect to an Azure Data Lake Storage Gen 2 Account. For more information, see Common Data Folder..

  • Connect to a Dataverse - Used when you want to connect to data sets in the Dataverse data lake. For more information, see Dataverse.

Throughout the remaining units, we'll examine each of these options in more detail.