Architecture of Azure Operator Insights

Azure Operator Insights is a fully managed service that enables the collection and analysis of massive quantities of network data gathered from complex multi-part or multi-vendor network functions. It delivers statistical, machine learning, and AI-based insights for operator-specific workloads to help operators understand the health of their networks and the quality of their subscribers' experiences in near real-time. For more information on the problem Azure Operator Insights solves, see the general overview of Azure Operator Insights.

Azure Operator Insights deploys a Data Product resource to encapsulate a specific category or namespace of data. Azure Operator Insights enables a fourth generation data mesh architecture, which offers query-time federation to correlate and query across multiple Data Products.

This following diagram shows the architecture of an Azure Operator Insights Data Product, and the surrounding services it interacts with.

Architecture diagram of Azure Operator Insights.

The rest of this article gives an overview of:

  • Deployment of Azure Operator Insights Data Products.
  • Data sources that feed an Azure Operator Insights Data Product.
  • Ingestion options for getting data from those sources into an Azure Operator Insights Data Product.
  • Azure connectivity options to get data from an on-premises private data center into Azure, where Azure Operator Insights Data Products reside.
  • Consumption URLs exposed by an Azure Operator Insights Data Product.
  • Configuration options and controls available when deploying or after deployment of an Azure Operator Insights Data Product.
  • Methods for monitoring an Azure Operator Insights Data Product.

Deployment of Data Products

You can deploy Azure Operator Insights Data Products with any standard Azure interface, including the Azure portal, Azure CLI, Azure PowerShell, or direct calls to the Azure Resource Manager (ARM) API. See Create an Azure Operator Insights Data Product for a quickstart guide to deploying with the Azure portal or the Azure CLI. When you deploy a Data Product, you can enable specific features such as integration with Microsoft Purview, customer-managed keys for data encryption, or restricted access to the Data Product. For more information on features you can enable at deployment, see Data Product configuration options and controls.

Each Azure Operator Insights Data Product is scoped for a given category or namespace of data. An example is the data from a single network function (NF) such as a voice SBC. Some Data Products might contain correlated data from multiple NFs, particularly if the NFs are from the same vendor, such as the UPF, SMF, and AMF from a mobile packet core vendor. Each Data Product appears as a single Azure resource in your resource group and subscription. You can deploy multiple Data Products, for different categories of data, for example different mobile packet core NFs from different vendors, or a mobile packet core plus a radio access network (RAN) Data Product.

Microsoft publishes several Data Products; the following image shows some examples. Partners and operators can also design and publish Data Products using the Azure Operator Insights data product factory (preview). For more information on the data product factory, see the overview of the data product factory.

Screenshot of the Azure portal displaying a selection of Data Products from Microsoft.

Deploying an Azure Operator Insights Data Product creates the resource itself and a managed resource group in your subscription. The managed resource group contains an Azure Key Vault instance. The Key Vault instance contains a shared access signature (SAS) that you can use to authenticate when you upload files to the Data Product's ingestion storage URL.

Once deployed, the Overview screen of the Azure Operator Insights Data Product resource shows essential information including:

  • Version, product (Data Product type), and publisher.
  • Ingestion storage URLs (see Data ingestion).
  • Consumption URLs for ADLS and KQL (see Data consumption).

Screenshot of the Azure portal displaying properties of a Data Product, including the version, product, publisher, and ingestion and consumption URLs.

Data sources

Each Azure Operator Insights Data Product ingests data from a particular data source. The data source could be:

  • A network function such as a mobile packet core (such as Azure Operator 5G Core), voice session border controller (SBC), radio access network (RAN), or transport switch.
  • A platform such as Azure Operator Nexus.

Data ingestion

There are a range of options for ingesting data from the source into your Azure Operator Insights Data Product.

  • Using an Azure Operator Insights ingestion agent – This can consume data from different sources and upload the data to an Azure Operator Insights Data Product. For example, it supports pulling data from an SFTP server, or terminating a TCP stream of enhanced data records (EDRs). For more information, see Ingestion agent overview.
  • Using other Azure services and tools – Multiple tools can upload data to an Azure Operator Insights Data Product. For example:
  • Using the code samples available in the Azure Operator Insights sample repository as a basis for creating your own ingestion agent or script for uploading data to an Azure Operator Insights Data Product.

Azure connectivity

There are multiple ways to connect your on-premises private data centers where your network function data sources reside to the Azure cloud. For a general overview of the options, see Connectivity to Azure - Cloud Adoption Framework. For telco-specific recommendations, see the Network Analytics Landing Zone for Operators.

Data consumption

Azure Operator Insights Data Products offer two consumption URLs for accessing the data in the Data Product:

  • ADLS consumption URL giving access to Parquet files for batch style consumption or integration with AI / ML tools.
  • KQL consumption URL supporting the Kusto Query Language for real-time analytics, reporting, and adhoc queries.

There are multiple possible integrations that can be built on top of one or both of these consumption URLs.

Supported with Data Product ADLS consumption URL Supported with Data Product KQL consumption URL
Azure Data Explorer dashboards
Azure Data Explorer follower database
Power BI reports
Microsoft Fabric
Azure Machine Learning Studio
Azure Databricks
Azure Logic Apps
Azure Storage Explorer
AzCopy

Data Product configuration options and controls

Azure Operator Insights Data Products have several configuration options that can be set when first deploying or modified after deployment.

Description When configurable More information
Integration with Microsoft Purview Enabling Purview integration during deployment causes the existence of the Data Product and its data type tables, schemas, and lineage to be published to Purview and visible to your organization in Purview's data catalog. At deployment Use Microsoft Purview with an Azure Operator Insights Data Product
Customer Managed Keys for Data Product storage Azure Operator Insights Data Products can secure your data using Microsoft Managed Keys or Customer Managed Keys. At deployment Set up resources for CMK-based data encryption or Microsoft Purview
Connectivity for ingestion and ADLS consumption URLs Azure Operator Insights Data Products can be configured to allow public access from all networks or selected virtual networks and IP addresses. At deployment. If you deploy with selected virtual networks and IP addresses, you can add or remove networks and IP addresses after deployment. --
Connectivity for the KQL consumption URL Azure Operator Insights Data Products can be configured to allow public access from all networks or selected IP addresses. At deployment. If you deploy with selected IP addresses, you can add or remove IP addresses after deployment. --
Data retention and hot cache size Azure Operator Insights Data Products are initially deployed with default retention periods and KQL hot cache durations for each data type (group of data within a Data Product). You can set custom thresholds After deployment Data types in Azure Operator Insights
Access control for ADLS consumption URL Access to the ADLS consumption URL is managed on an Azure Operator Insights Data Product by generating a SAS token after deployment. After deployment --
Access control for KQL consumption URL Access to the KQL consumption URL is granted by adding a principal (which can be an individual user, group, or managed identity) as a Reader or Restricted Reader. After deployment Manage permissions to the KQL consumption URL

Monitoring

After you deploy a Data Product, you can monitor it for healthy operation or troubleshooting purposes using metrics, resource logs, and activity logs. For more information, see Monitoring Azure Operator Insights.

Next step