Sources of monitoring data for Azure Monitor
Azure Monitor is based on a common monitoring data platform that includes Logs and Metrics. Collecting data into this platform allows data from multiple resources to be analyzed together using a common set of tools in Azure Monitor. Monitoring data may also be sent to other locations to support certain scenarios, and some resources may write to other locations before they can be collected into Logs or Metrics.
This article describes the different sources of monitoring data collected by Azure Monitor in addition to the monitoring data created by Azure resources. Links are provided to detailed information on configuration required to collect this data to different locations.
Sources of monitoring data from Azure applications can be organized into tiers, the highest tiers being your application itself and the lower tiers being components of Azure platform. The method of accessing data from each tier varies. The application tiers are summarized in the table below, and the sources of monitoring data in each tier are presented in the following sections. See Monitoring data locations in Azure for a description of each data location and how you can access its data.
The following table briefly describes the application tiers that are specific to Azure. Following the link for further details on each in the sections below.
|Azure Tenant||Data about the operation of tenant-level Azure services, such as Azure Active Directory.||View AAD data in portal or configure collection to Azure Monitor using a tenant diagnostic setting.|
|Azure subscription||Data related to the health and management of cross-resource services in your Azure subscription such as Resource Manager and Service Health.||View in portal or configure collection to Azure Monitor using a log profile.|
|Azure resources||Data about the operation and performance of each Azure resource.||Metrics collected automatically, view in Metrics Explorer.
Configure diagnostic settings to collect logs in Azure Monitor.
Monitoring solutions and Insights available for more detailed monitoring for specific resource types.
Azure, other cloud, or on-premises
The following table briefly describes the application tiers that may be in Azure, another cloud, or on-premises. Following the link for further details on each in the sections below.
|Operating system (guest)||Data about the operating system on compute resources.||Install Log Analytics agent to collect client data sources into Azure Monitor and Dependency agent to collect dependencies supporting Azure Monitor for VMs.
For Azure virtual machines, install Azure Diagnostic Extension to collect logs and metrics into Azure Monitor.
|Application Code||Data about the performance and functionality of the actual application and code, including performance traces, application logs, and user telemetry.||Instrument your code to collect data into Application Insights.|
|Custom sources||Data from external services or other components or devices.||Collect log or metrics data into Azure Monitor from any REST client.|
Telemetry related to your Azure tenant is collected from tenant-wide services such as Azure Active Directory.
Azure Active Directory Audit Logs
Azure Active Directory reporting contains the history of sign-in activity and audit trail of changes made within a particular tenant.
|Azure Monitor Logs||Configure Azure AD logs to be collected in Azure Monitor to analyze them with other monitoring data.||Integrate Azure AD logs with Azure Monitor logs (preview)|
|Azure Storage||Export Azure AD logs to Azure Storage for archiving.||Tutorial: Archive Azure AD logs to an Azure storage account (preview)|
|Event Hub||Stream Azure AD logs to other locations using Event Hubs.||Tutorial: Stream Azure Active Directory logs to an Azure event hub (preview).|
Telemetry related to the health and operation of your Azure subscription.
Azure Activity log
The Azure Activity log includes service health records along with records on any configuration changes made to the resources in your Azure subscription. The Activity log is available to all Azure resources and represents their external view.
|Activity log||The Activity log is collected into its own data store that you can view from the Azure Monitor menu or use to create Activity log alerts.||Query the Activity log in the Azure portal|
|Azure Monitor Logs||Configure Azure Monitor Logs to collect the Activity log to analyze it with other monitoring data.||Collect and analyze Azure activity logs in Log Analytics workspace in Azure Monitor|
|Azure Storage||Export the Activity log to Azure Storage for archiving.||Archive Activity log|
|Event Hubs||Stream the Activity log to other locations using Event Hubs||Stream Activity log to Event Hub.|
Azure Service Health
Azure Service Health provides information about the health of the Azure services in your subscription that your application and resources rely on.
Azure Monitor Logs
|Service Health records are stored in the Azure Activity log, so you can view them in the Azure portal or perform any other activities you can perform with the Activity log.||View service health notifications by using the Azure portal|
Metrics and resource level diagnostic logs provide information about the internal operation of Azure resources. These are available for most Azure services, and monitoring solutions and insights collect additional data for particular services.
|Azure Monitor Metrics||Platform metrics will write to the Azure Monitor metrics database with no configuration. Access platform metrics from Metrics Explorer.||Getting started with Azure Metrics Explorer
Supported metrics with Azure Monitor
|Azure Monitor Logs||Copy platform metrics to Logs for trending and other analysis using Log Analytics.||Azure diagnostics direct to Log Analytics|
|Event Hubs||Stream metrics to other locations using Event Hubs.||Stream Azure monitoring data to an event hub for consumption by an external tool|
Diagnostic logs provide insights into the internal operation of an Azure resource. Diagnostic logs are not enabled by default. You must enable them and specify a destination for each resource.
The configuration requirements and content of Diagnostic logs vary by resource type, and not all services yet create Diagnostic logs. See Supported services, schemas, and categories for Azure Diagnostic Logs for details on each service and links to detailed configuration procedures. If the service isn’t listed in this article, then that service doesn’t currently write to diagnostic logs.
|Azure Monitor Logs||Send Diagnostic Logs to Azure Monitor Logs for analysis with other collected log data. Some resources can write directly to Azure Monitor while others write to a storage account before being imported into a Log Analytics workspace.||Stream Azure Diagnostic Logs to Log Analytics workspace in Azure Monitor
Use the Azure portal to collect logs from Azure Storage
|Storage||Send Diagnostic logs to Azure Storage for archiving.||Archive Azure Diagnostic Logs|
|Event Hubs||Stream Diagnostic logs to other locations using Event Hubs.||Stream Azure Diagnostic Logs to an event hub|
Operating system (guest)
Compute resources in Azure, in other clouds, and on-premises have a guest operating system to monitor. With the installation of one or more agents, you can gather telemetry from the guest into Azure Monitor to analyze it with the same monitoring tools as the Azure services themselves.
Azure Diagnostic extension
Enabling the Azure Diagnostics extension for Azure Virtual machines allows you to collect logs and metrics from the guest operating system of Azure compute resources including Azure Cloud Service (classic) Web and Worker Roles, Virtual Machines, virtual machine scale sets, and Service Fabric.
|Storage||When you enable the Diagnostics Extension, it will write to a storage account by default.||Store and view diagnostic data in Azure Storage|
|Azure Monitor Metrics||When you configure the Diagnostics Extension to collect performance counters, they are written to the Azure Monitor metrics database.||Send Guest OS metrics to the Azure Monitor metric store using a Resource Manager template for a Windows virtual machine|
|Application Insights Logs||Collect logs and performance counters from the compute resource supporting your application to be analyzed with other application data.||Send Cloud Service, Virtual Machine, or Service Fabric diagnostic data to Application Insights|
|Event Hubs||Configure the Diagnostics Extension to stream the data to other locations using Event Hubs.||Streaming Azure Diagnostics data in the hot path by using Event Hubs|
Log Analytics agent
Install the Log Analytics agent for comprehensive monitoring and management of your Windows or Linux virtual machines. The virtual machine can be running in Azure, another cloud, or on-premises.
|Azure Monitor Logs||The Log Analytics agent connects to Azure Monitor either directly or through System Center Operations Manager and allows you to collect data from data sources that you configure or from monitoring solutions that provide additional insights into applications running on the virtual machine.||Agent data sources in Azure Monitor
Connect Operations Manager to Azure Monitor
Azure Monitor for VMs
Azure Monitor for VMs provides a customized monitoring experience for virtual machines providing features beyond core Azure Monitor functionality, including service status and VM health. It requires a Dependency Agent on Windows and Linux virtual machines that integrates with the Log Analytics agent to collect discovered data about processes running on the virtual machine and external process dependencies.
|Azure Monitor Logs||Stores data about processes and dependencies on the agent.||Using Azure Monitor for VMs (preview) Map to understand application components|
|VM Storage||Azure Monitor for VMs stores heath state information in a custom location. This is only available to Azure Monitor for VMs in the Azure portal in addition to the Azure Resource health REST API.||Understand the health of your Azure virtual machines
Azure Resource health REST API
Detailed application monitoring in Azure Monitor is done with Application Insights which collects data from applications running on a variety of platforms. The application can be running in Azure, another cloud, or on-premises.
When you enable Application Insights for an application by installing an instrumentation package, it collects metrics and logs related to the performance and operation of the application. Application Insights stores the data it collects in the same Azure Monitor data platform used by other data sources. It includes extensive tools for analyzing this data, but you can also analyze it with data from other sources using tools such as Metrics Explorer and Log Analytics.
|Azure Monitor Logs||Operational data about your application including page views, application requests, exceptions, and traces.||Analyze log data in Azure Monitor|
|Dependency information between application components to support Application Map and telemetry correlation.||Telemetry correlation in Application Insights
|Results of availability tests that test the availability and responsiveness of your application from different locations on the public Internet.||Monitor availability and responsiveness of any web site|
|Azure Monitor Metrics||Application Insights collects metrics describing the performance and operation of the application in addition to custom metrics that you define in your application into the Azure Monitor metrics database.||Log-based and pre-aggregated metrics in Application Insights
Application Insights API for custom events and metrics
|Azure Storage||Send application data to Azure Storage for archiving.||Export telemetry from Application Insights|
|Details of availability tests are stored in Azure Storage. Use Application Insights in the Azure portal to download for local analysis. Results of availability tests are stored in Azure Monitor Logs.||Monitor availability and responsiveness of any web site|
|Profiler trace data is stored in Azure Storage. Use Application Insights in the Azure portal to download for local analysis.||Profile production applications in Azure with Application Insights|
|Debug snapshot data that is captured for a subset of exceptions is stored in Azure Storage. Use Application Insights in the Azure portal to download for local analysis.||How snapshots work|
Monitoring Solutions and Insights
Monitoring solutions and Insights collect data to provide additional insights into the operation of a particular service or application. They may address resources in different application tiers and even multiple tiers.
|Azure Monitor Logs||Monitoring solutions collect data into Azure Monitor logs where it may be analyzed using the query language or views that are typically included in the solution.||Data collection details for monitoring solutions in Azure|
Azure Monitor for Containers
|Azure Monitor Logs||Stores monitoring data for AKS including inventory, logs, and events. Metric data is also stored in Logs in order to leverage its analysis functionality in the portal.||Understand AKS cluster performance with Azure Monitor for containers|
|Azure Monitor Metrics||Metric data is stored in the metric database to drive visualization and alerts.||View container metrics in metrics explorer|
|Azure Kubernetes Service||In order to a near real time experience, Azure Monitor for Containers presents data directly from the Azure Kubernetes service in the Azure portal.||How to view container logs real time with Azure Monitor for containers (preview)|
Azure Monitor for VMs
Azure Monitor for VMs provides a customized experience for monitoring virtual machines. A description of the data collected by Azure Monitor for VMs is included in the Operating System (guest) section above.
In addition to the standard tiers of an application, you may need to monitor other resources that have telemetry that can't be collected with the other data sources. For these resources, write this data to either Metrics or Logs using an Azure Monitor API.
|Azure Monitor Logs||Data Collector API||Collect log data from any REST client and store in Log Analytics workspace.||Send log data to Azure Monitor with the HTTP Data Collector API (public preview)|
|Azure Monitor Metrics||Custom Metrics API||Collect metric data from any REST client and store in Azure Monitor metrics database.||Send custom metrics for an Azure resource to the Azure Monitor metric store by using a REST API|
Other services in Azure write data to the Azure Monitor data platform. This allows you to analyze data collected by these services with data collected by Azure Monitor and leverage the same analysis and visualization tools.
|Azure Security Center||Azure Monitor Logs||Azure Security Center stores the security data it collects in a Log Analytics workspace which allows it to be analyzed with other log data collected by Azure Monitor.||Data collection in Azure Security Center|
|Azure Sentinel||Azure Monitor Logs||Azure Sentinel stores the data it collects from different data sources in a Log Analytics workspace which allows it to be analyzed with other log data collected by Azure Monitor.||Connect data sources|
- Learn more about the types of monitoring data collected by Azure Monitor and how to view and analyze this data.
- List the different locations where Azure resources store data and how you can access it.