Azure Stream Analytics monitoring data reference

This article contains all the monitoring reference information for this service.

See Monitor Azure Stream Analytics for details on the data you can collect for Azure Stream Analytics and how to use it.

Metrics

This section lists all the automatically collected platform metrics for this service. These metrics are also part of the global list of all platform metrics supported in Azure Monitor.

For information on metric retention, see Azure Monitor Metrics overview.

Supported metrics for Microsoft.StreamAnalytics/streamingjobs

The following table lists the metrics available for the Microsoft.StreamAnalytics/streamingjobs resource type.

• All columns might not be present in every table.
• Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.

Table headings

• Category - The metrics group or classification.
• Metric - The metric display name as it appears in the Azure portal.
• Name in REST API - The metric name as referred to in the REST API.
• Unit - Unit of measure.
• Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
• Dimensions - Dimensions available for the metric.
• Time Grains - Intervals at which the metric is sampled. For example, PT1M indicates that the metric is sampled every minute, PT30M every 30 minutes, PT1H every hour, and so on.
• DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.

Category	Metric	Name in REST API	Unit	Aggregation	Dimensions	Time Grains	DS Export
Failed Function Requests Failed Function Requests	AMLCalloutFailedRequests	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Function Events Function Events	AMLCalloutInputEvents	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Function Requests Function Requests	AMLCalloutRequests	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Data Conversion Errors Data Conversion Errors	ConversionErrors	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Input Deserialization Errors Input Deserialization Errors	DeserializationError	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Out of order Events Out of order Events	DroppedOrAdjustedEvents	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Early Input Events Early Input Events	EarlyInputEvents	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Runtime Errors Runtime Errors	Errors	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Input Event Bytes Input Event Bytes	InputEventBytes	Bytes	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Input Events Input Events	InputEvents	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Backlogged Input Events Backlogged Input Events	InputEventsSourcesBacklogged	Count	Average, Maximum, Minimum	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Input Sources Received Input Sources Received	InputEventsSourcesPerSecond	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Late Input Events Late Input Events	LateInputEvents	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Output Events Output Events	OutputEvents	Count	Total	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
Watermark Delay Watermark Delay	OutputWatermarkDelaySeconds	Seconds	Average, Maximum, Minimum	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
CPU % Utilization CPU % Utilization	ProcessCPUUsagePercentage	Percent	Average, Maximum, Minimum	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes
SU (Memory) % Utilization SU (Memory) % Utilization	ResourceUtilization	Percent	Average, Maximum, Minimum	LogicalName, PartitionId, ProcessorInstance, NodeName	PT1M	Yes

Metrics descriptions

Azure Stream Analytics provides the following metrics for you to monitor your job's health.

Metric	Definition
Backlogged Input Events	Number of input events that are backlogged. A nonzero value for this metric implies that your job can't keep up with the number of incoming events. If this value is slowly increasing or is consistently nonzero, you should scale out your job. To learn more, see Understand and adjust streaming units.
Data Conversion Errors	Number of output events that couldn't be converted to the expected output schema. To drop events that encounter this scenario, you can change the error policy to Drop.
CPU % Utilization (preview)	Percentage of CPU that your job utilizes. Even if this value is very high (90 percent or more), you shouldn't increase the number of SUs based on this metric alone. If the number of backlogged input events or watermark delays increases, you can then use this metric to determine if the CPU is the bottleneck. This metric might have intermittent spikes. We recommend that you do scale tests to determine the upper bound of your job after which inputs are backlogged or watermark delays increase because of a CPU bottleneck.
Early Input Events	Events whose application time stamp is earlier than their arrival time by more than 5 minutes.
Failed Function Requests	Number of failed Azure Machine Learning function calls (if present).
Function Events	Number of events sent to the Azure Machine Learning function (if present).
Function Requests	Number of calls to the Azure Machine Learning function (if present).
Input Deserialization Errors	Number of input events that couldn't be deserialized.
Input Event Bytes	Amount of data that the Stream Analytics job receives, in bytes. You can use this metric to validate that events are being sent to the input source.
Input Events	Number of records deserialized from the input events. This count doesn't include incoming events that result in deserialization errors. Stream Analytics can ingest the same events multiple times in scenarios like internal recoveries and self-joins. Don't expect Input Events and Output Events metrics to match if your job has a simple pass-through query.
Input Sources Received	Number of messages that the job receives. For Azure Event Hubs, a message is a single EventData item. For Azure Blob Storage, a message is a single blob. Note that input sources are counted before deserialization. If there are deserialization errors, input sources can be greater than input events. Otherwise, input sources can be less than or equal to input events because each message can contain multiple events.
Late Input Events	Events that arrived later than the configured tolerance window for late arrivals. Learn more about Azure Stream Analytics event order considerations.
Out-of-Order Events	Number of events received out of order that were either dropped or given an adjusted time stamp, based on the event ordering policy. This metric can be affected by the configuration of the Out-of-Order Tolerance Window setting.
Output Events	Amount of data that the Stream Analytics job sends to the output target, in number of events.
Runtime Errors	Total number of errors related to query processing. It excludes errors found while ingesting events or outputting results.
SU (Memory) % Utilization	Percentage of memory that your job utilizes. If this metric is consistently over 80 percent, the watermark delay is rising, and the number of backlogged events is rising, consider increasing streaming units (SUs). High utilization indicates that the job is using close to the maximum allocated resources.
Watermark Delay	Maximum watermark delay across all partitions of all outputs in the job.

Metric dimensions

For information about what metric dimensions are, see Multi-dimensional metrics.

This service has the following dimensions associated with its metrics.

• Logical Name: The input or output name for an Azure Stream Analytics job.
• Partition ID: The ID of the input data partition from an input source.
• Node Name: The identifier of a streaming node that's provisioned when a job runs.

For detailed information, see Dimensions for Azure Stream Analytics metrics.

Resource logs

This section lists the types of resource logs you can collect for this service. The section pulls from the list of all resource logs category types supported in Azure Monitor.

Supported resource logs for Microsoft.StreamAnalytics/streamingjobs

Category	Category display name	Log table	Supports basic log plan	Supports ingestion-time transformation	Example queries	Costs to export
Authoring	Authoring	AzureDiagnostics Logs from multiple Azure resources.	No	No	Queries	No
Execution	Execution	AzureDiagnostics Logs from multiple Azure resources.	No	No	Queries	No

For the resource logs schema and properties for data errors and events, see Resource logs schema.

Azure Monitor Logs tables

This section lists the Azure Monitor Logs tables relevant to this service, which are available for query by Log Analytics using Kusto queries. The tables contain resource log data and possibly more depending on what is collected and routed to them.

Stream Analytics jobs

microsoft.streamanalytics/streamingjobs

• AzureActivity
• AzureMetrics
• AzureDiagnostics

Activity log

The linked table lists the operations that can be recorded in the activity log for this service. These operations are a subset of all the possible resource provider operations in the activity log.

For more information on the schema of activity log entries, see Activity Log schema.

• Microsoft.StreamAnalytics resource provider operations

Related content

• Monitor Azure resources with Azure Monitor
• Monitor Azure Stream Analytics
• Dimensions for Azure Stream Analytics metrics
• Understand and adjust streaming units