Create, view, and manage log alerts using Azure Monitor

Overview

Log alerts allow users to use a Log Analytics query to evaluate resources logs every set frequency, and fire an alert based on the results. Rules can trigger one or more actions using Action Groups. Learn more about functionality and terminology of log alerts.

This article shows you how to create and manage log alerts using Azure Monitor. Alert rules are defined by three components:

  • Target: A specific Azure resource to monitor.
  • Criteria: Logic to evaluate. If met, the alert fires.
  • Action: Notifications or automation - email, SMS, webhook, and so on.

You can also create log alert rules using Azure Resource Manager templates, which are described in a separate article.

Note

Log data from a Log Analytics workspace can be sent to the Azure Monitor metrics store. Metrics alerts have different behavior, which may be more desirable depending on the data you are working with. For information on what and how you can route logs to metrics, see Metric Alert for Logs.

Create a log alert rule with the Azure portal

Here the steps to get started writing queries for alerts:

  1. Go to the resource you would like to alert on.

  2. Under Monitor, select Logs.

  3. Query the log data that can indicate the issue. You can use the alert query examples topic to understand what you can discover or get started on writing your own query. Also, learn how to create optimized alert queries.

  4. Press on '+ New Alert Rule' button to start the alert creation flow.

    Log Analytics - Set Alert

Note

It is recommended that you create alerts at scale, when using resource access mode for logs, which runs on multiple resources using a resource group or subscription scope. Alerting at scale reduces rule management overhead. To be able to target the resources, please include the resource ID column in the results. Learn more about splitting alerts by dimensions.

Log alert for Log Analytics and Application Insights

  1. If the query syntax is correct, then historical data for the query appears as a graph with the option to tweak the chart period from the last six hours to last week.

    If your query results contain summarized data or project specific columns without time column, the chart shows a single value.

    Configure alert rule

  2. Choose the time range over which to assess the specified condition, using Period option.

  3. Log Alerts can be based on two types of Measures:

    1. Number of results - Count of records returned by the query.
    2. Metric measurement - Aggregate value calculated using summarize grouped by expressions chosen and bin() selection. For example:
    // Reported errors
    union Event, Syslog // Event table stores Windows event records, Syslog stores Linux records
    | where EventLevelName == "Error" // EventLevelName is used in the Event (Windows) records
    or SeverityLevel== "err" // SeverityLevel is used in Syslog (Linux) records
    | summarize AggregatedValue = count() by Computer, bin(TimeGenerated, 15m)
    
  4. For metric measurements alert logic, you can optionally specify how to split the alerts by dimensions using the Aggregate on option. Row grouping expression must be unique and sorted.

    Note

    As bin() can result in uneven time intervals, the alert service will automatically convert bin() function to bin_at() function with appropriate time at runtime, to ensure results with a fixed point.

    Note

    Split by alert dimensions is only available for the current scheduledQueryRules API. If you use the legacy Log Analytics Alert API, you will need to switch. Learn more about switching. Resource centric alerting at scale is only supported in the API version 2020-05-01-preview and above.

    aggregate on option

  5. Next, based on the preview data set the Operator, Threshold Value, and Frequency.

  6. You can also optionally set the number of violations to trigger an alert by using Total or Consecutive Breaches.

  7. Select Done.

  8. Define the Alert rule name, Description, and select the alert Severity. These details are used in all alert actions. Additionally, you can choose to not activate the alert rule on creation by selecting Enable rule upon creation.

  9. Choose if you want to suppress rule actions for a time after an alert is fired, use the Suppress Alerts option. The rule will still run and create alerts but actions won't be triggered to prevent noise. Mute actions value must be greater than the frequency of alert to be effective.

    Suppress Alerts for Log Alerts

  10. Specify if the alert rule should trigger one or more Action Groups when alert condition is met.

    Note

    Refer to the Azure subscription service limits for limits on the actions that can be performed.

  11. You can optionally customize actions in log alert rules:

    • Custom Email Subject: Overrides the e-mail subject of email actions. You can't modify the body of the mail and this field isn't for email addresses.
    • Include custom Json payload: Overrides the webhook JSON used by Action Groups assuming the action group contains a webhook action. Learn more about webhook action for Log Alerts.

    Action Overrides for Log Alerts

  12. If all fields are correctly set, the Create alert rule button can be clicked and an alert is created.

    Within a few minutes, the alert is active and triggers as previously described.

    Rule Creation

Creating log alert for Log Analytics and Application Insights from the alerts management

Note

Creation from alerts management is currently not supported for resource centric logs

  1. In the portal, select Monitor then choose Alerts.

    Monitoring

  2. Select New Alert Rule.

    Add Alert

  3. The Create Alert pane appears. It has four parts:

    • The resource to which the alert applies.
    • The condition to check.
    • The actions to take if the condition is true.
    • The details to name and describe the alert.

    Create rule

  4. Press on Select Resource button. Filter by choosing the Subscription, Resource Type, and select a resource. Ensure the resource has logs available.

    Select resource

  5. Next, use the add Condition button to view list of signal options available for the resource. Select Custom log search option.

    Select a resource - custom log search

    Note

    The alerts portal lists saved queries from Log Analytics and Application Insights and they can be used as template alert queries.

  6. Once selected, write, paste, or edit the alerting query in the Search Query field.

  7. Continue to the next steps described in the last section.

Log alert for all other resource types

Note

There are currently no additional charges for the API version 2020-05-01-preview and resource centric log alerts. Pricing for features that are in preview will be announced in the future and a notice provided prior to start of billing. Should you choose to continue using new API version and resource centric log alerts after the notice period, you will be billed at the applicable rate.

  1. Start from the Condition tab:

    1. Check that the Measure, Aggregation type, and Aggregation granularity are correct.

      1. By default, the rule counts the number of results in the last 5 minutes.
      2. If we detect summarized query results, the rule will be updated automatically within a few seconds to capture that.
    2. Choose alert splitting by dimensions, if needed:

      • Resource ID column is selected automatically, if detected, and changes the context of the fired alert to the record's resource.
      • Resource ID column can be de-selected to fire alerts on subscription or resource groups. De-selecting is useful when query results are based on cross-resources. For example, a query that check if 80% of the resource group's virtual machines are experiencing high CPU usage.
      • Up to six additional splittings can be also selected for any number or text columns types using the dimensions table.
      • Alerts are fired separately according to splitting based on unique combinations and alert payload includes this information.

      Select aggregation parameters and splitting

    3. The Preview chart shows query evaluations results over time. You can change the chart period or select different time series that resulted from unique alert splitting by dimensions.

      Preview chart

    4. Next, based on the preview data, set the Alert logic; Operator, Threshold Value, and Frequency.

      Preview chart with threshold and alert logic

    5. You can optionally set Number of violations to trigger the alert in the Advanced options section.

      Advanced options

  2. In the Actions tab, select or create the required action groups.

    Actions tab

  3. In the Details tab, define the Alert rule details, and Project details. You can optionally set whether to not Start running now, or Mute Actions for a period after the alert rule fires.

    Note

    Log alert rules are currently stateless and fires an action every time an alert is created unless muting is defined.

    Details tab

  4. In the Tags tab, set any required tags on the alert rule resource.

    Tags tab

  5. In the Review + create tab, a validation will run and inform of any issues. Review and approve the rule definition.

  6. If all fields are correct, select the Create button and complete the alert rule creation. All alerts can be viewed from the alerts management.

    Review and create tab

View & manage log alerts in Azure portal

  1. In the portal, select the relevant resource or the Monitor service. Then select Alerts in the Monitor section.

  2. The alerts management displays all alerts that fired. Learn more about the alert management.

    Note

    Log alert rules are currently stateless and do not resolve.

  3. Select Manage alert rules button on the top bar to edit rules:

     manage alert rules

Managing log alerts using PowerShell

Note

This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure PowerShell.

Note

PowerShell is not currently supported in the API version 2020-05-01-preview

PowerShell cmdlets listed below are available to manage rules with the Scheduled Query Rules API.

Note

ScheduledQueryRules PowerShell cmdlets can only manage rules created in the current Scheduled Query Rules API. Log alert rules created using legacy Log Analytics Alert API can only be managed using PowerShell only after switching to Scheduled Query Rules API.

Here are example steps for creating a log alert rule using the PowerShell:

$source = New-AzScheduledQueryRuleSource -Query 'Heartbeat | summarize AggregatedValue = count() by bin(TimeGenerated, 5m), _ResourceId' -DataSourceId "/subscriptions/a123d7efg-123c-1234-5678-a12bc3defgh4/resourceGroups/contosoRG/providers/microsoft.OperationalInsights/workspaces/servicews"

$schedule = New-AzScheduledQueryRuleSchedule -FrequencyInMinutes 15 -TimeWindowInMinutes 30

$metricTrigger = New-AzScheduledQueryRuleLogMetricTrigger -ThresholdOperator "GreaterThan" -Threshold 2 -MetricTriggerType "Consecutive" -MetricColumn "_ResourceId"

$triggerCondition = New-AzScheduledQueryRuleTriggerCondition -ThresholdOperator "LessThan" -Threshold 5 -MetricTrigger $metricTrigger

$aznsActionGroup = New-AzScheduledQueryRuleAznsActionGroup -ActionGroup "/subscriptions/a123d7efg-123c-1234-5678-a12bc3defgh4/resourceGroups/contosoRG/providers/microsoft.insights/actiongroups/sampleAG" -EmailSubject "Custom email subject" -CustomWebhookPayload "{ `"alert`":`"#alertrulename`", `"IncludeSearchResults`":true }"

$alertingAction = New-AzScheduledQueryRuleAlertingAction -AznsAction $aznsActionGroup -Severity "3" -Trigger $triggerCondition

New-AzScheduledQueryRule -ResourceGroupName "contosoRG" -Location "Region Name for your Application Insights App or Log Analytics Workspace" -Action $alertingAction -Enabled $true -Description "Alert description" -Schedule $schedule -Source $source -Name "Alert Name"

You can also create the log alert using a template and parameters files using PowerShell:

Connect-AzAccount

Select-AzSubscription -SubscriptionName <yourSubscriptionName>

New-AzResourceGroupDeployment -Name AlertDeployment -ResourceGroupName ResourceGroupofTargetResource `
  -TemplateFile mylogalerttemplate.json -TemplateParameterFile mylogalerttemplate.parameters.json

Managing log alerts using CLI

Note

Azure CLI support is only available for the scheduledQueryRules API version 2020-05-01-preview and above. Pervious API version can use the Azure Resource Manager CLI with templates as described below. If you use the legacy Log Analytics Alert API, you will need to switch to use CLI. Learn more about switching.

The previous sections described how to create, view, and manage log alert rules using Azure portal. This section will describe how to do the same using cross-platform Azure CLI. Quickest way to start using Azure CLI is through Azure Cloud Shell. For this article, we'll use Cloud Shell.

  1. Go to Azure portal, select Cloud Shell.

  2. At the prompt, you can use commands with --help option to learn more about the command and how to use it. For example, the following command shows you the list of commands available for creating, viewing, and managing log alerts:

    az monitor scheduled-query --help
    
  3. You can create a log alert rule that monitors count of system event errors:

    az monitor scheduled-query create -g {ResourceGroup} -n {nameofthealert} --scopes {vm_id} --condition "count \'union Event, Syslog | where TimeGenerated > ago(1h) | where EventLevelName == \"Error\" or SeverityLevel== \"err\"\' > 2" --description {descriptionofthealert}
    
  4. You can view all the log alerts in a resource group using the following command:

    az monitor scheduled-query list -g {ResourceGroup}
    
  5. You can see the details of a particular log alert rule using the name or the resource ID of the rule:

    az monitor scheduled-query show -g {ResourceGroup} -n {AlertRuleName}
    
    az monitor scheduled-query show --ids {RuleResourceId}
    
  6. You can disable a log alert rule using the following command:

    az monitor scheduled-query update -g {ResourceGroup} -n {AlertRuleName} --enabled false
    
  7. You can delete a log alert rule using the following command:

    az monitor scheduled-query delete -g {ResourceGroup} -n {AlertRuleName}
    

You can also use Azure Resource Manager CLI with templates files:

az login

az group deployment create \
    --name AlertDeployment \
    --resource-group ResourceGroupofTargetResource \
    --template-file mylogalerttemplate.json \
    --parameters @mylogalerttemplate.parameters.json

On success for creation, 201 is returned. On success for update, 200 is returned.

Next steps