Azure networking monitoring solutions in Azure Monitor

Note

This article has been updated to use the Azure Az PowerShell module. The Az PowerShell module is the recommended PowerShell module for interacting with Azure. To get started with the Az PowerShell module, see Install Azure PowerShell. To learn how to migrate to the Az PowerShell module, see Migrate Azure PowerShell from AzureRM to Az.

Azure Monitor offers the following solutions for monitoring your networks:

  • Network Performance Monitor (NPM) to
    • Monitor the health of your network
  • Azure Application Gateway analytics to review
    • Azure Application Gateway logs
    • Azure Application Gateway metrics
  • Solutions to monitor and audit network activity on your cloud network

Network Performance Monitor (NPM)

The Network Performance Monitor management solution is a network monitoring solution, that monitors the health, availability and reachability of networks. It is used to monitor connectivity between:

  • Public cloud and on-premises
  • Data centers and user locations (branch offices)
  • Subnets hosting various tiers of a multi-tiered application.

For more information, see Network Performance Monitor.

Network Security Group analytics

  1. Add the management solution to Azure Monitor, and
  2. Enable diagnostics to direct the diagnostics to a Log Analytics workspace in Azure Monitor. It is not necessary to write the logs to Azure Blob storage.

If diagnostic logs are not enabled, the dashboard blades for that resource are blank and display an error message.

Azure Application Gateway analytics

  1. Enable diagnostics to direct the diagnostics to a Log Analytics workspace in Azure Monitor.
  2. Consume the detailed summary for your resource using the workbook template for Application Gateway.

If diagnostic logs are not enabled for Application Gateway, only the default metric data would be populated within the workbook.

Note

In January 2017, the supported way of sending logs from Application Gateways and Network Security Groups to a Log Analytics workspace changed. If you see the Azure Networking Analytics (deprecated) solution, refer to migrating from the old Networking Analytics solution for steps you need to follow.

Review Azure networking data collection details

The Azure Application Gateway analytics and the Network Security Group analytics management solutions collect diagnostics logs directly from Azure Application Gateways and Network Security Groups. It is not necessary to write the logs to Azure Blob storage and no agent is required for data collection.

The following table shows data collection methods and other details about how data is collected for Azure Application Gateway analytics and the Network Security Group analytics.

Platform Direct agent Systems Center Operations Manager agent Azure Operations Manager required? Operations Manager agent data sent via management group Collection frequency
Azure when logged

Enable Azure Application Gateway diagnostics in the portal

  1. In the Azure portal, navigate to the Application Gateway resource to monitor.

  2. Select Diagnostics Settings to open the following page.

    Screenshot of the Diagnostics Settings config for Application Gateway resource.

    Screenshot of the page for configuring Diagnostics settings.

  3. Click the checkbox for Send to Log Analytics.

  4. Select an existing Log Analytics workspace, or create a workspace.

  5. Click the checkbox under Log for each of the log types to collect.

  6. Click Save to enable the logging of diagnostics to Azure Monitor.

Enable Azure network diagnostics using PowerShell

The following PowerShell script provides an example of how to enable resource logging for application gateways.

$workspaceId = "/subscriptions/d2e37fee-1234-40b2-5678-0b2199de3b50/resourcegroups/oi-default-east-us/providers/microsoft.operationalinsights/workspaces/rollingbaskets"

$gateway = Get-AzApplicationGateway -Name 'ContosoGateway'

Set-AzDiagnosticSetting -ResourceId $gateway.ResourceId  -WorkspaceId $workspaceId -Enabled $true

Accessing Azure Application Gateway analytics via Azure Monitor Network insights

Application insights can be accessed via the insights tab within your Application Gateway resource.

Screenshot of Application Gateway insights

The "view detailed metrics" tab will open up the pre-populated workbook summarizing the data from your Application Gateway.

Screenshot of Application Gateway workbook

New capabilities with Azure Monitor Network Insights workbook

Note

There are no additional costs associated with Azure Monitor Insights workbook. Log Analytics workspace will continue to be billed as per usage.

The Network Insights workbook allows you to take advantage of the latest capabilities of Azure Monitor and Log Analytics including:

  • Centralized console for monitoring and troubleshooting with both metric and log data.

  • Flexible canvas to support creation of custom rich visualizations.

  • Ability to consume and share workbook templates with wider community.

To find more information about the capabilities of the new workbook solution check out Workbooks-overview

Migrating from Azure Gateway analytics solution to Azure Monitor workbooks

Note

Azure Monitor Network Insights workbook is the recommended solution for accessing metric and log analytics for your Application Gateway resources.

  1. Ensure diagnostics settings are enabled to store logs into a Log Analytics workspace. If it is already configured, Azure Monitor Network Insights workbook will be able to consume data from the same location and no additional changes are required.

Note

All past data is already available within the workbook from the point diagnostic settings were originally enabled. There is no data transfer required.

  1. Access the default insights workbook for your Application Gateway resource. All existing insights supported by the Application Gateway analytics solution will be already present in the workbook. You can extend this by adding custom visualizations based on metric & log data.

  2. After you are able to see all your metric and log insights, to clean up the Azure Gateway analytics solution from your workspace, you can delete the solution from the solution resource page.

Screenshot of the delete option for Azure Application Gateway analytics solution.

Azure Network Security Group analytics solution in Azure Monitor

Azure Network Security Group Analytics symbol

Note

The Network Security Group analytics solution is moving to community support since its functionality has been replaced by Traffic Analytics.

  • The solution is now available in Azure Quickstart Templates and will soon no longer be available in the Azure Marketplace.
  • For existing customers who already added the solution to their workspace, it will continue to function with no changes.
  • Microsoft will continue to support sending NSG resource logs to your workspace using Diagnostics Settings.

The following logs are supported for network security groups:

  • NetworkSecurityGroupEvent
  • NetworkSecurityGroupRuleCounter

Install and configure the solution

Use the following instructions to install and configure the Azure Networking Analytics solution:

  1. Enable the Azure Network Security Group analytics solution by using the process described in Add Azure Monitor solutions from the Solutions Gallery.
  2. Enable diagnostics logging for the Network Security Group resources you want to monitor.

Enable Azure network security group diagnostics in the portal

  1. In the Azure portal, navigate to the Network Security Group resource to monitor

  2. Select Diagnostics logs to open the following page

    Screenshot of the Diagnostics logs page for a Network Security Group resource showing the option to Turn on diagnostics.

  3. Click Turn on diagnostics to open the following page

    Screenshot of the page for configuring Diagnostics settings. Status is set to On, Send to Log Analytics is selected and two Log types are selected.

  4. To turn on diagnostics, click On under Status

  5. Click the checkbox for Send to Log Analytics

  6. Select an existing Log Analytics workspace, or create a workspace

  7. Click the checkbox under Log for each of the log types to collect

  8. Click Save to enable the logging of diagnostics to Log Analytics

Enable Azure network diagnostics using PowerShell

The following PowerShell script provides an example of how to enable resource logging for network security groups

$workspaceId = "/subscriptions/d2e37fee-1234-40b2-5678-0b2199de3b50/resourcegroups/oi-default-east-us/providers/microsoft.operationalinsights/workspaces/rollingbaskets"

$nsg = Get-AzNetworkSecurityGroup -Name 'ContosoNSG'

Set-AzDiagnosticSetting -ResourceId $nsg.ResourceId  -WorkspaceId $workspaceId -Enabled $true

Use Azure Network Security Group analytics

After you click the Azure Network Security Group analytics tile on the Overview, you can view summaries of your logs and then drill in to details for the following categories:

  • Network security group blocked flows
    • Network security group rules with blocked flows
    • MAC addresses with blocked flows
  • Network security group allowed flows
    • Network security group rules with allowed flows
    • MAC addresses with allowed flows

Screenshot of tiles with data for Network security group blocked flows, including Rules with blocked flows and MAC addresses with blocked flows.

Screenshot of tiles with data for Network security group allowed flows, including Rules with allowed flows and MAC addresses with allowed flows.

On the Azure Network Security Group analytics dashboard, review the summary information in one of the blades, and then click one to view detailed information on the log search page.

On any of the log search pages, you can view results by time, detailed results, and your log search history. You can also filter by facets to narrow the results.

Migrating from the old Networking Analytics solution

In January 2017, the supported way of sending logs from Azure Application Gateways and Azure Network Security Groups to a Log Analytics workspace changed. These changes provide the following advantages:

  • Logs are written directly to Azure Monitor without the need to use a storage account
  • Less latency from the time when logs are generated to them being available in Azure Monitor
  • Fewer configuration steps
  • A common format for all types of Azure diagnostics

To use the updated solutions:

  1. Configure diagnostics to be sent directly to Azure Monitor from Azure Application Gateways
  2. Configure diagnostics to be sent directly to Azure Monitor from Azure Network Security Groups
  3. Enable the Azure Application Gateway Analytics and the Azure Network Security Group Analytics solution by using the process described in Add Azure Monitor solutions from the Solutions Gallery
  4. Update any saved queries, dashboards, or alerts to use the new data type
    • Type is to AzureDiagnostics. You can use the ResourceType to filter to Azure networking logs.

      Instead of: Use:
      NetworkApplicationgateways | where OperationName=="ApplicationGatewayAccess" AzureDiagnostics | where ResourceType=="APPLICATIONGATEWAYS" and OperationName=="ApplicationGatewayAccess"
      NetworkApplicationgateways | where OperationName=="ApplicationGatewayPerformance" AzureDiagnostics | where ResourceType=="APPLICATIONGATEWAYS" and OperationName=="ApplicationGatewayPerformance"
      NetworkSecuritygroups AzureDiagnostics | where ResourceType=="NETWORKSECURITYGROUPS"
    • For any field that has a suffix of _s, _d, or _g in the name, change the first character to lower case

    • For any field that has a suffix of _o in name, the data is split into individual fields based on the nested field names.

  5. Remove the Azure Networking Analytics (Deprecated) solution.
    • If you are using PowerShell, use Set-AzureOperationalInsightsIntelligencePack -ResourceGroupName <resource group that the workspace is in> -WorkspaceName <name of the log analytics workspace> -IntelligencePackName "AzureNetwork" -Enabled $false

Data collected before the change is not visible in the new solution. You can continue to query for this data using the old Type and field names.

Troubleshooting

Troubleshoot Azure Diagnostics

If you receive the following error message, the Microsoft.insights resource provider is not registered:

Failed to update diagnostics for 'resource'. {"code":"Forbidden","message":"Please register the subscription 'subscription id' with Microsoft.Insights."}

To register the resource provider, perform the following steps in the Azure portal:

  1. In the navigation pane on the left, click Subscriptions
  2. Select the subscription identified in the error message
  3. Click Resource Providers
  4. Find the Microsoft.insights provider
  5. Click the Register link

Register microsoft.insights resource provider

Once the Microsoft.insights resource provider is registered, retry configuring diagnostics.

In PowerShell, if you receive the following error message, you need to update your version of PowerShell:

Set-AzDiagnosticSetting : A parameter cannot be found that matches parameter name 'WorkspaceId'.

Update your version of Azure PowerShell, follow the instructions in the Install Azure PowerShell article.

Next steps