Azure networking monitoring solutions in Log Analytics

Log Analytics offers the following solutions for monitoring your networks:

  • Network Performance Monitor (NPM) to
    • Monitor the health of your network
  • Azure Application Gateway analytics to review
    • Azure Application Gateway logs
    • Azure Application Gateway metrics
  • Azure Network Security Group analytics to review
    • Azure Network Security Group logs

Network Performance Monitor (NPM)

The Network Performance Monitor management solution is a network monitoring solution, that monitors the health, availability and reachability of networks. It is used to monitor connectivity between:

  • Public cloud and on-premises
  • Data centers and user locations (branch offices)
  • Subnets hosting various tiers of a multi-tiered application.

For more information, see Network Performance Monitor.

Azure Application Gateway and Network Security Group analytics

To use the solutions:

  1. Add the management solution to Log Analytics, and
  2. Enable diagnostics to direct the diagnostics to a Log Analytics workspace. It is not necessary to write the logs to Azure Blob storage.

You can enable diagnostics and the corresponding solution for either one or both of Application Gateway and Networking Security Groups.

If you do not enable diagnostic logging for a particular resource type, but install the solution, the dashboard blades for that resource are blank and display an error message.

Note

In January 2017, the supported way of sending logs from Application Gateways and Network Security Groups to Log Analytics changed. If you see the Azure Networking Analytics (deprecated) solution, refer to migrating from the old Networking Analytics solution for steps you need to follow.

Review Azure networking data collection details

The Azure Application Gateway analytics and the Network Security Group analytics management solutions collect diagnostics logs directly from Azure Application Gateways and Network Security Groups. It is not necessary to write the logs to Azure Blob storage and no agent is required for data collection.

The following table shows data collection methods and other details about how data is collected for Azure Application Gateway analytics and the Network Security Group analytics.

Platform Direct agent Systems Center Operations Manager agent Azure Operations Manager required? Operations Manager agent data sent via management group Collection frequency
Azure No No Yes No No when logged

Azure Application Gateway analytics solution in Log Analytics

Azure Application Gateway Analytics symbol

The following logs are supported for Application Gateways:

  • ApplicationGatewayAccessLog
  • ApplicationGatewayPerformanceLog
  • ApplicationGatewayFirewallLog

The following metrics are supported for Application Gateways:

  • 5 minute throughput

Install and configure the solution

Use the following instructions to install and configure the Azure Application Gateway analytics solution:

  1. Enable the Azure Application Gateway analytics solution from Azure marketplace or by using the process described in Add Log Analytics solutions from the Solutions Gallery.
  2. Enable diagnostics logging for the Application Gateways you want to monitor.

Enable Azure Application Gateway diagnostics in the portal

  1. In the Azure portal, navigate to the Application Gateway resource to monitor
  2. Select Diagnostics logs to open the following page

    image of Azure Application Gateway resource

  3. Click Turn on diagnostics to open the following page

    image of Azure Application Gateway resource

  4. To turn on diagnostics, click On under Status
  5. Click the checkbox for Send to Log Analytics
  6. Select an existing Log Analytics workspace, or create a workspace
  7. Click the checkbox under Log for each of the log types to collect
  8. Click Save to enable the logging of diagnostics to Log Analytics

Enable Azure network diagnostics using PowerShell

The following PowerShell script provides an example of how to enable diagnostic logging for application gateways.

$workspaceId = "/subscriptions/d2e37fee-1234-40b2-5678-0b2199de3b50/resourcegroups/oi-default-east-us/providers/microsoft.operationalinsights/workspaces/rollingbaskets"

$gateway = Get-AzureRmApplicationGateway -Name 'ContosoGateway'

Set-AzureRmDiagnosticSetting -ResourceId $gateway.ResourceId  -WorkspaceId $workspaceId -Enabled $true

Use Azure Application Gateway analytics

image of Azure Application Gateway analytics tile

After you click the Azure Application Gateway analytics tile on the Overview, you can view summaries of your logs and then drill in to details for the following categories:

  • Application Gateway Access logs
    • Client and server errors for Application Gateway access logs
    • Requests per hour for each Application Gateway
    • Failed requests per hour for each Application Gateway
    • Errors by user agent for Application Gateways
  • Application Gateway performance
    • Host health for Application Gateway
    • Maximum and 95th percentile for Application Gateway failed requests

image of Azure Application Gateway analytics dashboard

image of Azure Application Gateway analytics dashboard

On the Azure Application Gateway analytics dashboard, review the summary information in one of the blades, and then click one to view detailed information on the log search page.

On any of the log search pages, you can view results by time, detailed results, and your log search history. You can also filter by facets to narrow the results.

Azure Network Security Group analytics solution in Log Analytics

Azure Network Security Group Analytics symbol

The following logs are supported for network security groups:

  • NetworkSecurityGroupEvent
  • NetworkSecurityGroupRuleCounter

Install and configure the solution

Use the following instructions to install and configure the Azure Networking Analytics solution:

  1. Enable the Azure Network Security Group analytics solution from Azure marketplace or by using the process described in Add Log Analytics solutions from the Solutions Gallery.
  2. Enable diagnostics logging for the Network Security Group resources you want to monitor.

Enable Azure network security group diagnostics in the portal

  1. In the Azure portal, navigate to the Network Security Group resource to monitor
  2. Select Diagnostics logs to open the following page

    image of Azure Network Security Group resource

  3. Click Turn on diagnostics to open the following page

    image of Azure Network Security Group resource

  4. To turn on diagnostics, click On under Status
  5. Click the checkbox for Send to Log Analytics
  6. Select an existing Log Analytics workspace, or create a workspace
  7. Click the checkbox under Log for each of the log types to collect
  8. Click Save to enable the logging of diagnostics to Log Analytics

Enable Azure network diagnostics using PowerShell

The following PowerShell script provides an example of how to enable diagnostic logging for network security groups

$workspaceId = "/subscriptions/d2e37fee-1234-40b2-5678-0b2199de3b50/resourcegroups/oi-default-east-us/providers/microsoft.operationalinsights/workspaces/rollingbaskets"

$nsg = Get-AzureRmNetworkSecurityGroup -Name 'ContosoNSG'

Set-AzureRmDiagnosticSetting -ResourceId $nsg.ResourceId  -WorkspaceId $workspaceId -Enabled $true

Use Azure Network Security Group analytics

After you click the Azure Network Security Group analytics tile on the Overview, you can view summaries of your logs and then drill in to details for the following categories:

  • Network security group blocked flows
    • Network security group rules with blocked flows
    • MAC addresses with blocked flows
  • Network security group allowed flows
    • Network security group rules with allowed flows
    • MAC addresses with allowed flows

image of Azure Network Security Group analytics dashboard

image of Azure Network Security Group analytics dashboard

On the Azure Network Security Group analytics dashboard, review the summary information in one of the blades, and then click one to view detailed information on the log search page.

On any of the log search pages, you can view results by time, detailed results, and your log search history. You can also filter by facets to narrow the results.

Migrating from the old Networking Analytics solution

In January 2017, the supported way of sending logs from Azure Application Gateways and Azure Network Security Groups to Log Analytics changed. These changes provide the following advantages:

  • Logs are written directly to Log Analytics without the need to use a storage account
  • Less latency from the time when logs are generated to them being available in Log Analytics
  • Fewer configuration steps
  • A common format for all types of Azure diagnostics

To use the updated solutions:

  1. Configure diagnostics to be sent directly to Log Analytics from Azure Application Gateways
  2. Configure diagnostics to be sent directly to Log Analytics from Azure Network Security Groups
  3. Enable the Azure Application Gateway Analytics and the Azure Network Security Group Analytics solution by using the process described in Add Log Analytics solutions from the Solutions Gallery
  4. Update any saved queries, dashboards, or alerts to use the new data type

    • Type is to AzureDiagnostics. You can use the ResourceType to filter to Azure networking logs.

      Instead of: Use:
      Type=NetworkApplicationgateways OperationName=ApplicationGatewayAccess Type=AzureDiagnostics ResourceType=APPLICATIONGATEWAYS OperationName=ApplicationGatewayAccess
      Type=NetworkApplicationgateways OperationName=ApplicationGatewayPerformance Type=AzureDiagnostics ResourceType=APPLICATIONGATEWAYS OperationName=ApplicationGatewayPerformance
      Type=NetworkSecuritygroups Type=AzureDiagnostics ResourceType=NETWORKSECURITYGROUPS
    • For any field that has a suffix of _s, _d, or _g in the name, change the first character to lower case

    • For any field that has a suffix of _o in name, the data is split into individual fields based on the nested field names.
  5. Remove the Azure Networking Analytics (Deprecated) solution.
    • If you are using PowerShell, use Set-AzureOperationalInsightsIntelligencePack -ResourceGroupName <resource group that the workspace is in> -WorkspaceName <name of the log analytics workspace> -IntelligencePackName "AzureNetwork" -Enabled $false

Data collected before the change is not visible in the new solution. You can continue to query for this data using the old Type and field names.

Troubleshooting

Troubleshoot Azure Diagnostics

If you receive the following error message, the Microsoft.insights resource provider is not registered:

Failed to update diagnostics for 'resource'. {"code":"Forbidden","message":"Please register the subscription 'subscription id' with Microsoft.Insights."}

To register the resource provider, perform the following steps in the Azure portal:

  1. In the navigation pane on the left, click Subscriptions
  2. Select the subscription identified in the error message
  3. Click Resource Providers
  4. Find the Microsoft.insights provider
  5. Click the Register link

Register microsoft.insights resource provider

Once the Microsoft.insights resource provider is registered, retry configuring diagnostics.

In PowerShell, if you receive the following error message, you need to update your version of PowerShell:

Set-AzureRmDiagnosticSetting : A parameter cannot be found that matches parameter name 'WorkspaceId'.

Update your version of PowerShell to the November 2016 (v2.3.0), or later, release using the instructions in the Get started with Azure PowerShell cmdlets article.

Next steps