Configure Network Performance Monitor for ExpressRoute
Network Performance Monitor (NPM) is a cloud-based network monitoring solution that monitors connectivity between Azure cloud deployments and on-premises locations (Branch offices, etc.). NPM is part of Microsoft Operations Management Suite (OMS). NPM now offers an extension for ExpressRoute that lets you monitor network performance over ExpressRoute circuits that are configured to use Private Peering. When you configure NPM for ExpressRoute, you can detect network issues to identify and eliminate.
Monitor loss and latency across various VNets and set alerts
Monitor all paths (including redundant paths) on the network
Troubleshoot transient and point-in-time network issues that are difficult to replicate
Help determine a specific segment on the network that is responsible for degraded performance
Get throughput per virtual network (If you have agents installed in each VNet)
See the ExpressRoute system state from a previous point in time
You can monitor ExpressRoute circuits in any part of the world by using a workspace that is hosted in one of the following regions:
- West Europe
- West Central US
- East US
- South East Asia
- South East Australia
Monitoring agents are installed on multiple servers, both on-premises and in Azure. The agents communicate with each other, but do not send data, they send TCP handshake packets. The communication between the agents allows Azure to map the network topology and path the traffic could take.
- Create an NPM Workspace in the one of the supported regions.
- Install and configure software agents:
- Install monitoring agents on the on-premises servers and the Azure VMs.
- Configure settings on the monitoring agent servers to allow the monitoring agents to communicate. (Open firewall ports, etc.)
- Configure network security group (NSG) rules to allow the monitoring agent installed on Azure VMs to communicate with on-premises monitoring agents.
- Set up monitoring: Auto-Discover and manage which networks are visible in NPM.
If you are already using Network Performance Monitor to monitor other objects or services, and you already have Workspace in one of the supported regions, you can skip Step 1 and Step 2, and begin your configuration with Step 3.
Step 1: Create a Workspace
Create a workspace in the subscription that has the VNets link to the ExpressRoute circuit(s).
- In the Azure portal, select the Subscription that has the VNETs peered to your ExpressRoute circuit. Then search the list of services in the Marketplace for 'Network Performance Monitor'. In the return, click to open the Network Performance Monitor page.
You may create a new workspace or use an existing workspace. If you wish to use an existing workspace, you must ensure that the workspace has been migrated to the new query language. More information...
- At the bottom of the main Network Performance Monitor page, click Create to open Network Performance Monitor - Create new solution page. Click OMS Workspace - select a workspace to open the Workspaces page. Click + Create New Workspace to open the Workspace page.
On the OMS Workspace page, select Create New and configure the following settings:
- OMS Workspace - Type a name for your Workspace.
- Subscription - If you have multiple subscriptions, choose the one you want to associate with the new Workspace.
- Resource group - Create a resource group, or use an existing one.
- Location - You must select a supported region.
- Pricing tier - Select 'Free'
The ExpressRoute circuit could be anywhere in the world and does not have to be in the same region as the Workspace.
- Click OK to save and deploy the settings template. Once the template validates, click Create to deploy the Workspace.
After the Workspace has been deployed, navigate to the NetworkMonitoring(name) resource that you created. Validate the settings, then click Solution requires additional configuration.
Step 2: Install and configure agents
2.1: Download the agent setup file
Go to the Common Settings tab of the Network Performance Monitor Configuration page for your resource. Click the agent that corresponds to your server's processor from the Install OMS Agents section and download the setup file.
The agent must be installed on a Windows Server (2008 SP1 or later). Monitoring of ExpressRoute circuits using Windows Desktop OS and Linux OS is not supported.
- Next, copy the Workspace ID and Primary Key to Notepad.
From the Configure OMS Agents for monitoring using TCP protocol section, download the Powershell Script. The PowerShell script helps you open the relevant firewall port for the TCP transactions.
2.2: Install a monitoring agent on each monitoring server (on each VNET that you want to monitor)
We recommend that you install at least two agents on each side of the ExpressRoute connection (i.e., on-premises, Azure VNETs) for redundancy. Use the following steps to install agents:
- Run Setup to install the agent on each server that you want to use for monitoring ExpressRoute. The server you use for monitoring can either be a VM, or on-premises and must have Internet access. You need to install at least one agent on-premises, and one agent on each network segment that you want to monitor in Azure.
- On the Welcome page, click Next.
- On the License Terms page, read the license, and then click I Agree.
- On the Destination Folder page, change or keep the default installation folder, and then click Next.
On the Agent Setup Options page, you can choose to connect the agent to Azure Log Analytics (OMS) or Operations Manager. Or, you can leave the choices blank if you want to configure the agent later. After making your selection(s), click Next.
If you chose to connect to Azure Log Analytics (OMS), paste the Workspace ID and Workspace Key (Primary Key) that you copied into Notepad in the previous section. Then, click Next.
If you chose to connect to Operations Manager, on the Management Group Configuration page, type the Management Group Name, Management Server, and the Management Server Port. Then, click Next.
On the Agent Action Account page, choose either the Local System account, or Domain or Local Computer Account. Then, click Next.
- On the Ready to Install page, review your choices, and then click Install.
- On the Configuration completed successfully page, click Finish.
When complete, the Microsoft Monitoring Agent appears in the Control Panel. You can review your configuration there, and verify that the agent is connected to Operational Insights (OMS). When connected to OMS, the agent displays a message stating: The Microsoft Monitoring Agent has successfully connected to the Microsoft Operations Management Suite service.
Please repeat this for each VNET that you need to be monitored.
2.3: Configure proxy settings (optional)
If you are using a web proxy to access the Internet, use the following steps to configure proxy settings for the Microsoft Monitoring Agent. Perform these steps for each server. If you have many servers that you need to configure, you might find it easier to use a script to automate this process. If so, see To configure proxy settings for the Microsoft Monitoring Agent using a script.
To configure proxy settings for the Microsoft Monitoring Agent using the Control Panel:
- Open the Control Panel.
- Open Microsoft Monitoring Agent.
- Click the Proxy Settings tab.
Select Use a proxy server and type the URL and port number, if one is needed. If your proxy server requires authentication, type the username and password to access the proxy server.
2.4: Verify agent connectivity
You can easily verify whether your agents are communicating.
- On a server with the monitoring agent, open the Control Panel.
- Open the Microsoft Monitoring Agent.
- Click the Azure Log Analytics (OMS) tab.
In the Status column, you should see that the agent connected successfully to the Operations Management Suite service.
2.5: Open the firewall ports on the monitoring agent servers
To use the TCP protocol, you must open firewall ports to ensure that the monitoring agents can communicate.
You can run a PowerShell script that creates the registry keys required by the Network Performance Monitor, as well as creating the Windows Firewall rules to allow monitoring agents to create TCP connections with each other. The registry keys created by the script also specify whether to log the debug logs, and the path for the logs file. It also defines the agent TCP port used for communication. The values for these keys are automatically set by the script, so you should not manually change these keys.
Port 8084 is opened by default. You can use a custom port by providing the parameter 'portNumber' to the script. However, if you do so, you must specify the same port for all the servers on which you run the script.
The 'EnableRules' PowerShell script configures Windows Firewall rules only on the server where the script is run. If you have a network firewall, you should make sure that it allows traffic destined for the TCP port being used by Network Performance Monitor.
On the agent servers, open a PowerShell window with administrative privileges. Run the EnableRules PowerShell script (which you downloaded earlier). Don't use any parameters.
Step 3: Configure network security group rules
For monitoring agent servers that are in Azure, you must configure network security group (NSG) rules to allow TCP traffic on a port used by NPM for synthetic transactions. The default port is 8084. This allows a monitoring agent installed on Azure VM to communicate with an on-premises monitoring agent.
For more information about NSG, see Network Security Groups.
Make sure that you have installed the agents (both the on-premises server agent and the Azure server agent) and have run the PowerShell script before proceeding with this step.
Step 4: Configure NPM for ExpressRoute monitoring
After you complete the previous sections, you can set up monitoring.
Navigate to the Network Performance Monitor overview tile by going to the All Resources page, and clicking on the whitelisted NPM Workspace.
Click the Network Performance Monitor overview tile to bring up the dashboard. The dashboard contains an ExpressRoute page, which shows that ExpressRoute is in an 'unconfigured state'. Click Feature Setup to open the Network Performance Monitor configuration page.
On the configuration page, navigate to the 'ExpressRoute Peerings' tab, located on the left side panel. Click Discover Now.
When discovery completes, you see rules for unique Circuit name and VNet name. Initially, these rules are disabled. Enable the rules, then select the monitoring agents and threshold values.
After enabling the rules and selecting the values and agents you want to monitor, there is a wait of approximately 30-60 minutes for the values to begin populating and the ExpressRoute Monitoring tiles to become available. Once you see the monitoring tiles, your ExpressRoute circuits and connection resources are being monitored by NPM.
Step 5: View monitoring tiles
Network Performance Monitor page
The NPM page contains a page for ExpressRoute that shows an overview of the health of ExpressRoute circuits and peerings.
List of circuits
To see a list of all monitored ExpressRoute circuits, click on the ExpressRoute circuits tile. You can select a circuit and view its health state, trend charts for packet loss, bandwidth utilization, and latency. The charts are interactive. You can select a custom time window for plotting the charts. You can drag the mouse over an area on the chart to zoom in and see fine-grained data points.
Trend of Loss, Latency and Throughput
The bandwidth, latency, and loss charts are interactive. You can zoom into any section of these charts, using mouse controls. You can also see the bandwidth, latency, and loss data for other intervals by clicking Date/Time, located below the Actions button on the upper left.
Clicking on the Private Peerings tile on the dashboard brings up a list of all connections to virtual networks over private peering. Here, you can select a virtual network connection and view its health state, trend charts for packet loss, bandwidth utilization, and latency.
To view circuit topology, click on the Topology tile. This takes you to the topology view of the selected circuit or peering. The topology diagram provides the latency for each segment on the network and each layer 3 hop is represented by a node of the diagram. Clicking on a hop reveals more details about the hop. You can increase the level of visibility to include on-premises hops by moving the slider bar below Filters. Moving the slider bar to the left or right, increases/decreases the number of hops in the topology graph. The latency across each segment is visible, which allows for faster isolation of high latency segments on your network.
Detailed Topology view of a circuit
This view shows VNet connections.