Tutorial: Log network traffic to and from a virtual machine using the Azure portal
A network security group (NSG) enables you to filter inbound traffic to, and outbound traffic from, a virtual machine (VM). You can log network traffic that flows through an NSG with Network Watcher's NSG flow log capability.
In this tutorial, you learn how to:
- Create a VM with a network security group
- Enable Network Watcher and register the Microsoft.Insights provider
- Enable a traffic flow log for an NSG, using Network Watcher's NSG flow log capability
- Download logged data
- View logged data
Prerequisites
- An Azure account with an active subscription. Create an account for free.
Create a virtual machine
Sign in to the Azure portal.
In the search box at the top of the portal, enter Virtual machine. Select Virtual machines.
In Virtual machines, select + Create then + Virtual machine.
Enter or select the following information in Create a virtual machine.
Setting Value Project details Subscription Select your subscription. Resource group Select Create new. Enter myResourceGroup in Name. Select OK. Instance details Virtual machine name Enter myVM. Region Select (US) East US. Availability options Select No infrastructure redundancy required. Security type Leave the default of Standard. Image Select Windows Server 2022 Datacenter: Azure Edition - Gen2. Azure Spot instance Leave the default. Size Select a size. Administrator account Authentication type Select SSH public key. Username Enter a username. Password Enter a password. Confirm password Confirm password. Inbound port rules Public inbound ports Leave the default of Allow selected ports. Select inbound ports Leave the default of RDP (3389). Select Review + create.
Select Create.
The virtual machine takes a few minutes to create. Don't continue with remaining steps until the VM has finished creating. While the portal creates the virtual machine, it also creates a network security group with the name myVM-nsg, and associates it to the network interface for the VM.
Enable Network Watcher
If you already have a network watcher enabled in the East US region, skip to Register Insights provider.
In the search box at the top of the portal, enter Network Watcher. Select Network Watcher in the search results.
In the Overview page of Network Watcher, select + Add.
Select your subscription in Add network watcher. Select (US) East US in Region.
Select Add.
Register Insights provider
NSG flow logging requires the Microsoft.Insights provider. To register the provider, complete the following steps:
In the search box at the top of the portal, enter Subscriptions. Select Subscriptions in the search results.
Select the subscription you want to enable the provider for in Subscriptions.
Select Resource providers in Settings of your subscription.
Enter Microsoft.Insights in the filter box.
Confirm the status of the provider displayed is Registered. If the status is Unregistered, select the provider then select Register.
Enable NSG flow log
NSG flow log data is written to an Azure Storage account. Complete the following steps to create a storage account for the log data.
In the search box at the top of the portal, enter Storage account. Select Storage accounts in the search results.
In Storage accounts, select + Create.
Enter or select the following information in Create a storage account.
Setting Value Project details Subscription Select your subscription. Resource group Select myResourceGroup. Instance details Storage account name Enter a name for your storage account. Must be 3-24 characters in length, can only contain lowercase letters and numbers, and must be unique across all Azure Storage. Region Select (US)East US. Performance Leave the default of Standard. Redundancy Leave the default of Geo-redundant storage (GRS). Select Review + create.
Select Create.
The storage account may take around minute to create. Don't continue with remaining steps until the storage account is created. In all cases, the storage account must be in the same region as the NSG.
In the search box at the top of the portal, enter Network Watcher. Select Network Watcher in the search results.
Select NSG flow logs in Logs.
In Network Watcher | NSG flow logs, select + Create.
Enter or select the following information in Create a flow log.
Setting Value Project details Subscription Select your subscription. Network Security Group Select myVM-nsg. Flow Log Name Leave the default of myVM-nsg-myResourceGroup-flowlog. Instance details Select storage account Subscription Select your subscription. Storage Accounts Select the storage account you created in the previous steps. Retention (days) Enter a retention time for the logs. Select Review + create.
Select Create.
Download flow log
In the search box at the top of the portal, enter Storage account. Select Storage accounts in the search results.
Select the storage account you created in the previous steps.
In Data storage, select Containers.
Select the insights-logs-networksecuritygroupflowevent container.
In the container, navigate the folder hierarchy until you get to a PT1H.json file. Log files are written to a folder hierarchy that follows the following naming convention:
https://{storageAccountName}.blob.core.windows.net/insights-logs-networksecuritygroupflowevent/resourceId=/SUBSCRIPTIONS/{subscriptionID}/RESOURCEGROUPS/{resourceGroupName}/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/{nsgName}/y={year}/m={month}/d={day}/h={hour}/m=00/macAddress={macAddress}/PT1H.json
Select ... to the right of the PT1H.json file, then select Download.
View flow log
The following example json displays data that you'll see in the PT1H.json file for each flow logged:
Version 1 flow log event
{
"time": "2018-05-01T15:00:02.1713710Z",
"systemId": "<Id>",
"category": "NetworkSecurityGroupFlowEvent",
"resourceId": "/SUBSCRIPTIONS/<Id>/RESOURCEGROUPS/MYRESOURCEGROUP/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/MYVM-NSG",
"operationName": "NetworkSecurityGroupFlowEvents",
"properties": {
"Version": 1,
"flows": [
{
"rule": "UserRule_default-allow-rdp",
"flows": [
{
"mac": "000D3A170C69",
"flowTuples": [
"1525186745,192.168.1.4,10.0.0.4,55960,3389,T,I,A"
]
}
]
}
]
}
}
Version 2 flow log event
{
"time": "2018-11-13T12:00:35.3899262Z",
"systemId": "a0fca5ce-022c-47b1-9735-89943b42f2fa",
"category": "NetworkSecurityGroupFlowEvent",
"resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-000000000000/RESOURCEGROUPS/FABRIKAMRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/FABRIAKMVM1-NSG",
"operationName": "NetworkSecurityGroupFlowEvents",
"properties": {
"Version": 2,
"flows": [
{
"rule": "DefaultRule_DenyAllInBound",
"flows": [
{
"mac": "000D3AF87856",
"flowTuples": [
"1542110402,94.102.49.190,10.5.16.4,28746,443,U,I,D,B,,,,",
"1542110424,176.119.4.10,10.5.16.4,56509,59336,T,I,D,B,,,,",
"1542110432,167.99.86.8,10.5.16.4,48495,8088,T,I,D,B,,,,"
]
}
]
},
{
"rule": "DefaultRule_AllowInternetOutBound",
"flows": [
{
"mac": "000D3AF87856",
"flowTuples": [
"1542110377,10.5.16.4,13.67.143.118,59831,443,T,O,A,B,,,,",
"1542110379,10.5.16.4,13.67.143.117,59932,443,T,O,A,E,1,66,1,66",
"1542110379,10.5.16.4,13.67.143.115,44931,443,T,O,A,C,30,16978,24,14008",
"1542110406,10.5.16.4,40.71.12.225,59929,443,T,O,A,E,15,8489,12,7054"
]
}
]
}
]
}
}
The value for mac in the previous output is the MAC address of the network interface that was created when the VM was created. The comma-separated information for flowTuples, is as follows:
| Example data | What data represents | Explanation |
|---|---|---|
| 1542110377 | Time stamp | The time stamp of when the flow occurred, in UNIX EPOCH format. In the previous example, the date converts to May 1, 2018 at 2:59:05 PM GMT. |
| 10.0.0.4 | Source IP address | The source IP address that the flow originated from. 10.0.0.4 is the private IP address of the VM you created in Create a virtual machine. |
| 13.67.143.118 | Destination IP address | The destination IP address that the flow was destined to. |
| 44931 | Source port | The source port that the flow originated from. |
| 443 | Destination port | The destination port that the flow was destined to. Since the traffic was destined to port 443, the rule named UserRule_default-allow-rdp, in the log file processed the flow. |
| T | Protocol | Whether the protocol of the flow was TCP (T) or UDP (U). |
| O | Direction | Whether the traffic was inbound (I) or outbound (O). |
| A | Action | Whether the traffic was allowed (A) or denied (D). |
| C | Flow State Version 2 Only | Captures the state of the flow. Possible states are B: Begin, when a flow is created. Statistics aren't provided. C: Continuing for an ongoing flow. Statistics are provided at 5-minute intervals. E: End, when a flow is ended. Statistics are provided. |
| 30 | Packets sent - Source to destination Version 2 Only | The total number of TCP or UDP packets sent from source to destination since last update. |
| 16978 | Bytes sent - Source to destination Version 2 Only | The total number of TCP or UDP packet bytes sent from source to destination since last update. Packet bytes include the packet header and payload. |
| 24 | Packets sent - Destination to source Version 2 Only | The total number of TCP or UDP packets sent from destination to source since last update. |
| 14008 | Bytes sent - Destination to source Version 2 Only | The total number of TCP and UDP packet bytes sent from destination to source since last update. Packet bytes include packet header and payload. |
Next steps
In this tutorial, you learned how to:
- Enable NSG flow logging for an NSG
- Download and view data logged in a file.
The raw data in the json file can be difficult to interpret. To visualize Flow Logs data, you can use Azure Traffic Analytics and Microsoft Power BI.
For alternate methods of enabling NSG Flow Logs, see PowerShell, Azure CLI, REST API, and Resource Manager templates.
Advance to the next article to learn how to monitor network communication between two virtual machines: