Implement a hub-spoke network topology in Azure

This reference architecture shows how to implement a hub-spoke topology in Azure. The hub is a virtual network (VNet) in Azure that acts as a central point of connectivity to your on-premises network. The spokes are VNets that peer with the hub, and can be used to isolate workloads. Traffic flows between the on-premises datacenter and the hub through an ExpressRoute or VPN gateway connection. Deploy this solution.

[0]

Download a Visio file of this architecture

The benefits of this toplogy include:

  • Cost savings by centralizing services that can be shared by multiple workloads, such as network virtual appliances (NVAs) and DNS servers, in a single location.
  • Overcome subscriptions limits by peering VNets from different subscriptions to the central hub.
  • Separation of concerns between central IT (SecOps, InfraOps) and workloads (DevOps).

Typical uses for this architecture include:

  • Workloads deployed in different environments, such as development, testing, and production, that require shared services such as DNS, IDS, NTP, or AD DS. Shared services are placed in the hub VNet, while each environment is deployed to a spoke to maintain isolation.
  • Workloads that do not require connectivity to each other, but require access to shared services.
  • Enterprises that require central control over security aspects, such as a firewall in the hub as a DMZ, and segregated management for the workloads in each spoke.

Architecture

The architecture consists of the following components.

  • On-premises network. A private local-area network running within an organization.

  • VPN device. A device or service that provides external connectivity to the on-premises network. The VPN device may be a hardware device, or a software solution such as the Routing and Remote Access Service (RRAS) in Windows Server 2012. For a list of supported VPN appliances and information on configuring selected VPN appliances for connecting to Azure, see About VPN devices for Site-to-Site VPN Gateway connections.

  • VPN virtual network gateway or ExpressRoute gateway. The virtual network gateway enables the VNet to connect to the VPN device, or ExpressRoute circuit, used for connectivity with your on-premises network. For more information, see Connect an on-premises network to a Microsoft Azure virtual network.

Note

The deployment scripts for this reference architecture use a VPN gateway for connectivity, and a VNet in Azure to simulate your on-premises network.

  • Hub VNet. Azure VNet used as the hub in the hub-spoke topology. The hub is the central point of connectivity to your on-premises network, and a place to host services that can be consumed by the different workloads hosted in the spoke VNets.

  • Gateway subnet. The virtual network gateways are held in the same subnet.

  • Shared services subnet. A subnet in the hub VNet used to host services that can be shared among all spokes, such as DNS or AD DS.

  • Spoke VNets. One or more Azure VNets that are used as spokes in the hub-spoke topology. Spokes can be used to isolate workloads in their own VNets, managed separately from other spokes. Each workload might include multiple tiers, with multiple subnets connected through Azure load balancers. For more information about the application infrastructure, see Running Windows VM workloads and Running Linux VM workloads.

  • VNet peering. Two VNets in the same Azure region can be connected using a peering connection. Peering connections are non-transitive, low latency connections between VNets. Once peered, the VNets exchange traffic by using the Azure backbone, without the need for a router. In a hub-spoke network topology, you use VNet peering to connect the hub to each spoke.

Note

This article only covers Resource Manager deployments, but you can also connect a classic VNet to a Resource Manager VNet in the same subscription. That way, your spokes can host classic deployments and still benefit from services shared in the hub.

Recommendations

The following recommendations apply for most scenarios. Follow these recommendations unless you have a specific requirement that overrides them.

Resource groups

The hub VNet, and each spoke VNet, can be implemented in different resource groups, and even different subscriptions, as long as they belong to the same Azure Active Directory (Azure AD) tenant in the same Azure region. This allows for a decentralized management of each workload, while sharing services maintained in the hub VNet.

VNet and GatewaySubnet

Create a subnet named GatewaySubnet, with an address range of /27. This subnet is required by the virtual network gateway. Allocating 32 addresses to this subnet will help to prevent reaching gateway size limitations in the future.

For more information about setting up the gateway, see the following reference architectures, depending on your connection type:

For higher availability, you can use ExpressRoute plus a VPN for failover. See Connect an on-premises network to Azure using ExpressRoute with VPN failover.

A hub-spoke topology can also be used without a gateway, if you don't need connectivity with your on-premises network.

VNet peering

VNet peering is a non-transitive relationship between two VNets. If you require spokes to connect to each other, consider adding a separate peering connection between those spokes.

However, if you have several spokes that need to connect with each other, you will run out of possible peering connections very quickly due to the limitation on number of VNets peerings per VNet. In this scenario, consider using user defined routes (UDRs) to force traffic destined to a spoke to be sent to an NVA acting as a router at the hub VNet. This will allow the spokes to connect to each other.

You can also configure spokes to use the hub VNet gateway to communicate with remote networks. To allow gateway traffic to flow from spoke to hub, and connect to remote networks, you must:

  • Configure the VNet peering connection in the hub to allow gateway transit.
  • Configure the VNet peering connection in each spoke to use remote gateways.
  • Configure all VNet peering connections to allow forwarded traffic.

Considerations

Spoke connectivity

If you require connectivity between spokes, consider implementing an NVA for routing in the hub, and using UDRs in the spoke to forward traffic to the hub.

[2]

In this scenario, you must configure the peering connections to allow forwarded traffic.

Overcoming VNet peering limits

Make sure you consider the limitation on number of VNets peerings per VNet in Azure. If you decide you need more spokes than the limit will allow, consider creating a hub-spoke-hub-spoke topology, where the first level of spokes also act as hubs. The following diagram shows this approach.

[3]

Also consider what services are shared in the hub, to ensure the hub scales for a larger number of spokes. For instance, if your hub provides firewall services, consider the bandwidth limits of your firewall solution when adding multiple spokes. You might want to move some of these shared services to a second level of hubs.

Deploy the solution

A deployment for this architecture is available on GitHub. It uses Ubuntu VMs in each VNet to test connectivity. There are no actual services hosted in the shared-services subnet in the hub VNet.

Prerequisites

Before you can deploy the reference architecture to your own subscription, you must perform the following steps.

  1. Clone, fork, or download the zip file for the AzureCAT reference architectures GitHub repository.

  2. If you prefer to use the Azure CLI, make sure you have the Azure CLI 2.0 installed on your computer. To install the CLI, follow the instructions in Install Azure CLI 2.0.

  3. If you prefer to use PowerShell, make sure you have the latest PowerShell module for Azure installed on you computer. To install the latest Azure PowerShell module, follow the instructions in Install PowerShell for Azure.

  4. From a command prompt, bash prompt, or PowerShell prompt, login to your Azure account by using one of the commands below, and follow the prompts.

    az login
    
    Login-AzureRmAccount
    

Deploy the simulated on-premises datacenter

To deploy the simulated on-premises datacenter as an Azure VNet, perform the following steps.

  1. Navigate to the hybrid-networking\hub-spoke\onprem folder for the repository you downloaded in the pre-requisites step above.

  2. Open the onprem.vm.parameters.json file and enter a username and password between the quotes in line 11 and 12, as shown below, then save the file.

    "adminUsername": "XXX",
    "adminPassword": "YYY",
    
  3. Run the bash or PowerShell command below to deploy the simulated on-premises environment as a VNet in Azure. Substitute the values with your subscription, resource group name, and Azure region.

    sh ./onprem.deploy.sh --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
     --resourcegroup ra-onprem-rg \
     --location westus
    
    ./onprem.deploy.ps1 -Subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx `
     -ResourceGroup ra-onprem-rg `
     -Location westus
    

    Note

    If you decide to use a different resource group name (other than ra-onprem-rg), make sure to search for all parameter files that use that name and edit them to use your own resource group name.

  4. Wait for the deployment to finish. This deployment creates a virtual network, a virtual machine running Ubuntu, and a VPN gateway. The VPN gateway creation can take more than 40 minutes to complete.

Azure hub VNet

To deploy the hub VNet, and connect to the simulated on-premises VNet created above, perform the following steps.

  1. Navigate to the hybrid-networking\hub-spoke\hub folder for the repository you downloaded in the pre-requisites step above.

  2. Open the hub.vm.parameters.json file and enter a username and password between the quotes in line 11 and 12, as shown below, then save the file.

    "adminUsername": "XXX",
    "adminPassword": "YYY",
    
  3. Open the hub.gateway.parameters.json file and enter a shared key between the quotes in line 23, as shown below, then save the file. Keep a note of this value, you will need to use it later in the deployment.

    "sharedKey": "",
    
  4. Run the bash or PowerShell command below to deploy the simulated on-premises environment as a VNet in Azure. Substitute the values with your subscription, resource group name, and Azure region.

    sh ./hub.deploy.sh --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
     --resourcegroup ra-hub-rg \
     --location westus
    
    ./hub.deploy.ps1 -Subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx `
     -ResourceGroup ra-hub-rg `
     -Location westus
    

    Note

    If you decide to use a different resource group name (other than ra-hub-rg), make sure to search for all parameter files that use that name and edit them to use your own resource group name.

  5. Wait for the deployment to finish. This deployment creates a virtual network, a virtual machine running Ubuntu, a VPN gateway, and a connection to the gateway created in the previous section. The VPN gateway creation can take more than 40 minutes to complete.

Connection from on-premises to the hub

To connect from the simulated on-premises datacenter to the hub VNet, perform the following steps.

  1. Navigate to the hybrid-networking\hub-spoke\onprem folder for the repository you downloaded in the pre-requisites step above.

  2. Open the onprem.connection.parameters.json file and enter a shared key between the quotes in line 9, as shown below, then save the file. This shared key value must be the same used in the on-premises gateway you deployed previously.

    "sharedKey": "",
    
  3. Run the bash or PowerShell command below to deploy the simulated on-premises environment as a VNet in Azure. Substitute the values with your subscription, resource group name, and Azure region.

    sh ./onprem.connection.deploy.sh --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
     --resourcegroup ra-onprem-rg \
     --location westus
    
    ./onprem.connection.deploy.ps1 -Subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx `
     -ResourceGroup ra-onprem-rg `
     -Location westus
    

    Note

    If you decide to use a different resource group name (other than ra-onprem-rg), make sure to search for all parameter files that use that name and edit them to use your own resource group name.

  4. Wait for the deployment to finish. This deployment creates a connection between the VNet used to simulate an on-premises datacenter, and the hub VNet.

Azure spoke VNets

To deploy the spoke VNets, and connect to the hub VNet created above, perform the following steps.

  1. Switch to the hybrid-networking\hub-spoke\spokes folder for the repository you download in the pre-requisites step above.

  2. Open the spoke1.web.parameters.json file and enter a username and password between the quotes in line 53 and 54, as shown below, then save the file.

    "adminUsername": "XXX",
    "adminPassword": "YYY",
    
  3. Repeat the previous step for file spoke2.web.parameters.json.

  4. Run the bash or PowerShell command below to deploy the first spoke and connect it to the hub. Substitute the values with your subscription, resource group name, and Azure region.

    sh ./spoke.deploy.sh --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
     --resourcegroup ra-spoke1-rg \
     --location westus \
     --spoke 1
    
    ./spoke.deploy.ps1 -Subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx `
     -ResourceGroup ra-spoke1-rg `
     -Location westus `
     -Spoke 1
    

    Note

    If you decide to use a different resource group name (other than ra-spoke1-rg), make sure to search for all parameter files that use that name and edit them to use your own resource group name.

  5. Wait for the deployment to finish. This deployment creates a virtual network, a load balancer with three virtual machine running Ubuntu and Apache, and a VNet peering connection to the hub VNet created in the previous section. This deployment may take over 20 minutes.

  6. Run the bash or PowerShell command below to deploy the first spoke and connect it to the hub. Substitute the values with your subscription, resource group name, and Azure region.

    sh ./spoke.deploy.sh --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
     --resourcegroup ra-spoke2-rg \
     --location westus \
     --spoke 2
    
    ./spoke.deploy.ps1 -Subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx `
     -ResourceGroup ra-spoke2-rg `
     -Location westus `
     -Spoke 2
    

    Note

    If you decide to use a different resource group name (other than ra-spoke2-rg), make sure to search for all parameter files that use that name and edit them to use your own resource group name.

  7. Wait for the deployment to finish. This deployment creates a virtual network, a load balancer with three virtual machine running Ubuntu and Apache, and a VNet peering connection to the hub VNet created in the previous section. This deployment may take over 20 minutes.

Azure hub VNet peering to spoke VNets

To deploy the VNet peering connections for the hub VNet, perform the following steps.

  1. Switch to the hybrid-networking\hub-spoke\hub folder for the repository you download in the pre-requisites step above.

  2. Run the bash or PowerShell command below to deploy the peering connection to the first spoke. Substitute the values with your subscription, resource group name, and Azure region.

    sh ./hub.peering.deploy.sh --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
     --resourcegroup ra-hub-rg \
     --location westus \
     --spoke 1
    
    ./hub.peering.deploy.ps1 -Subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx `
     -ResourceGroup ra-hub-rg `
     -Location westus `
     -Spoke 1
    
  3. Run the bash or PowerShell command below to deploy the peering connection to the second spoke. Substitute the values with your subscription, resource group name, and Azure region.

    sh ./hub.peering.deploy.sh --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
     --resourcegroup ra-hub-rg \
     --location westus \
     --spoke 2
    
    ./hub.peering.deploy.ps1 -Subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx `
     -ResourceGroup ra-hub-rg `
     -Location westus `
     -Spoke 2
    

Test connectivity

To verify that the hub-spoke topology connected to an on-premises datacenter deployment worked, follow these steps.

  1. From the [Azure portal][portal], connect to your subscription, and navigate to the ra-onprem-vm1 virtual machine in the ra-onprem-rg resource group.

  2. In the Overview blade, note the Public IP address for the VM.

  3. Use an SSH client to connect to the IP address you noted above using the user name and password you specified during deployment.

  4. From the command prompt on the VM you connected to, run the command below to test connectivity from the on-premises VNet to the Spoke1 VNet.

    ping 10.1.1.37
    

Add connectivity between spokes

If you want to allow spokes to connect to each other, you must deploy UDRs to each spoke that forward traffic destined to other spokes to the gateway in the hub VNet. Perform the following steps to verify that currently you are not able to connect from a spoke to another, then deploy the UDRs and test connectivity again.

  1. Repeat steps 1 to 4 above, if you are not connected to the jumpbox VM any longer.

  2. Connect to one of the web servers in spoke 1.

    ssh 10.1.1.37
    
  3. Test the connectivity between spoke 1 and spoke 2. It should fail.

    ping 10.1.2.37
    
  4. Switch back to your computer's command prompt.

  5. Switch to the hybrid-networking\hub-spoke\spokes folder for the repository you downloaded in the pre-requisites step above.

  6. Run the bash or PowerShell command below to deploy an UDR to the first spoke. Substitute the values with your subscription, resource group name, and Azure region.

    sh ./spoke.udr.deploy.sh --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
     --resourcegroup ra-spoke1-rg \
     --location westus \
     --spoke 1
    
    ./spoke.udr.deploy.ps1 -Subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx `
     -ResourceGroup ra-spoke1-rg `
     -Location westus `
     -Spoke 1
    
  7. Run the bash or PowerShell command below to deploy an UDR to the second spoke. Substitute the values with your subscription, resource group name, and Azure region.

    sh ./spoke.udr.deploy.sh --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
     --resourcegroup ra-spoke2-rg \
     --location westus \
     --spoke 2
    
    ./spoke.udr.deploy.ps1 -Subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx `
     -ResourceGroup ra-spoke2-rg `
     -Location westus `
     -Spoke 2
    
  8. Switch back to the ssh terminal.

  9. Test the connectivity between spoke 1 and spoke 2. It should succeed.

    ping 10.1.2.37