Azure VMware Solution private cloud and cluster concepts
Azure VMware Solution delivers VMware-based private clouds in Azure. The private cloud hardware and software deployments are fully integrated and automated in Azure. You deploy and manage the private cloud through the Azure portal, CLI, or PowerShell.
A private cloud includes clusters with:
- Dedicated bare-metal server hosts provisioned with VMware ESXi hypervisor
- vCenter Server for managing ESXi and vSAN
- VMware NSX-T software-defined networking for vSphere workload VMs
- VMware vSAN datastore for vSphere workload VMs
- VMware HCX for workload mobility
- Resources in the Azure underlay (required for connectivity and to operate the private cloud)
As with other resources, private clouds are installed and managed from within an Azure subscription. The number of private clouds within a subscription is scalable. Initially, there's a limit of one private cloud per subscription. There's a logical relationship between Azure subscriptions, Azure VMware Solution private clouds, vSAN clusters, and hosts.
The diagram shows a single Azure subscription with two private clouds that represent a development and production environment. In each of those private clouds are two clusters.
Hosts
Azure VMware Solution clusters are based on hyper-converged, bare-metal infrastructure. The following table shows the RAM, CPU, and disk capacities of the host.
| Host Type | CPU | RAM (GB) | vSAN NVMe cache Tier (TB, raw) | vSAN SSD capacity tier (TB, raw) |
|---|---|---|---|---|
| AV36 | dual Intel 18 core 2.3 GHz | 576 | 3.2 | 15.20 |
Hosts used to build or scale clusters come from an isolated pool of hosts. Those hosts have passed hardware tests and have had all data securely deleted.
Clusters
For each private cloud created, there's one vSAN cluster by default. You can add, delete, and scale clusters. The minimum number of hosts per cluster and the initial deployment is three.
You use vSphere and NSX-T Manager to manage most other aspects of cluster configuration or operation. All local storage of each host in a cluster is under the control of vSAN.
Tip
You can always extend the cluster and add additional clusters later if you need to go beyond the initial deployment number.
The following table describes the maximum limits for Azure VMware Solution.
| Resource | Limit |
|---|---|
| Clusters per private cloud | 12 |
| Minimum number of hosts per cluster | 3 |
| Maximum number of hosts per cluster | 16 |
| hosts per private cloud | 96 |
| vCenter per private cloud | 1 |
| HCX site pairings | 25 (any edition) |
| Azure VMware Solution ExpressRoute max linked private clouds | 4 The virtual network gateway used determines the actual max linked private clouds. For more details, see About ExpressRoute virtual network gateways |
| Azure VMware Solution ExpressRoute port speed | 10 Gbps The virtual network gateway used determines the actual bandwidth. For more details, see About ExpressRoute virtual network gateways |
| Public IPs exposed via vWAN | 100 |
| vSAN capacity limits | 75% of total usable (keep 25% available for SLA) |
For other VMware-specific limits, use the VMware configuration maximum tool!.
VMware software versions
The VMware software versions used in new deployments of Azure VMware Solution private clouds clusters are:
| Software | Version |
|---|---|
| vCenter | 6.7 U3p |
| ESXi | 6.7 P05 |
| vSAN | 6.7 P05 |
| HCX | 4.2.2 |
| NSX-T NOTE: NSX-T is the only supported version of NSX. |
3.1.2 |
The currently running software version is applied to new clusters added to an existing private cloud. For more information, see the VMware software version requirements.
Host maintenance and lifecycle management
One benefit of Azure VMware Solution private clouds is the platform is maintained for you. Microsoft is responsible for the lifecycle management of VMware software (ESXi, vCenter, and vSAN). Microsoft is also responsible for the lifecycle management of NSX-T appliances, bootstrapping the network configuration, such as creating the Tier-0 gateway and enabling North-South routing. You're responsible for NSX-T SDN configuration: network segments, distributed firewall rules, Tier 1 gateways, and load balancers.
Microsoft is responsible for applying any patches, updates, or upgrades to ESXi, vCenter, vSAN, and NSX-T in your private cloud. The impact of patches, updates, and upgrades on ESXi, vCenter, and NSX-T is different.
ESXi - There's no impact to workloads running in your private cloud. Access to vCenter and NSX-T isn't blocked during this time. It's recommended that, during this time, you don't plan any other activities like scaling up private cloud, and so on, in your private cloud.
vCenter - There's no impact to workloads running in your private cloud. During this time, vCenter will be unavailable and you won't be able to manage VMs (stop, start, create, or delete). It's recommended that, during this time, you don't plan any other activities like scaling up private cloud, creating new networks, and so on, in your private cloud.
NSX-T - There's workload impact and when a particular host is being upgraded, the VMs on that host might lose connectivity from 2 seconds to maximum 1 minute with any and all of the following symptoms:
Ping errors
Packet loss
Error messages (for example, Destination Host Unreachable and Net unreachable)
During this upgrade window, all access to the NSX-T management plane will be blocked. You can't make configuration changes to the NSX-T environment for the duration. However, your workloads will continue to run as normal, subject to the upgrade impact detailed above.
It's recommended that, during the upgrade time, you don't plan any other activities like scaling up private cloud, and so on, in your private cloud. These can prevent the upgrade from starting or could have adverse impacts on the upgrade and the environment.
You'll be notified before patches/updates or upgrades are applied to your private clouds. We'll also work with you to schedule a maintenance window before applying updates or upgrades to your private cloud.
Software updates include:
Patches - Security patches or bug fixes released by VMware
Updates - Minor version change of a VMware stack component
Upgrades - Major version change of a VMware stack component
Note
Microsoft tests a critical security patch as soon as it becomes available from VMware.
Documented VMware workarounds are implemented in lieu of installing a corresponding patch until the next scheduled updates are deployed.
Host monitoring and remediation
Azure VMware Solution continuously monitors the health of both the underlay and the VMware components. When Azure VMware Solution detects a failure, it takes action to repair the failed components. When Azure VMware Solution detects a degradation or failure on an Azure VMware Solution node, it triggers the host remediation process.
Host remediation involves replacing the faulty node with a new healthy node in the cluster. Then, when possible, the faulty host is placed in VMware vSphere maintenance mode. VMware vMotion moves the VMs off the faulty host to other available servers in the cluster, potentially allowing zero downtime for live migration of workloads. If the faulty host can't be placed in maintenance mode, the host is removed from the cluster.
Azure VMware Solution monitors the following conditions on the host:
- Processor status
- Memory status
- Connection and power state
- Hardware fan status
- Network connectivity loss
- Hardware system board status
- Errors occurred on the disk(s) of a vSAN host
- Hardware voltage
- Hardware temperature status
- Hardware power status
- Storage status
- Connection failure
Note
Azure VMware Solution tenant admins must not edit or delete the above defined VMware vCenter alarms, as these are managed by the Azure VMware Solution control plane on vCenter. These alarms are used by Azure VMware Solution monitoring to trigger the Azure VMware Solution host remediation process.
Backup and restoration
Private cloud vCenter and NSX-T configurations are on an hourly backup schedule. Backups are kept for three days. If you need to restore from a backup, open a support request in the Azure portal to request restoration.
Azure VMware Solution continuously monitors the health of both the underlay and the VMware components. When Azure VMware Solution detects a failure, it takes action to repair the failed components.
Next steps
Now that you've covered Azure VMware Solution private cloud concepts, you may want to learn about: