Design Considerations For Scale Sets

This topic discusses design considerations for Virtual Machine Scale Sets. For information about what Virtual Machine Scale Sets are, refer to Virtual Machine Scale Sets Overview.

When to use scale sets instead of virtual machines?

Generally, scale sets are useful for deploying highly available infrastructure where a set of machines have similar configuration. However, some features are only available in scale sets while other features are only available in VMs. In order to make an informed decision about when to use each technology, we should first take a look at some of the commonly used features that are available in scale sets but not VMs:

Scale set-specific features

  • Once you specify the scale set configuration, you can simply update the "capacity" property to deploy more VMs in parallel. This is much simpler than writing a script to orchestrate deploying many individual VMs in parallel.
  • You can use Azure Autoscale to automatically scale a scale set but not individual VMs.
  • You can reimage scale set VMs but not individual VMs.
  • You can overprovision scale set VMs for increased reliability and quicker deployment times. You cannot do this with individual VMs unless you write custom code to do this.
  • You can specify an upgrade policy to make it easy to roll out upgrades across VMs in your scale set. With individual VMs, you must orchestrate updates yourself.

VM-specific features

On the other hand, some features are only available in VMs (at least for the time being):

  • You can attach data disks to specific individual VMs, but attached data disks are configured for all VMs in a scale set.
  • You can attach non-empty data disks to individual VMs but not VMs in a scale set.
  • You can snapshot an individual VM but not a VM in a scale set.
  • You can capture an image from an individual VM but not from a VM in a scale set.
  • You can migrate an individual VM from native disks to managed disks, but you cannot do this for VMs in a scale set.
  • You can assign IPv6 public IP addresses to individual VM nics but cannot do so for VMs in a scale set. Note that you can assign IPv6 public IP addresses to load balancers in front of either individual VMs or scale set VMs.

Storage

Scale sets with Azure Managed Disks

Scale sets can be created with Azure Managed Disks instead of traditional Azure storage accounts. Managed Disks provide the following benefits:

If you have an existing template, you can also update the template to use Managed Disks.

User-managed Storage

A scale set that is not defined with Azure Managed Disks relies on user-created storage accounts to store the OS disks of the VMs in the set. A ratio of 20 VMs per storage account or less is recommended to achieve maximum IO and also take advantage of overprovisioning (see below). It is also recommended that you spread the beginning characters of the storage account names across the alphabet. Doing so helps spread load across different internal systems.

Overprovisioning

Scale sets currently default to "overprovisioning" VMs. With overprovisioning turned on, the scale set actually spins up more VMs than you asked for, then deletes the extra VMs once the requested number of VMs are successfully provisioned. Overprovisioning improves provisioning success rates and reduces deployment time. You are not billed for the extra VMs, and they do not count toward your quota limits.

While overprovisioning does improve provisioning success rates, it can cause confusing behavior for an application that is not designed to handle extra VMs appearing and then disappearing. To turn overprovisioning off, ensure you have the following string in your template: "overprovision": "false". More details can be found in the Scale Set REST API documentation.

If your scale set uses user-managed storage, and you turn off overprovisioning, you can have more than 20 VMs per storage account, but it is not recommended to go above 40 for IO performance reasons.

Limits

A scale set built on a Marketplace image (also known as a platform image) and configured to use Azure Managed Disks supports a capacity of up to 1,000 VMs. If you configure your scale set to support more than 100 VMs, not all scenarios work the same (for example load balancing). For more information, see Working with large virtual machine scale sets.

A scale set configured with user-managed storage accounts is currently limited to 100 VMs (and 5 storage accounts are recommended for this scale).

A scale set built on a custom image (one built by you) can have a capacity of up to 300 VMs when configured with Azure Managed disks. If the scale set is configured with user-managed storage accounts, it must create all OS disk VHDs within one storage account. As a result, the maximum recommended number of VMs in a scale set built on a custom image and user-managed storage is 20. If you turn off overprovisioning, you can go up to 40.

For more VMs than these limits allow, you need to deploy multiple scale sets as shown in this template.