Performance guidelines for SQL Server in Azure Virtual Machines

Overview

This article provides guidance for optimizing SQL Server performance in Microsoft Azure Virtual Machine. While running SQL Server in Azure Virtual Machines, we recommend that you continue using the same database performance tuning options that are applicable to SQL Server in on-premises server environment. However, the performance of a relational database in a public cloud depends on many factors such as the size of a virtual machine, and the configuration of the data disks.

SQL Server images provisioned in the Azure portal follow general storage configuration best practices (for more information on how storage is configured, see Storage configuration for SQL Server VMs). After provisioning, consider applying other optimizations discussed in this article. Base your choices on your workload and verify through testing.

Tip

There is typically a trade-off between optimizing for costs and optimizing for performance. This article is focused on getting the best performance for SQL Server on Azure VMs. If your workload is less demanding, you might not require every optimization listed below. Consider your performance needs, costs, and workload patterns as you evaluate these recommendations.

Quick check list

The following is a quick check list for optimal performance of SQL Server on Azure Virtual Machines:

Area Optimizations
VM size - Use VM sizes with 4 or more vCPU like E4S_v3 or higher, or DS12_v2 or higher.

- Es, Eas, Ds and Das Series offers the optimum memory to vCPU ratio required for OLTP workload performance.

- M Series offers the highest memory to vCPU ratio required for mission critical performance and is ideal for data warehouse workloads.

- Collect the target workload's IOPS, throughput and latency requirements at peak times by following the application performance requirements checklist and then select the VM Size that can scale to your workload's performance requirements.
Storage - For detailed testing of SQL Server performance on Azure VMs with TPC-E and TPC_C benchmarks, refer to the blog Optimize OLTP performance.

- Use premium SSDs for the best price/performance advantages. Configure ReadOnly cache for data files and no cache for the log file.

- Use Ultra Disks if less than 1 ms storage latencies are required by the workload.

- Collect the storage latency requirements for SQL Server data, log, and Temp DB files by monitoring the application before choosing the disk type. If <1ms storage latencies are required, then use Ultra Disks, otherwise use premium SSD. If low latencies are only required for the log file and not for data files, then provision the Ultra Disk at required IOPS and throughput levels only for the log File.

- Premium file shares are recommended as shared storage for a SQL Server failover cluster Instance. Premium file shares do not support caching, and offer limited performance compared to premium SSD disks. Choose premium SSD-managed disks over premium file shares for standalone SQL instances; but leverage premium file shares for failover cluster instance shared storage for ease of maintenance and flexible scalability.

- Standard storage is only recommended for development and test purposes or for backup files and should not be used for production workloads.

- Keep the storage account and SQL Server VM in the same region.

- Disable Azure geo-redundant storage (geo-replication) on the storage account.
Disks - Use a minimum of 2 premium SSD disks (1 for log file and 1 for data files).

- For workloads requiring <1 ms IO latencies, enable write accelerator for M series and consider using Ultra SSD disks for Es and DS series.

- Enable read only caching on the disk(s) hosting the data files.

- Add additional 20% premium IOPS/throughput capacity than your workload requires when configuring storage for SQL Server data, log, and TempDB files

- Avoid using operating system or temporary disks for database storage or logging.

- Do not enable caching on disk(s) hosting the log file. Important: Stop the SQL Server service when changing the cache settings for an Azure VM disk.

- Stripe multiple Azure data disks to get increased storage throughput.

- Format with documented allocation sizes.

- Place TempDB on the local SSD D:\ drive for mission critical SQL Server workloads (after choosing correct VM size). If you create the VM from Azure portal or Azure quickstart templates and place Temp DB on the Local Disk then you do not need any further action; for all other cases follow the steps in the blog for Using SSDs to store TempDB to prevent failures after restarts. If the capacity of the local drive is not enough for your Temp DB size, then place Temp DB on a storage pool stripped on premium SSD disks with read-only caching.
I/O - Enable database page compression.

- Enable instant file initialization for data files.

- Limit autogrowth of the database.

- Disable autoshrink of the database.

- Move all databases to data disks, including system databases.

- Move SQL Server error log and trace file directories to data disks.

- Configure default backup and database file locations.

- Enable locked pages in memory.

- Apply SQL Server performance fixes.
Feature-specific - Back up directly to blob storage.

- Use file snapshot backups for databases larger than 12 TB.

- Use multiple Temp DB files, 1 file per core, up to 8 files.

- Set max server memory at 90% or up to 50 GB left for the Operating System.

- Enable soft NUMA.

For more information on how and why to make these optimizations, please review the details and guidance provided in the following sections.

VM size guidance

Start by collecting the cpu, memory, and storage throughput requirements of the workload at peak times. \LogicalDisk\Disk Reads/Sec and \LogicalDisk\Disk Writes/Sec performance counters can be used to collect read and write IOPS requirements and \LogicalDisk\Disk Bytes/Sec counter can be used to collect storage throughput requirements for Data, Log, and Temp DB files. After IOPS and throughput requirements at peak are defined then evaluate VM sizes offers that capacity. For example if your workload requires 20 K read IOPS and 10K write IOPS at peak, you can either choose E16s_v3 (with up to 32 K cached and 25600 uncached IOPS) or M16_s (with up to 20 K cached and 10K uncached IOPS) with 2 P30 disks. Make sure to understand both throughput and IOPS requirements of the workload as VMs has different scale limits for IOPS and throughput.

DSv_3 and Es_v3-series are hosted on general purpose hardware with Intel Haswell or Broadwell processors. M-series offers the highest vCPU count and memory for the largest SQL Server workloads and hosted on memory optimized hardware with Skylake processor family. These VM series support premium storage, which is recommended for the best performance with host level read cache. Both Es_v3 and M series are also available in constrained core sizes, which saves money for workloads with lower compute and high storage capacity demands.

Storage guidance

For detailed testing of SQL Server performance on Azure VMs with TPC-E and TPC_C benchmarks, refer to the blog Optimize OLTP performance.

Azure blob cache with premium SSDs is recommended for all production workloads.

Warning

Standard HDDs and SSDs have varying latencies and bandwidth and are only recommended for dev/test workloads. Production workloads should use premium SSDs.

In addition, we recommend that you create your Azure storage account in the same data center as your SQL Server virtual machines to reduce transfer delays. When creating a storage account, disable geo-replication as consistent write order across multiple disks is not guaranteed. Instead, consider configuring a SQL Server disaster recovery technology between two Azure data centers. For more information, see High Availability and Disaster Recovery for SQL Server in Azure Virtual Machines.

Disks guidance

There are three main disk types on an Azure VM:

  • OS disk: When you create an Azure Virtual Machine, the platform will attach at least one disk (labeled as the C drive) to the VM for your operating system disk. This disk is a VHD stored as a page blob in storage.
  • Temporary disk: Azure virtual machines contain another disk called the temporary disk (labeled as the D: drive). This is a disk on the node that can be used for scratch space.
  • Data disks: You can also attach additional disks to your virtual machine as data disks, and these will be stored in storage as page blobs.

The following sections describe recommendations for using these different disks.

Operating system disk

An operating system disk is a VHD that you can boot and mount as a running version of an operating system and is labeled as C drive.

Default caching policy on the operating system disk is Read/Write. For performance sensitive applications, we recommend that you use data disks instead of the operating system disk. See the section on Data Disks below.

Temporary disk

The temporary storage drive, labeled as the D drive, is not persisted to Azure blob storage. Do not store your user database files or user transaction log files on the D: drive.

Place TempDB on the local SSD D:\ drive for mission critical SQL Server workloads (after choosing correct VM size). If you create the VM from Azure portal or Azure quickstart templates and place Temp DB on the Local Disk, then you do not need any further action; for all other cases follow the steps in the blog for Using SSDs to store TempDB to prevent failures after restarts. If the capacity of the local drive is not enough for your Temp DB size, then place Temp DB on a storage pool stripped on premium SSD disks with read-only caching.

For VMs that support premium SSDs, you can also store TempDB on a disk that supports premium SSDs with read caching enabled.

Data disks

  • Use premium SSD disks for data and log files: If you are not using disk striping, use two premium SSD disks where one disk contains the log file and the other contains the data. Each premium SSD provides a number of IOPS and bandwidth (MB/s) depending on its size, as depicted in the article, Select a disk type. If you are using a disk striping technique, such as Storage Spaces, you achieve optimal performance by having two pools, one for the log file(s) and the other for the data files. However, if you plan to use SQL Server failover cluster instances (FCI), you must configure one pool, or utilize premium file shares instead.

    Tip

    Note

    When you provision a SQL Server VM in the portal, you have the option of editing your storage configuration. Depending on your configuration, Azure configures one or more disks. Multiple disks are combined into a single storage pool with striping. Both the data and log files reside together in this configuration. For more information, see Storage configuration for SQL Server VMs.

  • Disk Striping: For more throughput, you can add additional data disks and use Disk Striping. To determine the number of data disks, you need to analyze the number of IOPS and bandwidth required for your log file(s), and for your data and TempDB file(s). Notice that different VM sizes have different limits on the number of IOPs and bandwidth supported, see the tables on IOPS per VM size. Use the following guidelines:

    • For Windows 8/Windows Server 2012 or later, use Storage Spaces with the following guidelines:

      1. Set the interleave (stripe size) to 64 KB (65,536 bytes) for OLTP workloads and 256 KB (262,144 bytes) for data warehousing workloads to avoid performance impact due to partition misalignment. This must be set with PowerShell.
      2. Set column count = number of physical disks. Use PowerShell when configuring more than 8 disks (not Server Manager UI).

      For example, the following PowerShell creates a new storage pool with the interleave size to 64 KB and the number of columns to 2:

      $PoolCount = Get-PhysicalDisk -CanPool $True
      $PhysicalDisks = Get-PhysicalDisk | Where-Object {$_.FriendlyName -like "*2" -or $_.FriendlyName -like "*3"}
      
      New-StoragePool -FriendlyName "DataFiles" -StorageSubsystemFriendlyName "Storage Spaces*" -PhysicalDisks $PhysicalDisks | New-VirtualDisk -FriendlyName "DataFiles" -Interleave 65536 -NumberOfColumns 2 -ResiliencySettingName simple –UseMaximumSize |Initialize-Disk -PartitionStyle GPT -PassThru |New-Partition -AssignDriveLetter -UseMaximumSize |Format-Volume -FileSystem NTFS -NewFileSystemLabel "DataDisks" -AllocationUnitSize 65536 -Confirm:$false 
      
    • For Windows 2008 R2 or earlier, you can use dynamic disks (OS striped volumes) and the stripe size is always 64 KB. This option is deprecated as of Windows 8/Windows Server 2012. For information, see the support statement at Virtual Disk Service is transitioning to Windows Storage Management API.

    • If you are using Storage Spaces Direct (S2D) with SQL Server Failover Cluster Instances, you must configure a single pool. Although different volumes can be created on that single pool, they will all share the same characteristics, such as the same caching policy.

    • Determine the number of disks associated with your storage pool based on your load expectations. Keep in mind that different VM sizes allow different numbers of attached data disks. For more information, see Sizes for Virtual Machines.

    • If you are not using premium SSDs (dev/test scenarios), the recommendation is to add the maximum number of data disks supported by your VM size and use Disk Striping.

  • Caching policy: Note the following recommendations for caching policy depending on your storage configuration.

    • If you are using separate disks for data and log files, enable read caching on the data disks hosting your data files and TempDB data files. This can result in a significant performance benefit. Do not enable caching on the disk holding the log file as this causes a minor decrease in performance.

    • If you are using disk striping in a single storage pool, most workloads will benefit from read caching. If you have separate storage pools for the log and data files, enable read caching only on the storage pool for the data files. In certain heavy write workloads, better performance might be achieved with no caching. This can only be determined through testing.

    • The previous recommendations apply to premium SSDs. If you are not using premium SSDs, do not enable any caching on any data disks.

    • For instructions on configuring disk caching, see the following articles. For the classic (ASM) deployment model see: Set-AzureOSDisk and Set-AzureDataDisk. For the Azure Resource Manager deployment model, see: Set-AzOSDisk and Set-AzVMDataDisk.

      Warning

      Stop the SQL Server service when changing the cache setting of Azure VM disks to avoid the possibility of any database corruption.

  • NTFS allocation unit size: When formatting the data disk, it is recommended that you use a 64-KB allocation unit size for data and log files as well as TempDB.

  • Disk management best practices: When removing a data disk or changing its cache type, stop the SQL Server service during the change. When the caching settings are changed on the OS disk, Azure stops the VM, changes the cache type, and restarts the VM. When the cache settings of a data disk are changed, the VM is not stopped, but the data disk is detached from the VM during the change and then reattached.

    Warning

    Failure to stop the SQL Server service during these operations can cause database corruption.

I/O guidance

  • The best results with premium SSDs are achieved when you parallelize your application and requests. Premium SSDs are designed for scenarios where the IO queue depth is greater than 1, so you will see little or no performance gains for single-threaded serial requests (even if they are storage intensive). For example, this could impact the single-threaded test results of performance analysis tools, such as SQLIO.

  • Consider using database page compression as it can help improve performance of I/O intensive workloads. However, the data compression might increase the CPU consumption on the database server.

  • Consider enabling instant file initialization to reduce the time that is required for initial file allocation. To take advantage of instant file initialization, you grant the SQL Server (MSSQLSERVER) service account with SE_MANAGE_VOLUME_NAME and add it to the Perform Volume Maintenance Tasks security policy. If you are using a SQL Server platform image for Azure, the default service account (NT Service\MSSQLSERVER) isn’t added to the Perform Volume Maintenance Tasks security policy. In other words, instant file initialization is not enabled in a SQL Server Azure platform image. After adding the SQL Server service account to the Perform Volume Maintenance Tasks security policy, restart the SQL Server service. There could be security considerations for using this feature. For more information, see Database File Initialization.

  • autogrow is considered to be merely a contingency for unexpected growth. Do not manage your data and log growth on a day-to-day basis with autogrow. If autogrow is used, pre-grow the file using the Size switch.

  • Make sure autoshrink is disabled to avoid unnecessary overhead that can negatively affect performance.

  • Move all databases to data disks, including system databases. For more information, see Move System Databases.

  • Move SQL Server error log and trace file directories to data disks. This can be done in SQL Server Configuration Manager by right-clicking your SQL Server instance and selecting properties. The error log and trace file settings can be changed in the Startup Parameters tab. The Dump Directory is specified in the Advanced tab. The following screenshot shows where to look for the error log startup parameter.

    SQL ErrorLog Screenshot

  • Setup default backup and database file locations. Use the recommendations in this article, and make the changes in the Server properties window. For instructions, see View or Change the Default Locations for Data and Log Files (SQL Server Management Studio). The following screenshot demonstrates where to make these changes.

    SQL Data Log and Backup files

  • Enable locked pages to reduce IO and any paging activities. For more information, see Enable the Lock Pages in Memory Option (Windows).

  • If you are running SQL Server 2012, install Service Pack 1 Cumulative Update 10. This update contains the fix for poor performance on I/O when you execute select into temporary table statement in SQL Server 2012. For information, see this knowledge base article.

  • Consider compressing any data files when transferring in/out of Azure.

Feature-specific guidance

Some deployments may achieve additional performance benefits using more advanced configuration techniques. The following list highlights some SQL Server features that can help you to achieve better performance:

Back up to Azure Storage

When performing backups for SQL Server running in Azure virtual machines, you can use SQL Server Backup to URL. This feature is available starting with SQL Server 2012 SP1 CU2 and recommended for backing up to the attached data disks. When you backup/restore to/from Azure storage, follow the recommendations provided at SQL Server Backup to URL Best Practices and Troubleshooting and Restoring from Backups Stored in Azure Storage. You can also automate these backups using Automated Backup for SQL Server in Azure Virtual Machines.

Prior to SQL Server 2012, you can use SQL Server Backup to Azure Tool. This tool can help to increase backup throughput using multiple backup stripe targets.

SQL Server Data files in Azure

This new feature, SQL Server Data Files in Azure, is available starting with SQL Server 2014. Running SQL Server with data files in Azure demonstrates comparable performance characteristics as using Azure data disks.

Failover cluster instance and Storage Spaces

If you are using Storage Spaces, when adding nodes to the cluster on the Confirmation page, clear the checkbox labeled Add all eligible storage to the cluster.

Uncheck eligible storage

If you are using Storage Spaces and do not uncheck Add all eligible storage to the cluster, Windows detaches the virtual disks during the clustering process. As a result, they do not appear in Disk Manager or Explorer until the storage spaces are removed from the cluster and reattached using PowerShell. Storage Spaces groups multiple disks in to storage pools. For more information, see Storage Spaces.

Next steps

For more information about storage and performance, see Storage Configuration Guidelines for SQL Server on Azure VM

For security best practices, see Security Considerations for SQL Server in Azure Virtual Machines.

Review other SQL Server Virtual Machine articles at SQL Server on Azure Virtual Machines Overview. If you have questions about SQL Server virtual machines, see the Frequently Asked Questions.