Azure NetApp Files large volume performance benchmarks for Linux
This article describes the tested performance capabilities of a single Azure NetApp Files large volumes as it pertains to Linux use cases. The tests explored scenarios for both scale-out and scale-up read and write workloads, involving one and many virtual machines (VMs). Knowing the performance envelope of large volumes helps you facilitate volume sizing.
Testing summary
The Azure NetApp Files large volumes feature offers three service levels, each with throughput limits. The service levels can be scaled up or down nondisruptively as your performance needs change.
- Ultra service level: 10,240 MiB/s
- Premium service level: 6,400 MiB/s
- Standard service level: 1,600 MiB/s
The Ultra service level was used in these tests.
Sequential I/O: 100% sequential writes max out at 8,500 MiB/second, while a single large volume is capable of 10 GiB/second (10,240 MiB/second) throughput.
Random I/O: The same single large volume delivers over 700,000 operations per second.
Metadata-heavy workloads are advantageous for Azure NetApp File large volumes due to the large volume’s increased parallelism. Performance benefits are noticeable in workloads heavy in file creation, unlink, and file renames as typical with VCS applications, and EDA workloads where there are high file counts present. For more information on performance of high metadata workloads, see Benefits of using Azure NetApp Files for electronic design automation.
FIO, a synthetic workload generator designed as a storage stress test, was used to drive these test results. There are fundamentally two models of storage performance testing:
- Scale-out compute, which refers to using multiple VMs to generate the maximum load possible on a single Azure NetApp Files volume.
- Scale-up compute, which refers to using a large VM to test the upper boundaries of a single client on a single Azure NetApp Files volume.
Linux scale-out test
Tests observed performance thresholds of a single large volume on scale-out and were conducted with the following configuration:
| Component | Configuration |
|---|---|
| Azure VM size | E32s_v5 |
| Azure VM egress bandwidth limit | 2000MiB/s (2GiB/s) |
| Operating system | RHEL 8.4 |
| Large volume size | 101 TiB Ultra (10,240 MiB/s throughput) |
| Mount options | hard,rsize=65536,wsize=65536,vers=3 NOTE: Use of both 262144 and 65536 had similar performance results. |
256-KiB sequential workloads (MiB/s)
The graph represents a 256 KiB sequential workload and a 1 TiB working set. It shows that a single Azure NetApp Files large volume can handle between approximately 8,518 MiB/s pure sequential writes and 9,970 MiB/s pure sequential reads.
8-KiB random workload (IOPS)
The graph represents an 8-KiB random workload and a 1 TiB working set. The graph shows that an Azure NetApp Files large volume can handle between approximately 474,000 pure random writes and approximately 709,000 pure random reads.
Linux scale-up tests
Whereas scale-out tests are designed to find the limits of a single large volume, scale-up tests are designed to find the upper limits of a single instance against said large volume. Azure places network egress limits on its VMs; for network attached storage that means that the write bandwidth is capped per VM. These scale-up tests demonstrate capabilities given the large available bandwidth cap and with sufficient processors to drive said workload.
The tests in this section were run with the following configuration:
| Component | Configuration |
|---|---|
| Azure VM size | E104id_v5 |
| Azure VM egress bandwidth limit | 12,500MiB/s (12.2GiB/s) |
| Operating system | RHEL 8.4 |
| Large volume size | 101 TiB Ultra (10,240 MiB/s throughput) |
| Mount options | hard,rsize=65536,wsize=65536,vers=3 NOTE: Use of both 262144 and 65536 had similar performance results |
The graphs in this section show the results for the client-side mount option of nconnect with NFSv3. For more information, see Linux NFS mount options best practices for Azure NetApp File.
The following graphs compare the advantages of nconnect with an NFS-mounted volume without nconnect. In the tests, FIO generated the workload from a single E104id-v5 instance in the East US Azure region using a 64-KiB sequential workload; a 256 I/0 size was used, which is the largest I/O size recommended by Azure NetApp Files resulted in comparable performance numbers. For more information, see rsize and wsize.
Linux read throughput
The following graphs show 256-KiB sequential reads of ~10,000MiB/s with nconnect, which is roughly ten times the throughput achieved without nconnect.
Note that 10,000 MiB/s bandwidth is offered by a large volume in the Ultra service level.
Linux write throughput
The following graphs show sequential writes. Using nconnect provides observable benefits for sequential writes at 6,600 MiB/s, roughly four times that of mounts without nconnect.
Linux read IOPS
The following graphs show 8-KiB random reads of ~426,000 read IOPS with nconnect, roughly seven times what is observed without nconnect.
Linux write IOPS
The following graphs show 8-KiB random writes of ~405,000 write IOPS with nconnect, roughly 7.2 times that what is observed without nconnect.
Next steps
Feedback
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
Submit and view feedback for