question

TonyGedge-0532 avatar image
0 Votes"
TonyGedge-0532 asked ·

VMM: Implementation of Maximum IOPs

in VMM 2016, disks can be configured with a Maximum IOPs option:
18594-g1.png

However, I don't know how this IOPs figure is calculated or enforced on the VM. My own tests show that it doesn't appear to match the IOPs reported by my SAN. What I can't tell is whether this is an artifact of the underlying storage layers or something else. The storage for the VM is a VHDX located on a Cluster Shared Volume on an iSCSI-connected LUN. The VM is definitely being rate-limited, but the SAN reports nothing like the IOPs figure configured for the VM.

I want to understand how this works, so that I can appropriately configure IOPs limits to avoid swamping the SAN storage backend and better share the available throughput between the competing VMs.

So, what's the relationship between:

  • the Maximum IOPs figure in VMM,

  • the IOPs reported by the guest, and

  • the IOPs reported by the underlying storage




msc-virtual-machine-manager-hyper-vmsc-virtual-machine-manager-general
g1.png (28.6 KiB)
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

TonyGedge-0532 avatar image
0 Votes"
TonyGedge-0532 answered ·

Thanks for the detailed response Andy, I now understand the context of your answer.

However, I still don't understand how VMM limits the IOPs used by the client as my observed figures are orders of magnitude different from the configured value.

I'm assuming it's actually implemented as some Hyper-V function under the covers. My storage subsystem is:

iSCSI LUN <-> Failover cluster CSV <-> VHD <-> VM

I have figures for observed IOPs values at the iSCSI LUN level, but of course those are several layers under the actual VHD. I'm assuming, but have no documentation to confirm it, that VMM/Hyper-V is monitoring IOPs at the VM <-> VHD layer, which would be using 4K blocks. Now the LUN is at 512 byte sector size, the NTFS volume underlying the CSV is at 64K clusters and the CSV will also have it's own caching, so what would I monitor at the Hypervisor level to see the actual IOPs presented by the VM?

· 3 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Basically, the IOPs is implemented or enforced on Windows server, it's not limited by SCVMM. SCVMM just deploys the policy settings of IOPs.

The following blog article introduces the Storage Qos in more depth, and I hope it can help you understand it better.

Where Does Storage QoS Live In Windows Server 2012 R2 Hyper-V

Plus, in Windows server 2016, Storage Qos has been improved, and the you can refer to the following resources for more details.

Take a look at Storage QoS Policies in Windows Server 2016
Storage Quality of Service


1 Vote 1 ·

Thanks Andy, I really do appreciate your detailed responses.

After digging through all the doco, it appears I can't use Storage QoS with VMM 2016 as I'd hoped, as even though it is supported on CSVs, it's only supported on hyperconverged (i.e. S2D) infrastructure in VMM 2016. I can still set Maximum IOPs directly, though and the pointers to "normalized IOPs" have at least given me a better understanding of what the server thinks is an IOP!

0 Votes 0 ·

You are welcome! It's good to hear it helps.

0 Votes 0 ·
AndyLiu-MSFT avatar image
0 Votes"
AndyLiu-MSFT answered ·

It's not reported by either the guest or underlying storage. You can get the IOPs value by using the PowerShell cmdlet on the Hyper-V host, on which the VM was created.

You can run the following cmdlets by opening the PowerShell console with Administrator privileges on the host.

 Get-VM -Name aws02 | Enable-VMResourceMetering
 Measure-VM -Name aws02 | Select-Object -Property AggregatedAverageNormalizedIOPS

Please replace the 'aws02' with the VM name in your environment. Plus, you can generate read/write disk activities so that you can view the changing of values.

The following article introduces the info about IOPs.

Storage Quality of Service for Hyper-V



· 2 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi Andy,

Thanks once again for your response! I have a few questions. Some of these questions might be answered by your linked article, but the link you provided doesn't work for me (can you give the title of the document so that I can search for it?)

  • Does VMM require Hyper-V resource metering to be enabled on the VM for the MaximumIOPs function to work? In other words, does this configuration do nothing unless metering is on?

  • How does Hyper-V use this AggregatedAverageNormalizedIOPS figure to limit IOPs?

  • How do I ensure VMM always enables Hyper-V resource metering on new VMs without having to manually enable it on the deployed hypervisor? I notice that new VMs don't have it turned on.












0 Votes 0 ·

I have updated the link for the documentation in the previous answer. You can also see below.

Storage Quality of Service for Hyper-V

  1. Resource Metering is used for reporting metrics only. VMM doesn't require for Resource Metering to be enabled on the VM for the MaximumIOPs function to work.

  2. AggregatedAverageNormalizedIOPS is just a reported metric. It's not used to limit IOPs by Hyper-V.

  3. There is no GUI configuration settings. You only can use PowerShell to enable Resource Metering. Please click here for more details about Enable-VMResourceMetering.


0 Votes 0 ·