question

ArtemBurdin-6224 avatar image
0 Votes"
ArtemBurdin-6224 asked ArtemBurdin-6224 answered

Hyper-V CSV datastore Latency issues

I have a Hyper-V 20219 FT clusters with FC SAN storage.

I have collected both perfmon counters and also the statistics from the storage and have observed latency issues:
From the storage I observe average 6.9 ms latency for the datastore and from the Hyper-V hosts I observe 100.2 ms latency. A whooping 14x difference
I do understand that there can be a difference in measurement intervals and added latency from the SAN, but this seems too much of difference.

What should I do to the recheck if there is really a latency problem?

What should I also check to find the root cause?

windows-server-hyper-v
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi,

I would like to check if the reply could be of help? If yes, please help accept answer, so that others meet a similar issue can find useful information quickly. If you have any other concerns or questions, please feel free to feedback.

Best Regards,
Joan

0 Votes 0 ·

Hi,

I would like to check if the reply could be of help? If yes, please help accept answer, so that others meet a similar issue can find useful information quickly. If you have any other concerns or questions, please feel free to feedback.

Best Regards,
Joan

0 Votes 0 ·
jiayaozhu-MSFT avatar image
0 Votes"
jiayaozhu-MSFT answered

Hi,

Thanks for your posting!

Based on your descriptions, I would like to check where do the two sets of data "average 6.9 ms latency for the datastore and from the Hyper-V hosts I observe 100.2 ms" actually come from. In addition, what are the names for the two metrics respectively?

Secondly, the article I gave you before:

https://techcommunity.microsoft.com/t5/failover-clustering/cluster-shared-volume-performance-counters/ba-p/371980

includes details about how to evaluate the performance metrics. If the evaluation is fine then no matter how huge the difference is between the two metrics, your CSV is running properly.

In addition, here is an article for assessing performance collected from performance monitor:

https://docs.microsoft.com/en-us/archive/blogs/askcore/measuring-disk-latency-with-windows-performance-monitor-perfmon

Thirdly, if you really want to check your latency in your CSV, for one thing, I will continue to research this topic for you, for another, I may suggest you to analyze logs, which is beyond our Q&A scope, the logs are included in %SystemRoot%\System32\winevt, you can just filter logs about CSV performance from this path.

Thanks for your understanding!

BR,
Joan


If the Answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

ArtemBurdin-6224 avatar image
0 Votes"
ArtemBurdin-6224 answered

I have noticed that the latency observed from perfmon counters inside a VM seems at times much higher than the latency for datastore observed from hyper-v host level.

The datastore hosts 3 VMs, all of which are on one host.

The Physical disk performance object -> Avg. Disk sec/Read counter inside a VM at times show latency from 100 ms to 1000 ms in 1 hour period while CSVFS FS -> Avg. Disk sec/Read counter for the datastore at a host level where VMs are hosted shows values less then 100 ms


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.