question

RasalGhul1 avatar image
0 Votes"
RasalGhul1 asked RasalGhul1 commented

Storage Spaces Direct Inter-Node SMB Traffic

Hi,

I'm justing learning how to setup a hyperconverged S2D Cluster (thanks to Jaromir's MSLabs scripts on Github).

There is a question to which I can't find an answer.
Which NICs can be used to set the SMB/Inter-Node traffic for storage replication?
Can dedicated NICs be used for replication?

I come from VMware ESXi and I am used to be able to choose which services the NICs can use.

For network configuration of S2D, there are various posts related to RDMA, mainly RoCE with DCB. Dedicated NICs are not discussed here either. Configuration without RDMA is almost not discussed at all.

Similar questions were also asked in these older posts. But unfortunately without a clear answer.

Storage spaces direct Force Storage traffic on an specified nic:
https://social.technet.microsoft.com/Forums/en-US/e422c99f-b74e-40b4-a122-9ae15466b1ec/storage-spaces-direct-force-storage-traffic-on-an-specified-nic?forum=ws2016

Storage Spaces Direct networking:
https://social.technet.microsoft.com/Forums/windows/en-US/a16cf824-5345-4baa-a21d-f6cd21ed8e5d/storage-spaces-direct-networking?forum=winserverClustering

I would be happy if someone can give me a hint!
Regards

windows-serverwindows-server-hyper-v
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

jiayaozhu-MSFT avatar image
0 Votes"
jiayaozhu-MSFT answered RasalGhul1 commented

Hi,

Thanks for replying!

When you use SMB Multichannel with a RSS-capable network adapter, SMB creates multiple TCP/IP connections for that particular session. so, yes, SMB multichannel does automatically seek for high speed NIC ports for your storage. Besides, in S2D, SMB multichannel is enabled by default, so you do not have to worry about how to configure it. You just need to ensure all of your hardware device meet the configuration requirements and to follow the guidance to configure your S2D in a correct way. And if you have encountered any problems during your use of S2D, please feel free to feedback, we will try to help improve your user experience.

Here is some articles you may find useful:

Manage SMB Multichannel
https://docs.microsoft.com/en-us/azure-stack/hci/manage/manage-smb-multichannel

Deploy Storage Spaces Direct
https://docs.microsoft.com/en-us/windows-server/storage/storage-spaces/deploy-storage-spaces-direct

Thanks for your support! Have a nice day! : )

BR,
Joan


If the Answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.



· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @jiayaozhu-MSFT

Thank you for taking the time. That answers my question very well.

TL;DR
When deploy a S2D Cluster the way as official described in the docs, everything just works in the best possible way.

0 Votes 0 ·
RasalGhul1 avatar image
0 Votes"
RasalGhul1 answered

Hello @jiayaozhu-MSFT

Thanks for your answer.

With "dedicated NIC" I actually mean one port from a network adapter. Let's assume I have the following server network configuration:

1x Onboard 1GbE NIC with 4x Ports
1x AddOn 10GbE NIC with 2x Ports

I would like to use the 2x Onboard 1GbE Ports for VM Traffic and the other 2x Onboard 1GbE Ports for Management / ClusterHB Traffic.
The 2x AddOn 10GbE Ports should be used only for S2D storage replication traffic.

I want to get sure that only the fastest ports are used for replication and the slower are not involved. Currently I don't know or see where to configure which ports are used for this.

It seems that SMB Multichannel automatically takes care of it without the need or possibly to configure it any further?

Regards

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

jiayaozhu-MSFT avatar image
0 Votes"
jiayaozhu-MSFT answered jiayaozhu-MSFT edited

Hi,

Thanks for your posting!

Based on your descriptions, firstly, I am not sure if dedicated NIC is a type of physical NIC, in my knowledge, storage replication require seperate storage network or bandwidth and that's why we call the NIC(s) that used to set up this seperate network as dedicated NIC. So firstly I would like to check if your dedicated NIC is what I think. If so, it seems like that you need to learn more about SMB multichannel in S2D.

Here are some articles for best practices for configuration SMB multi-channel in a S2D:

https://docs.microsoft.com/en-us/windows-server/failover-clustering/smb-multichannel

https://www.tech-coffee.net/design-the-network-for-a-storage-spaces-direct-cluster/

(Please note: Information posted in the given link is hosted by a third party. Microsoft does not guarantee the accuracy and effectiveness of information.)

Besides, it seems that you want to configure the network for storage replication as the fastest, so I will go to research on this demand and I will tell you anything useful as soon as possible.

Thanks for your support!

BR,
Joan


If the Answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.