NetAdapterCx receive side scaling (RSS)
Some information in this topic relates to prereleased product, which may be substantially modified before it's commercially released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
NetAdapterCx is preview only in Windows 10, version 1809.
Receive side scaling (RSS) is a network driver technology that enables the efficient distribution of network receive processing across multiple CPUs in multiprocessor systems. RSS improves system performance and increases network scalability by harnessing all available processors in a system and dynamically rebalancing CPU workloads.
This topic highlights RSS for NetAdapterCx client drivers and assumes knowledge of RSS concepts and terminology. For more information about RSS in general, including diagrams illustrating RSS in different hardware scenarios, see Receive Side Scaling.
Overview of RSS in NetAdapterCx
RSS in NetAdapterCx focuses on ease of configuration, simplicity of enablement and disablement, and abstraction of processor-to-interrupt complexity. A client driver for an RSS-capable NIC only needs to meet three criteria to support RSS in NetAdapterCx:
- The driver must set RSS capabilities when starting a net adapter, but before calling NetAdapterStart. This includes implementing four RSS callbacks and registering them in the RSS capabilities structure.
- The driver's datapath queues must be created and ready to accept requests.
- The driver must be in the D0 power state.
The design of RSS in NetAdapterCx guarantees that the system will not call a client's RSS callbacks and enable RSS until the very end of the power-up sequence. Clients do not have to manage indirection table move requests or handle other RSS events until everything they need is ready.
Later, when the driver is unloading, NetAdapterCx will not call RSS callbacks after datapath queues have been destroyed during the power-down sequence. Since datapath queues are torn down as the first step during power-down, this means that clients do not have to handle possible RSS events at any other stage during power-down.
Setting RSS capabilities
To get started with RSS in NetAdapterCx, follow these steps:
- When starting your net adapter, tell the system about your hardware's RSS capabilities and constraints using the NET_ADAPTER_RECEIVE_SCALING_CAPABILITIES structure.
- Initialize the capabilities structure by calling NET_ADAPTER_RECEIVE_SCALING_CAPABILITIES_INIT.
- When you initialize the RSS capabilities structure, set the structure's RSS callback members to register your implementations for these callbacks:
- Set the RSS capabilities structure's SynchronizeSetIndirectionEntries as appropriate.
- Pass the initialized RSS capabilities structures to the NetAdapterSetReceiveScalingCapabilities method.
Enabling and disabling RSS
After you set RSS capabilities, the system will continue with the power-up sequence for your driver. NetAdapterCx starts invoking your driver's RSS callbacks once the final step of creating datapath queues has finished. At this point, RSS can be enabled and disabled as needed by the system.
You should not clear or reset your indirection table when enabling or disabling RSS. The framework will set the your initial indirection table state.
NetAdapterCx enables RSS by invoking your driver's EvtNetAdapterReceiveScalingEnable callback. In the context of this callback, you typically enable control bits in your hardware.
For a code example of enabling RSS, see EvtNetAdapterReceiveScalingEnable.
NetAdapterCx disables RSS by invoking your driver's EvtNetAdapterReceiveScalingDisable callback. Here, you typically disable the control bit in your hardware that you previously set in EvtNetAdapterReceiveScalingEnable.
For a code example of disabling RSS, see EvtNetAdapterReceiveScalingDisable.
Setting the hash secret key
Once RSS is enabled, NetAdapterCx invokes the EvtNetAdapterReceiveScalingSetHashSecretKey callback to provide your driver with the hash secret key your NIC should use in verifying hash calculations. This callback can be invoked at any time when RSS is running if the hash secret key changes.
For a code example of setting the hash secret key, see EvtNetAdapterReceiveScalingSetHashSecretKey.
Moving indirection table entries
While RSS is running on the system, upper layer protocol drivers monitor processor workload and maintain an indirection table that maps receive queues to processors. When the protocol driver needs to rebalance processor workload in RSS, it first calculates a new mapping for each indirection table entry to a new processor. Then the protocol passes this information to NetAdapterCx, which handles the complexity of mapping receive queues and hardware interrupt vectors to the correct processor on behalf of your NIC client driver. NetAdapterCx stores the new indirection table, with entries mapped to receive queue IDs, in a NET_ADAPTER_RECEIVE_SCALING_INDIRECTION_ENTRIES structure and passes it to your driver when it invokes the EvtNetAdapterReceiveScalingSetIndirectionEntries callback function.
In this callback, you move each entry in your NIC's indirection table to the specified receive queue. Each NET_ADAPTER_RECEIVE_SCALING_INDIRECTION_ENTRY structure in the NET_ADAPTER_RECEIVE_SCALING_INDIRECTION_ENTRIES array contains the hash index for that entry in the table, the new receive queue to which to assign the entry, and a status field indicating if that individual move succeeded or not.
The method of assigning index entries to hardware receive queues depends on the design of your NIC and the number of receive queues it has. For more information and a code example, see EvtNetAdapterReceiveScalingSetIndirectionEntries.
We'd love to hear your thoughts. Choose the type you'd like to provide:
Our feedback system is built on GitHub Issues. Read more on our blog.