Packet descriptors and extensions

Warning

Some information in this topic relates to prereleased product, which may be substantially modified before it's commercially released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

NetAdapterCx is preview only in Windows 10, version 1809.

In NetAdapterCx, packet descriptors are small, compact, runtime-extensible structures that describe a network packet. Each packet requires the following:

  • One core descriptor
  • One or more fragment descriptors
  • Zero or more packet extensions

The core descriptor of the packet is the NET_PACKET structure. It contains only the most basic metadata applicable to all packets, such as the framing layout of a given packet and the index to the packet's first fragment descriptor.

Each packet must also have one or more fragment descriptors, or NET_PACKET_FRAGMENT structures, that describe the location within system memory where the packet data resides.

Packet extensions are optional and hold per-packet metadata for scenario-specific features. For instance, extensions can hold offload information for checksum, large send offload (LSO), and receive segment coalescence (RSC), or they can hold application-specific details.

Together, these descriptors and extensions hold all the metadata about a network packet. Here are two examples of how they describe a packet. The first figure shows a scenario where the entire packet is stored inside a single memory fragment and checksum offload has been turned on.

1 fragment packet layout

The second figure shows a packet stored across two memory fragments, with both RSC and checksum offload enabled.

2 fragments packet layout

Storage of packet descriptors

The core descriptors and fragment descriptors are stored indepenently in two separate ring buffers, the packet ring and fragment ring. Every core descriptor in the packet ring has indices into the fragment ring for locating that packet's fragment descriptors. Another data structure, NET_DATAPATH_DESCRIPTOR, groups the packet ring and fragment ring together for a given packet queue.

multi-ring layout

Every packet queue has its own NET_DATAPATH_DESCRIPTOR structure, and, consequently, its own packet ring, fragment ring, and descriptors in those rings. Therefore, the network data transfer operation of each packet queue is completely independent. To learn more about packet queues, see Transmit and receive queues.

Client drivers are advised to use predefined convenient macros to access the packet ring, the fragment ring and the descriptors they contain. To learn more about these macros, see Using the ring buffer.

Packet descriptor extensibility

Extensibility is a core feature of the NetAdapterCx packet descriptor, forming the foundation for the descriptor's versionability and performance. At runtime, the operating system allocates all packets descriptors for each packet queue in a contiguous block, together with any avaiable extensions. Each extension block is immediately behind the core descriptor, as shown in the following figure:

NetAdapterCx packet descriptor layout

NIC client drivers are not permitted to hardcode the offset to any extension block. Instead, they must query at runtime for the offset to any particular extension. For example, a driver might query the offset to Extension B and get back 70 bytes like in the following figure:

Querying the offset to an extension of the core packet descriptor

Once a packet queue and its descriptors are created, all their extension offsets are guaranteed by the system to be constant, so drivers don't have to re-query offsets often. Furthermore, because all extensions are pre-allocated by the system in a block at the time the packet queue is initialized, there is no need for runtime allocation of blocks, searching a list for a specific descriptor, or having to store pointers to every packet extension.

Packet descriptor versionability

NetAdapterCx's core packet descriptor can be easily extended in future releases by adding new fields to the end, such as in the following figure:

NetAdapterCx core packet descriptor versioning

Newer client drivers that know about the V2 fields can access them, while older V1-only drivers will use extension offsets to skip over the V2 fields so they can access the fields they do understand. In addition, each extension can be versioned in the same way, as the following figure shows:

NetAdapterCx packet extension versioning

A client driver that understands the new extension can use it. Other client drivers can skip over the new fields. This permits different parts of the packet descriptor to be versioned independently.

Packet descriptors and datapath performance

The extensibility feature outlined previously provides benefits to help client drivers meet the performance requirements of NICs that are capabable of hundreds of gigabits per second, with thousands of queues:

  1. The packet descriptors are kept as compact as possible to improve CPU cache hits, as features and extensions that aren't used occupy 0 bytes of space in the descriptors.
  2. There is no pointer dereferencing, only offset arithmetic because extensions are in-line, which not only saves space but also helps with CPU cache hits.
  3. Extensions are allocated at queue creation time, so drivers don't have to allocate and deallocate memory in the active data path or deal with lookaside lists of context blocks.

Using packet extensions

Important

Currently, client drivers are limited to pre-existing packet extensions defined by the operating system.

Registering packet extensions

The first step in working with packet extensions in your NIC client driver is to declare your supported hardware offloads. When you advertise support for offloads such as checksum and LSO, NetAdapterCx automatically registers the associated packet extensions on your behalf.

For a code example of advertising hardware offloads for checksum and LSO, see NetAdapterCx hardware offloads.

Querying packet extension offsets for datapath queues

After registering packet extensions by declaring your hardware offload support, you'll need the extension offsets to access each one as you process your packets. To reduce calls out of your driver and improve performance, you can query the offsets for your extensions during the EvtNetAdapterCreateTx(Rx)Queue callback function and store the offset information in your queue context. Here is an example for a transmit queue. This example is similar to the example on EvtNetAdapterCreateTxQueue but focuses only on packet extensions.

NTSTATUS
MyAdapterCreateTxQueue(
    _In_    NETADAPTER          Adapter,
    _Inout_ PNETTXQUEUE_INIT    TxQueueInit
)
{
    NTSTATUS status = STATUS_SUCCESS;

    // Prepare the configuration structure
    NET_PACKET_QUEUE_CONFIG txConfig;
    NET_PACKET_QUEUE_CONFIG_INIT(
        &txConfig,
        EvtTxQueueAdvance,
        EvtTxQueueSetNotificationEnabled,
        EvtTxQueueCancel);

    // Configure other Tx queue properties such as packet contexts
    ...

    // Create the transmit queue
    NETPACKETQUEUE txQueue;
    status = NetTxQueueCreate(
        txQueueInit,
        &txAttributes,
        &txConfig,
        &txQueue);

    // Get the queue context for storing the queue ID and packet extension offset info
    PMY_TX_QUEUE_CONTEXT queueContext = GetMyTxQueueContext(txQueue);

    // Query checksum packet extension offset and store it in the context
    NET_PACKET_EXTENSION_QUERY extension;
    NET_PACKET_EXTENSION_QUERY_INIT(
        &extension,
        NET_PACKET_EXTENSION_CHECKSUM_NAME,
        NET_PACKET_EXTENSION_CHECKSUM_VERSION_1);

    queueContext->ChecksumExtensionOffset = NetTxQueueGetPacketExtensionOffset(txQueue, &extension);

    // Query Large Send Offload packet extension offset and store it in the context
    NET_PACKET_EXTENSION_QUERY_INIT(
        &extension,
        NET_PACKET_EXTENSION_LSO_NAME,
        NET_PACKET_EXTENSION_LSO_VERSION_1);
    
    queueContext->LsoExtensionOffset = NetTxQueueGetPacketExtensionOffset(txQueue, &extension);

    return status;
}

Getting packet extensions at runtime

Once you have stored extension offsets in your queue context, you can use them any time you need information in an extension. For example, you could call the NetPacketGetPacketChecksum method while you program descriptors to hardware:

    // Get the extension offset from the device context
    PMY_TX_QUEUE_CONTEXT queueContext = GetMyTxQueueContext(txQueue);
    size_t checksumOffset = queueContext->ChecksumExtensionOffset;

    // Get the checksum info for this packet
    NET_PACKET_CHECKSUM* checksumInfo = NetPacketGetChecksum(packet, checksumOffset);

    // Do work with the checksum info
    if(checksumInfo->Layer4 == NET_PACKET_TX_CHECKSUM_REQUIRED)
    {
        ...
    }

Predefined packet extension constants and helper methods

NetAdapterCx provides definitions for known packet extensions constants.

Constant Definition
NET_PACKET_EXTENSION_INVALID_OFFSET Guards against invalid offset sizes.
  • NET_PACKET_EXTENSION_CHECKSUM_NAME
  • NET_PACKET_EXTENSION_CHECKSUM_VERSION_1
  • NET_PACKET_EXTENSION_CHECKSUM_VERSION_1_SIZE
The name, version, and size of the checksum packet extension.
  • NET_PACKET_EXTENSION_LSO_NAME
  • NET_PACKET_EXTENSION_LSO_VERSION_1
  • NET_PACKET_EXTENSION_LSO_VERSION_1_SIZE
The name, version, and size of the large send offload (LSO) packet extension.
  • NET_PACKET_EXTENSION_RSC_NAME
  • NET_PACKET_EXTENSION_RSC_VERSION_1
  • NET_PACKET_EXTENSION_RSC_VERSION_1_SIZE
The name, version, and size of the receive segment coalescence (RSC) packet extension.

Additionally, NetAdapterCx provides three helper methods that act as wrappers around the NetPacketGetExtension method. Each of these methods returns a pointer to the appropriate type of structure.

Method Structure
NetPacketGetPacketChecksum NET_PACKET_CHECKSUM
NetPacketGetPacketLargeSendSegmentation NET_PACKET_LARGE_SEND_SEGMENTATION
NetPacketGetPacketReceiveSegmentCoalescence NET_PACKET_RECEIVE_SEGMENT_COALESCENCE