question

LevStipakov-4683 avatar image
0 Votes"
LevStipakov-4683 asked LevStipakov-4683 answered

Indicating packets from WSK to NetAdapterCx

Hello,

I am working on the virtual network adapter driver which uses NetAdapterCx, WSK and CNG.

Here is what driver does on Tx path:

  • iterate over packets in "post" subsection of NET_RING

  • for each packet, get the first fragment and fragment's MDL. If MDL is NULL (packet was bounced by NetAdapter), allocate MDL from fragment's VA.

  • encrypt MDL chain with CNG using chaining mode (account for plaintext length must be multiplier of block size)

  • send MDL chain with WskSendTo/WskSend, pass NET_PACKET as completion routine context

  • move sent fragments and packets to "drain" subsection by adjusting NextIndex

  • in WSK completion routine, set NET_PACKET::Scratch to 1 to indicate that packet has been sent

  • iterate over packets in "drain" subsection and drain them to OS by adjusting BeginIndex if Scratch is set to 1

The code could be found here: https://github.com/lstipakov/ovpn-dco-win/blob/zerocopy/txqueue.cpp#L139

My question is - what would be the proper way to implement the same "zerocopy" approach on Rx path? At the moment it works like this:

  • packet is received by WskReceiveFromEvent callback

  • packet is decrypted into ciphertext buffer, fetched from pre-allocated "producer" pool (I use DMF_BufferQueue)

  • ciphertext buffer is enqueued into "consumer" queue

  • call NetRxQueueNotifyMoreReceivedPacketsAvailable() to trigger Rx queue's Advance callback

  • in Rx Advance callback, iterate over fragments, dequeue buffer from "consumer" queue and copy buffer content to fragment's VA

  • Buffer is "reused" by placing into "producer" pool

I can do decryption in-place and make WSK retain data by returning STATUS_PENDING from WskReceiveFromEvent callback, but how do I "indicate" data provided by WSK to NetAdapter without memcpying? Can I somehow tell NET_FRAGMENT "hey use this MDL which I got from WSK and decrypted in-place" ?

Ping @JeffreyTippetMSFT-8232


windows-hardware-code-ndis
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

LevStipakov-4683 avatar image
0 Votes"
LevStipakov-4683 answered

(copied from https://community.osr.com/discussion/comment/302680/#Comment_302680)

I think I figured it out by reading NetAdapter's code. Here is how I made it work in my driver:

  • When setting datapath capabilities, we use NET_ADAPTER_RX_CAPABILITIES_INIT_DRIVER_MANAGED macro. We also specify EvtAdapterReturnRxBuffer callback.

  • In WskReceiveFromEvent callback we iterate over WSK_DATAGRAM_INDICATION list and enqueue each item into cosumer pool (from DMF_BufferQueue)

          // each datagram indication is one UDP datagram
           for (PWSK_DATAGRAM_INDICATION next; dataIndication != NULL; dataIndication = next) {
               next = dataIndication->Next;
    
               // break list so that we can pass individual WSK_DATAGRAM_INDICATION to WskRelease
               dataIndication->Next = NULL;
    
               // fetch buffer from producer
               OVPN_RX_BUFFER* rxBuffer;
               LOG_IF_NOT_NT_SUCCESS(OvpnBufferQueueFetch(device->DataRxBufferQueue, &rxBuffer));
               rxBuffer->DatagramIndication = dataIndication;
    
               // enqueue buffer to consumer, which is consumed by NetAdapter's RX Advance callback
               OvpnBufferQueueEnqueue(device->DataRxBufferQueue, rxBuffer);
           }
    
           // tell NetAdapter that we have something to consume
           OvpnAdapterNotifyRx(device->Adapter);
    

  • In RX queue's Advance callback we iterate over queue's fragments. We dequeue datagram indication and set fragment properties based on datagram indication's buffer and MDL:

        while (NetFragmentIteratorHasAny(&fi)) {
             OVPN_RX_BUFFER* buffer;
             // nothing has arrived and decrypted yet?
             if (!NT_SUCCESS(OvpnBufferQueueDequeue(bufferQueue, &buffer))) {
                 break;
             }
    
             fragment = NetFragmentIteratorGetFragment(&fi);
    
             PWSK_DATAGRAM_INDICATION datagramIndication = buffer->DatagramIndication;
             // return buffer to producer queue
             OvpnBufferQueueReuse(bufferQueue, buffer);
    
             PMDL mdl = datagramIndication->Buffer.Mdl;
    
             // TODO: correctly adjust crypto/ovpn overhead
             fragment->ValidLength = datagramIndication->Buffer.Length - 8;
             fragment->Offset = datagramIndication->Buffer.Offset + 8;
             fragment->Capacity = MmGetMdlByteCount(mdl);
    
             NET_FRAGMENT_VIRTUAL_ADDRESS* virtualAddr = NetExtensionGetFragmentVirtualAddress(&queue->VirtualAddressExtension, NetFragmentIteratorGetIndex(&fi));
             virtualAddr->VirtualAddress = (PUCHAR)(MmGetSystemAddressForMdlSafe(mdl, LowPagePriority));
    
             // TODO: handle case when packet (DataIndication) contains multiple fragments
             NET_PACKET* packet = NetPacketIteratorGetPacket(&pi);
             packet->FragmentIndex = NetFragmentIteratorGetIndex(&fi);
             packet->FragmentCount = 1;
    
             packet->Layout = {};
    
             // NetAdapter will call ReturnRxBuffer callback when it is done with buffers, there we return datagramIndication back to WSK
             returnCtx = NetExtensionGetFragmentReturnContext(&queue->ReturnContextExtension, NetFragmentIteratorGetIndex(&fi));
             returnCtx->Handle = (NET_FRAGMENT_RETURN_CONTEXT_HANDLE)datagramIndication;
    
             NetFragmentIteratorAdvance(&fi);
             NetPacketIteratorAdvance(&pi);
         }
         NetFragmentIteratorSet(&fi);
         NetPacketIteratorSet(&pi);
    

  • In NetAdapter's EvtAdapterReturnRxBuffer callback we call WskRelease on a datagram indication, which is passed as NET_FRAGMENT_RETURN_CONTEXT_HANDLE by NetAdapter:

        _Use_decl_annotations_
         void
         OvpnEvtAdapterReturnRxBuffer(NETADAPTER netAdapter, NET_FRAGMENT_RETURN_CONTEXT_HANDLE rxReturnContext)
         {
             POVPN_ADAPTER adapter = OvpnGetAdapterContext(netAdapter);
             POVPN_DEVICE device = OvpnGetDeviceContext(adapter->WdfDevice);
    
             PWSK_DATAGRAM_INDICATION dataIndication = (PWSK_DATAGRAM_INDICATION)rxReturnContext;
    
             LOG_IF_NOT_NT_SUCCESS(((WSK_PROVIDER_DATAGRAM_DISPATCH*)device->Socket.Socket->Dispatch)->WskRelease(device->Socket.Socket, dataIndication));
         }
    

Not sure if this is the "best practices", but it works and doesn't crash on iperf3 tests.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.