Network Monitor 3.3 RWS Parser Basics, Part 3: Observing single-connection DNS traffic

  • Article

Published: April 2010

Author:
Jim Harrison - Program Manager

Reviewers:
Yuri Diogenes - Sr Security Support Escalation Engineer
Michael Hawker - Program Manager

Editors:
Michelle Friedmann – Technical Editor
Meir Feinberg - Technical Writer

Introduction

In part 1 of this article series, I detailed the RWS message set and described the relationship between those messages. In part 2, I discussed single-connection TCP using HTTP as the application protocol. In this part, I’ll describe and discuss single-connection UDP traffic using DNS as the application protocol.

DNS via Forefront TMG Client Traffic Examples

In the following examples, I’ll use the nslookup.exe utility to take you through four scenarios:

  • A Successful scenario (we need to know what looks right).

  • A failed scenario based on destination rejection by a rule. See Deny Rule Action.

  • A failed scenario based on name resolution failure. See Name Resolution Failure.

  • A failed scenario based on connection failure (not rule-based). See Connection Failure.

Successful scenario

In this example, I use the nslookup.exe utility to issue a name lookup request to the DNS server at 4.2.2.2 for isatools.org. In order to do this, I’ll run it in a command window using the following command: nslookup.exe -type=a isatools.org 4.2.2.2. As with previous examples, I need to start a network capture on the Forefront TMG Client computer before I run my tests. As a reminder, the command I’ll use is nmcap /capture /network *. Now let’s examine the results of this effort. I open the network capture and apply the most basic Forefront TMG Client filter, as shown in the following figure:

figure 1

Figure 1 - Nslookup.exe via Forefront TMG Client traffic profile

In this case, the traffic profile behaves a bit differently than it did for the HTTP examples. The differences are:

  1. There is no Forefront TMG Client Get host by name message or resulting Forefront TMG Host entry response in this conversation. This message is not seen because nslookup.exe was instructed to use direct queries to a specific DNS server at 4.2.2.2.

  2. The following three RWS messages are not seen in the HTTP examples in part 2 of this series of articles:

    1. Bind v12— this message tells Forefront TMG that the application needs to create a socket binding. Because UDP traffic is not managed by the network stack in the same way as TCP, a BIND action is necessary before traffic can be sent using UDP.

      - RWS: 0x2 Bind v12 request from nslookup.exe to UDP Dynamic. nslookup.exe bound to 10.10.255.129/1205
  - RwstPacket: Bind v12 request from nslookup.exe to UDP Dynamic. nslookup.exe bound to 10.10.255.129/1205
     NullChar: 0 (0x0)
     ProtoSig: RWS
     PktLen: 281 (0x119)
     Reserved1: 0 (0x0)
     Flags: 0 (0x0)
     Reserved2: 0 (0x0)
     OpCode: Bind v12
     DataLength: 0 (0x0)
   + ClientConnectionHandle: 0x2
   + ServerConnectionHandle: 0x0
   + ClientMappingHandle: 0x0
   + ServerMappingHandle: 0x0
   - RwsMessage: request from nslookup.exe to UDP Dynamic. nslookup.exe bound to 10.10.255.129/1205
    - Bind12Data: request from nslookup.exe to UDP Dynamic. nslookup.exe bound to 10.10.255.129/1205
     - BindEndpoint: Dynamic
      - Socket: Dynamic
         Family: IPv4
         Port: 0 (0x0)
         Addr: 0.0.0.0
         Padding: Binary Large Object (8 Bytes)
        Padding: Binary Large Object (12 Bytes)
     - ClientEndpoint: Dynamic/1205
      - Socket: Dynamic/1205
         Family: IPv4
         Port: 1205 (0x4B5)
         Addr: 0.0.0.0
         Padding: Binary Large Object (8 Bytes)
        Padding: Binary Large Object (12 Bytes)
       BindProtocol: UDP
       BindFlags: 0 (0x0)
       Padding: Binary Large Object (36 Bytes)
     - AppAndClientInfo: nslookup.exe
        Application: nslookup.exe
      + ClientInfo: - No detailed client information -

      • The ClientConnectionHandle field provides a socket mapping identifier for Forefront TMG and the Forefront TMG Client. This field is used to populate the RWSId properties.

      • The BindEndpoint struct tells Forefront TMG how it should perform the bind action. When the application issues a Winsock bind() using an empty IP address and port, this is referred to as “dynamic binding” and essentially allows Winsock (and thus Forefront TMG) to select the IP address dynamically and the port from the ephemeral port range.

      • The ClientEndpoint struct tells Forefront TMG which IP address and port the Forefront TMG Client will use to exchange traffic for the successful bind socket. An IP address of “0.0.0.0” means “assume the same IP address used for the control channel”.

        Note

        This is the same information that is supplied in the ClientEndpoint struct portion of the Connect v12 message associated with this RWSId.

      • The BindProtocol field tells Forefront TMG which transport protocol to use for this binding. Only TCP and UDP are valid in this field.

    2. Bind Reply v12— this message tells the Forefront TMG Client that Forefront TMG successfully issued a bind request to Winsock on the Forefront TMG computer and is providing some information derived from that action.

      - RWS: 0x2 Bind Reply v12 UDP to nslookup.exe on Dynamic/46441 successful
  - RwstPacket: Bind Reply v12 UDP to nslookup.exe on Dynamic/46441 successful
     NullChar: 0 (0x0)
     ProtoSig: RWS
     PktLen: 281 (0x119)
     Reserved1: 0 (0x0)
     Flags: 0 (0x0)
     Reserved2: 0 (0x0)
     OpCode: Bind Reply v12
     DataLength: 0 (0x0)
   + ClientConnectionHandle: 0x2
   + ServerConnectionHandle: 0x101a0
   + ClientMappingHandle: 0x0
   + ServerMappingHandle: 0x0
   - RwsMessage: UDP to nslookup.exe on Dynamic/46441 successful
    - BindReply12Data: UDP to nslookup.exe on Dynamic/46441 successful
     - ExternalEndpoint: Dynamic/46441
      - Socket: Dynamic/46441
         Family: IPv4
         Port: 46441 (0xB569)
         Addr: 0.0.0.0
         Padding: Binary Large Object (8 Bytes)
        Padding: Binary Large Object (12 Bytes)
     - ClientEndpoint: Dynamic/1205
      - Socket: Dynamic/1205
         Family: IPv4
         Port: 1205 (0x4B5)
         Addr: 0.0.0.0
         Padding: Binary Large Object (8 Bytes)
        Padding: Binary Large Object (12 Bytes)
       BoundProtocol: UDP
       Padding: Binary Large Object (40 Bytes)
     - AppAndClientInfo: nslookup.exe
        Application: nslookup.exe
      + ClientInfo: - No detailed client information -

      • The ClientConnectionHandle field associates this response message with the Bind v12 request issued by the Forefront TMG Client.

      • The ServerConnectionHandle field provides the Forefront TMG Client with a value Forefront TMG uses to refer to the successful bind action.

      • The ExternalEndpoint struct is updated to reflect the port bound by Forefront TMG on the interface that faces the destination network.

      • The ClientEndpoint struct is a copy of the one sent by the client.

      • The BoundProtocol field indicates the transport protocol used for this binding.

    3. Close— this message tells Forefront TMG to stop exchanging data and release any sockets related to ClientConnectionHandle and ServerConnectionHandle.

      Note

      The Forefront TMG Client does not expect an acknowledgement of this message from Forefront TMG.

      - RWS: 0x3 Close  
  - RwstPacket: Close  
     NullChar: 0 (0x0)
     ProtoSig: RWS
     PktLen: 281 (0x119)
     Reserved1: 0 (0x0)
     Flags: 0 (0x0)
     Reserved2: 0 (0x0)
     OpCode: Close
     DataLength: 0 (0x0)
   + ClientConnectionHandle: 0x3
   + ServerConnectionHandle: 0x101a1
   + ClientMappingHandle: 0x0
   + ServerMappingHandle: 0x0
     RwsMessage:  
     Unparsed: Binary Large Object (246 Bytes)

      • The ClientConnectionHandle field indicates the sockets used by the Forefront TMG Client that will not be used again.

      • The ServerConnectionHandle field indicates the sockets used by Forefront TMG that should be released.

  3. The WSP traffic is UDP-based because nslookup.exe uses DNS queries on UDP to accomplish its tasks. Thus, the highest-layer protocol in these examples will appear in the frame summary window as “DNS”. The frame details window provides details similar to the HTTP examples, except that UDP replaces TCP and DNS replaces HTTP in the protocol hierarchy.

    figure 2 Figure 2 – WSP on UDP protocol hierarchy

  4. The Connect v12 request messages in this conversation change in a very significant way; they now contain a ServerConnectionHandle value that is derived from the preceding Bind v12 operation.

    - RWS: 0x3 Connect v12 request from nslookup.exe to 4.2.2.2/53; client will send from 10.10.255.129/1206
  - RwstPacket: Connect v12 request from nslookup.exe to 4.2.2.2/53; client will send from 10.10.255.129/1206
     NullChar: 0 (0x0)
     ProtoSig: RWS
     PktLen: 281 (0x119)
     Reserved1: 0 (0x0)
     Flags: 0 (0x0)
     Reserved2: 0 (0x0)
     OpCode: Connect v12
     DataLength: 0 (0x0)
   + ClientConnectionHandle: 0x3
   + ServerConnectionHandle: 0x101a1
   + ClientMappingHandle: 0x0
   + ServerMappingHandle: 0x0
   - RwsMessage: request from nslookup.exe to 4.2.2.2/53; client will send from 10.10.255.129/1206
    - Connect12Data: request from nslookup.exe to 4.2.2.2/53; client will send from 10.10.255.129/1206
     - ConnectEndpoint: 4.2.2.2/53
      - Socket: 4.2.2.2/53
         Family: IPv4
         Port: 53 (0x35)
         Addr: 4.2.2.2
         Padding: Binary Large Object (8 Bytes)
        Padding: Binary Large Object (12 Bytes)
     - ClientEndpoint: Dynamic/1206
      - Socket: Dynamic/1206
         Family: IPv4
         Port: 1206 (0x4B6)
         Addr: 0.0.0.0
         Padding: Binary Large Object (8 Bytes)
        Padding: Binary Large Object (12 Bytes)
       Padding: Binary Large Object (44 Bytes)
     + AppAndClientInfo: nslookup.exe

    • The inclusion of the ServerConnectionHandle in the Connect v12 request at frame 22 tells Forefront TMG that the connection to be made to 4.2.2.2 on port 53 should be performed in the context of the socket created by the UDP Bind v12 operation that was completed in frame 21.

      Note

      Unlike TCP, UDP does not include the concept of a “connection”. Thus, this is only a logical connection that will not actually produce any packets between Forefront TMG and the IP address specified in the ConnectEndpoint struct.

One point worth reiterating for this example is the use of the Conversation.RwsId property. In this example, we can see that nslookup.exe created, used and destroyed two separate traffic paths between itself and the DNS server at 4.2.2.2 in order to make two separate DNS queries. Since the query of interest in this example is for isatools.org, we can see from Figure 2 that the RWS ID for that query is 0x3. If we apply the filter as shown below, we will see only the RWS messages that helped to establish the traffic path for that DNS query.

Selective RWS ID filter

Figure 3 - Selective RWS ID filter for isatools.org query

Thus, we can reasonably surmise that unlike TCP traffic, UDP traffic sent via the Forefront TMG Client must be preceded with (at the very least) successful RWS Bind v12 and Connect v12 operations. If name resolution were part of the process, a Get host by name and Host entry exchange would also be included in the control channel conversation.

Deny Rule Action

In this example, the Forefront TMG policy does not allow the Forefront TMG Client to exchange DNS traffic with remote hosts. Figure 4 illustrates this process.

fig4

Figure 4 - Deny rule

You may have noticed that the display filter fails to include “wsp”, but if you were to select any of the frames that include “DNS” in the frame summary window, you will find that that they are indeed “DNS traveling by way of WSP”. Figure 5 illustrates this effect for frame 28.

fig5

Figure 5 - DNS on WSP

One of the benefits of Network Monitor 3.3 is that a display filter will include frames that include the desired protocol no matter where it exists in the protocol stack for that frame. The advantage to using this filter is that it allows us to see any DNS traffic that may have occurred between Forefront TMG and the DNS server, as well as the DNS traffic that passed between the Forefront TMG Client and Forefront TMG. By using this display filter, we manage to keep the bird-to-stone ration within acceptable limits.

What we observe from this capture is that there are three separate instances where the Forefront TMG Client:

  1. Successfully negotiates a Bind v12 process.

  2. Successfully negotiates a Connect v12 process.

  3. Sends a DNS query.

  4. Receives no response.

  5. Closes the WSP session.

The first of these cycles is an attempt by nslookup.exe to resolve the DNS server address (4.2.2.2) to a name. The remaining two cycles represent nslookup.exe trying, and then retrying to resolve the name “isatools.org” to an IP address; both of which fail to receive a response.

Because there is no DNS traffic indicated between the Forefront TMG computer and the DNS server at 4.2.2.2, we can conclude that Forefront TMG:

  1. Never received the DNS traffic from the Forefront TMG Client.

  2. Chose to block the DNS traffic from the Forefront TMG Client.

  3. Failed to send the DNS traffic to the DNS server.

The only way to know the difference between these possibilities is to examine the Forefront TMG logs. Because we’re trying to view log data for events that occurred in the very recent past, our log query will be defined as shown in Figure 6.

fig6

Figure 6 - DNS log filter

The results derived from this query are shown in Figure 7.

fig7

Figure 7 - DNS deny log results

You can see from the log query results that Forefront TMG actively denied this traffic, and then closed the session where it was received. Because UDP has no traffic management mechanisms as found in TCP traffic (SYN, ACK, RST, FIN, etc.), there is no way for TMG to indicate to the application that the traffic was not allowed. In fact, if Forefront TMG were not involved in this traffic, nslookup.exe would experience the same behavior; no response to the DNS queries, so the application and thus the user experience is consistent in this case.

One point in the log query results that is questioned from time to time is the inclusion of the “(?)” after the user name for Forefront TMG Client-managed traffic. This indicates that although Forefront TMG was provided with the user context, this information was not validated. When the Forefront TMG policies do not require authentication, the user will always be represented this way for Forefront TMG Client-managed traffic.

Name Resolution Failure

Because nslookup.exe does not call Winsock getaddrinfo(), it will never generate a Forefront TMG Client Get host by name cycle. Regardless, the results of such a failure do not change between TCP- or UDP-based traffic. If the client application cannot resolve the name to an IP address, the application will not generate any traffic through Forefront TMG. Part 2 of this series already describes how this would appear; I only included this section to keep the format consistent, so I won’t waste any more of your time with this.

Connection Failure

In the case of UDP traffic, there is no “connection” other than the logical state maintained by the client, server, and of course, Forefront TMG and Forefront TMG Client. Thus, UDP “connection failures” can best be described as a failure of the client to receive a response from the server. Figure 8 illustrates the frame summary for such an event.

fig8

Figure 8 - RWS or DNS

This conversation set looks almost identical to the process described in the Successful scenario example with two differences:

  1. The DNS queries sent from Forefront TMG to the DNS server at 10.10.10.10 elicit no response.

  2. The Bind v12, Connect v12 and Close cycles are repeated as in the Deny Rule Action example.

Because we can see Forefront TMG sending the traffic in frames 13, 22 and 29, but no DNS responses are seen by Network Monitor 3.3 at Forefront TMG, we can reasonably conclude that whatever is preventing the traffic from reaching the Forefront TMG Client, it very likely is not Forefront TMG or the Forefront TMG Client.

Summary

In this part of the series of articles, I discussed and illustrated the RWS and WSP protocol behavior for success and failure states for single-connection UDP traffic using DNS. Part 1 of this series provides an instruction to Network Monitor 3.3 and the RWS parser. Part 2 of this series discusses success and failure states for single-connection TCP traffic using HTTP as the application protocol.

I hope this series has been as useful to you as it has been fun for me to assemble. Many thanks to Yigal Edery for helping me take this mechanism public!

Tools and Techniques

References