Reassembly Made Easier

Article
11/04/2010

By using our latest 3.4.2455 release of the parsers and using a simple filter, you can now view reassembled traffic more easily for certain protocols. Normally when you reassemble a trace you see all the original frames plus the newly inserted reassembled frames. Using a filter with a brand new property, you can now see only the complete frames; whether or not they were fragmented to begin with.

Overview of Reassembly

For a full explanation of Reassembly, you can check out our Reassembly Blog. In short, the resulting reassembled view of the data contains both the original frames and fragments, and the newly inserted frames for each protocol that might fragment its payload. For instance, all the TCP frames that have been fragmented are reassembled and inserted into a new frame in the trace with a special header, PayloadHeader. Additionally if those reassembled TCP frames are also fragments at a higher layer protocol, like HTTP, new frames are also inserted for those. The result is Frame Summaries that seem to show as duplicates, but they are similar versions with a complete payload.

New Properties for Filtering

If you are interested in complete frames for a specific layer, it would be convenient to filter out all the fragments. To make this possible, we’ve added new properties into the parsers which tag frames that are complete for that protocol. By filtering on this property you can see all the complete frames for the layer in question. We have enabled this property for TCP, NBTSS, and HTTP. NBTSS is important because SMB uses it as a transport.

An Example of HTTP Fragmentation

If this first example, I’ll filter on HTTP traffic before it’s reassembled. Notice the last shaded frame below, it is a payload and thus an HTTP fragment of the frame above it. To see the entire HTTP frame we need to reassemble this trace.

HTTP:Request, GET /ads/7945/0000007945_000000000000000607478.gif

HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607478.gif - GIF: Version=GIF89a, Width=300, Length=250

HTTP:Request, GET /ads/7945/0000007945_000000000000000607449.gif

HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607449.gif - GIF: Version=GIF89a, Width=300, Length=250

HTTP:HTTP Payload, URL: /ads/7945/0000007945_000000000000000607449.gif - GIF: Version=

tÀ„‑, Width=8195, Length=49568

After reassembly, we filter on HTTP. Now you see the two sets of original traffic paired with their inserted responses (shaded in two colors below). The newly inserted responses are also bolded.

Summary Description	Notes
HTTP:Request, GET /ads/7945/0000007945_000000000000000607478.gif	Original Frame
HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607478.gif - GIF: Version=GIF89a, Width=300, Length=250	Original Frame
HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607478.gif - GIF: Version=GIF89a, Width=300, Length=250	Inserted TCP frame from TCP Fragments
HTTP:Request, GET /ads/7945/0000007945_000000000000000607449.gif	Original Frame
HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607449.gif - GIF: Version=GIF89a, Width=300, Length=250	Original Frame
HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607449.gif - GIF: Version=GIF89a, Width=300, Length=250	Inserted TCP Frame from TCP Fragments. Also the start of an HTTP Fragment
HTTP:HTTP Payload, URL: /ads/7945/0000007945_000000000000000607449.gif	Original Frame, HTTP End Fragment
HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607449.gif - GIF: Version=GIF89a, Width=300, Length=250	Inserted HTTP Fragment

As you can see this is confusing because each bolded inserted frame is actually a duplicate. There are 3 inserted frames because of fragmentation at both the TCP and HTTP layers. Due to this the final response is there two extra times: the first time because TCP has fragmented the data and the second because HTTP has also fragmented its payload.

Now perhaps you might think that I could simply filter out only those frames that have been reassembled. In the table below I’ve done that using the filter “PayloadHeader”

HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607478.gif - GIF: Version=GIF89a, Width=300, Length=250

HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607449.gif - GIF: Version=GIF89a, Width=300, Length=250

And you can see this isn’t helpful because not all frames are fragmented. So now let’s use the new Property and apply a filter:

Property.HTTPCompleteFrame == 1

HTTP:Request, GET /ads/7945/0000007945_000000000000000607478.gif

HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607478.gif - GIF: Version=GIF89a, Width=300, Length=250

HTTP:Request, GET /ads/7945/0000007945_000000000000000607449.gif

HTTP:Response, HTTP/1.1, Status: Ok, URL: /ads/7945/0000007945_000000000000000607449.gif - GIF: Version=GIF89a, Width=300, Length=250

And Voila! We now see only complete HTTP frames and the trace we actually wanted.

To dive into how this works. The parser marks each complete frame as it reads it. Since the Requests were never fragmented, they’re marked as complete and display. The Responses were fragmented and in the case of the last frame, only the last HTTP reassembled frame is considered a complete HTTP frame. The corresponding TCP reassembled frame is not displayed because from HTTP’s perspective, it wasn’t complete.

NBTSSCompleteFrame Property

We have enabled the same functionality for NBTSS. Since SMB relies on this as a transport, this makes viewing SMB traffic over NBTBSS also display in the same way as the HTTP example above.

TCP Fragmentation

In some cases a protocol is fragmented by TCP, but no further fragmentation occurs in the protocol layers above. In this case we don’t know what a complete frame is at the protocol layer, but you can use a property at the TCP layer to find all completed TCP frames. When you add this to those frames that have a payload header, this will provide you with a list of all complete TCP frames or frames which were originally fragmented. This filter would look like this:

Property.CompleteFrame == 1 OR PayloadHeader

For instance, for SMB2 traffic over TCP, this does a decent job of showing you the traffic you. The only caveat here is that since SMBOverTCP can fragment data, you see those extra fragments. Still the results are much easier to look at that the unfiltered reassembled trace.

Here’s an example of a reassembled SMB2 trace where you can see the original READ plus the inserted reassembled frame.

SMB2:C READ (0x8), FID=0xFFFFFFFF00000041 (dataWebGuide4_4109-VISTA.exe@#4848) , 0x7600 bytes from offset 4296704 (0x419000)

SMB2:R READ (0x8), File=dataWebGuide4_4109-VISTA.exe@#4848, 0x7600 bytes read

SMB2:C READ (0x8), FID=0xFFFFFFFF00000041 (dataWebGuide4_4109-VISTA.exe@#4848) , 0x8000 bytes from offset 32768 (0x8000)

SMB2:R READ (0x8), File=dataWebGuide4_4109-VISTA.exe@#4848, 0x8000 bytes read

SMB2:C READ (0x8), FID=0xFFFFFFFF00000041 (dataWebGuide4_4109-VISTA.exe@#4848) , 0x8000 bytes from offset 73728 (0x12000)

SMB2:R READ (0x8), File=dataWebGuide4_4109-VISTA.exe@#4848, 0x8000 bytes read

SMB2:C READ (0x8), FID=0xFFFFFFFF00000041 (dataWebGuide4_4109-VISTA.exe@#4848) , 0x2000 bytes from offset 65536 (0x10000)

SMB2:R READ (0x8), File=dataWebGuide4_4109-VISTA.exe@#4848, 0x2000 bytes read

Now using the above filter you see all the SMB Read fragments disappear.