How to write chunked transfer encoding web response
Recently I worked on a performance issue caused by IIS re-authenticate for every request. From network monitor log, it was due to poor designed of chunked encoding of a CGI and then IIS close HTTP Connection according HTTP 1.1 protocol, details below.
This issue was resolved after we fixed customer’s web application which composes HTTP Response with chunked transfer encoding. I will provide sample for sharing.
Based on HTTP protocol, (i.e.https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4)
Server will close the connection to determine the transfer-length of that body for HTTP respond if neither “Content-Length” nor “Transfer-Encoding: chunked” is present.
In IIS, if either of them are not present, then IIS adds the “Connection: close” to the response headers. This behavior conforms to above HTTP 1.1 protocol.
For web application hosted in IIS, the chunking behavior completely depends on the Application Framework you are using. There is no IIS configuration to either enable or disable chunked-encoding.
For example, ASP is an Application Framework running on IIS that has its own metabase properties (i.e. AspEnableChunkedEncoding in kb278998) to control chunking. But that is ASP-specific; there is no IIS "behavior" to enable chunking for arbitrary responses.
How to enable chunked transfer encoding with IIS
https://support.microsoft.com/kb/278998
Assume there is a CGI application running with integrated authentication on IIS, to avoid to be re-authenticated for each request, it is important to understand Chunked Transfer Encoding described in below article.
https://en.wikipedia.org/wiki/Chunked_transfer_encoding
https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1
Let me provide sample code for example:
#include <stdio.h> int main() { printf("%s", "Transfer-Encoding: chunked\r\n"); printf("%s", "\r\n"); ‘Each chunk starts with the number of octets of the data it embeds expressed in hexadecimal followed by optional parameters (chunk extension) and a terminating CRLF (carriage return and line feed) sequence, followed by the chunk data. ‘chunk 1, the length of data is 46 printf("%s\n", "2e"); printf("%s", "<html>"); printf("%s", "This is the data in the first chunk<br/>\n"); ‘chunk 2, the length of data is 26 printf("%s\n", "1a"); printf("%s", "And this is the second one\n"); ‘chunk 3 printf("%s\n", "28"); printf("%s","<p>This is some text in a paragraph.</p>\n"); ‘chunk 4 printf("%s\n", "7"); printf("%s", "</html>\n"); ‘The last chunk is a zero-length chunk, with the chunk size coded as 0, but without any chunk data section. printf("%s\n", "0"); printf("%s", "\n"); return 0; } |
Compile above code to generate text.exe and then configure IIS to run this CGI application
Configuring CGI Settings in IIS 7
https://technet.microsoft.com/en-us/library/cc731821(WS.10).aspx
The web page display like below in IE:
This is the data in the first chunkAnd this is the second one This is some text in a paragraph. |
We focus on the network http response as below, compared with our code to get more information.
- Http: Response, HTTP/1.1, Status: Ok, URL: /test.exe ProtocolVersion: HTTP/1.1 StatusCode: 200, Ok Reason: OK TransferEncoding: chunked Server: Microsoft-IIS/7.0 XPoweredBy: ASP.NET Date: Fri, 18 Feb 2011 06:14:40 GMT HeaderEnd: CRLF - chunkSize: 46 Size: 46 - ChunkPayload: HttpContentType = NetmonNull HTTPPayloadLine: <html>This is the data in the first chunk<br/> FooterEnd: CRLF - chunkSize: 26 Size: 26 ChunkPayloadContinuation: Binary Large Object (26 Bytes) FooterEnd: CRLF - chunkSize: 40 Size: 40 ChunkPayloadContinuation: Binary Large Object (40 Bytes) FooterEnd: CRLF - chunkSize: 7 Size: 7 ChunkPayloadContinuation: Binary Large Object (7 Bytes) FooterEnd: CRLF ChunkEnd: 0 FooterEnd: CRLF |
If the chunked code is not formatted promptly, “Page Cannot be displayed” will display in IE
If either TransferEncoding or Content-length isn’t present, you will find “Connection: Close” in HTTP response.
Enjoy!
Anik - Microsoft APGC DSI Team