The underlying connection was closed...

Article
11/29/2005

Please use the following KB article if you get any "Underlying connection was closed" error.

915599 You receive one or more error messages when you try to make an HTTP request in an application that is built on the .NET Framework 1.1 Service Pack 1
https://support.microsoft.com/default.aspx?scid=kb;EN-US;915599

********* Content below will not be updated for any new/additional info, but the KB article will be ****** So please use the KB article **********

Note: This discussion applies to .Net framework 1.1 and .Net framework 1.1 SP1 only.

The above exception is usually reported by System.Net.HttpWebRequest class (which may bubble up to System.Net.WebClient or System.Web.Services.Protocols.SoapHttpClientProtocol).

Working with Microsoft customers, I know how they hate the above message. But this error is actually very helpful in determining what went wrong with the underlying connection, during HTTP request/response communication sequence between client and server processes.

Types of 'Underlying connection closed' Errors:

We typically see one of the following. The second part of the error indicates what may be the cause.

"The underlying connection was closed: Unable to connect to the remote server."
--Seen when the webserver to which the HTTP request was sent, was unreachable. Usually caused by incorrect proxy configuration and/or by dns issues.

"The underlying connection was closed: An unexpected error occurred on a receive." --Seen when the client had sent the request in its entirety and got a TCP ACK-FIN or RST from server to close the connection, without a response from server.

"The underlying connection was closed: An unexpected error occurred on a Send." --Seen while the client is sending the request the underlying TCP connection was closed.

"The underlying connection was closed: The server committed an HTTP protocol violation." --Seen when the HTTP response from the server was deemed incorrect or not conforming to the guidelines specified in HTTP 1.1 RFC (https://www.w3.org/Protocols/rfc2616/rfc2616.html). May occur if the .Net client considers the response to be a security threat, as well.

"The underlying connection was closed: Could not establish secure channel for SSL/TLS."
--Seen when the client could not successfully establish SSL/TLS channel, for HTTPS request. Usually caused by invalid/expired client or server SSL certificates.

"The operation has timed-out" --Seen when the client timeout value expired before the server response came in. Though this is not an underlying connection closed error, the following suggestions help in addressing this.

Please refer https://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemnetwebexceptionstatusclasstopic.asp for determining the status when getting a WebException.

The following are also very helpful in effectively resolving or working around 'Underlying connection closed' errors.

1. Ensure your .Net framework is up to date:
==================================

Typically applying SP1 for .Net framework 1.1, which is included in Windows 2003 Server SP1, should resolve most known bugs/issues with incorrect TCP behaviour. If your application is using SSL for the HTTP request, I strongly recommend applying SP1 for .Net framework (or on a Windows 2003 Server, you may apply SP1 for the OS).

Once you apply, confirm that SP1 for .Net framework is correctly installed. To do this, you can check the registry key as per the following KB article.
318785 Determine whether service packs are installed on the .NET Framework
https://support.microsoft.com/?id=318785

Doing this may address:
--------------------------------
All the above exceptions

2. Ensure that HttpWebRequest settings are appropriate for the requests issued: ===============================================================
Note that the default connections recommended for a HTTP 1.1 client is 2 and that is the default with HttpWebRequest. If your application requires more than two concurrent connections then you will have to increase the number of concurrent connections HttpWebRequest can use. If
(a) number of connections is not increased,
(b) both concurrent connections are busy and
(c) if there are pending requests which do not get a chance to use a connection,
then the pending requests may get a "The operation has timed-out" exception.

Note: If you are making a request from ASP.Net code, since the ASP.Net application is a service and may have to have more than 2 connections for handling incoming requests, increasing concurrent connections for HttpWebRequest is recommended.

Note: The webservice reference proxy code, generated by Visual Studio WSDL.exe tool (which is used when you add a webreference to a C# or VB.Net project) uses System.Net.HttpWebRequest to make the HTTP requests. So if you are seeing underlying connection closed errors on a webservice call, you will have to confirm that your webservice client code/configuration is tuned as for System.Net.HttpWebRequest.

To increase number of concurrent connections, increase ServicePointManager.DefaultConnectionLimit property starting at 12 per processor as per the following MSDN article:
https://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemNetServicePointManagerClassDefaultConnectionLimitTopic.asp

Note: Please ensure that any change in default values for ServicePointManager properties is done before any HTTP request is sent out.

If you want to set it in a config file, and not through code, use the <connectionManagement> Element as per the following MSDN article:
https://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpgenref/html/gngrfconnectionmanagementelement.asp
The <connectionManagement> element defines the maximum number of connections to a server or group of servers.

Doing this may address:
---------------------------------
"The operation has timed-out" exception

3. KeepAlive connection issues:
=========================
We could see the problem if the Keep Alive connection is not correctly handled by .Net client process or the WebServer.
Note: KeepAlive is required for NTLM and Kerberos authentication. So if you are NOT using NTLM or Kerberos as authentication and NOT require Keep Alive for the connection, you can try setting KeepAlive to false.

For setting Timeout in HttpWebRequest object: https://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemnethttpwebrequestclasstimeouttopic.asp
For setting Timeout from a webservice client:
1. Create a new class by inheriting from the generated proxy class.
2. Add the following method to override GetWebRequest method to access the underlying HttpWebRequest object.
C#:
protected override WebRequest GetWebRequest(Uri uri)
{
HttpWebRequest webRequest = (HttpWebRequest) base.GetWebRequest(uri);
//Setting KeepAlive to false
webRequest.KeepAlive = false;
return webRequest;
}

VB.Net:
Protected Overrides Function GetWebRequest(ByVal uri As Uri) As System.Net.WebRequest
Dim webRequest As System.Net.HttpWebRequest
webRequest = CType(MyBase.GetWebRequest(uri), System.Net.HttpWebRequest)
'Setting KeepAlive to false
webRequest.KeepAlive = False
GetWebRequest = webRequest
End Function

Doing this may address:
----------------------------------
"The underlying connection was closed: An unexpected error occurred on a Send." and "The underlying connection was closed: An unexpected error occurred on a receive." exceptions

4. For closing a KeepAlive connection before server times out:
===================================================
If the server KeepAlive connection timeout is close to the client KeepAlive connection timeout, and if the client is sending a request on the connection almost at the end of timeout duration, then we may see "The underlying connection was closed: An unexpected error occurred on a Send." and "The underlying connection was closed: An unexpected error occurred on a receive." errors.

For this problem, we can try reducing the duration at which the client will actively purge the existing unused connection through ServicePointManager.MaxServicePointIdleTime Property.
Please refer https://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemnetservicepointmanagerclassmaxservicepointidletimetopic.asp.

Please try setting this value less than the server KeepAlive connection timeout.

Note: Please ensure that any change in default values for ServicePointManager properties is done before any HTTP request is sent out.

Doing this may address:
---------------------------------
"The underlying connection was closed: An unexpected error occurred on a Send." and "The underlying connection was closed: An unexpected error occurred on a receive." exceptions

5. Setting ASP.Net executionTimeout if client code is in ASP.Net:
==================================================
If your client code is running in ASP.Net, please make sure that your timeouts are implemented as specified below. The ASP.Net process executionTimeout and the webservice client call timeout should be set as per the following article.

904262 The request that is sent by the HttpWebRequest class may stop responding
https://support.microsoft.com/?id=904262

Ensure that your ASP.Net webservice client timeout MUST be higher than the maximum time taken by the webservice to respond for longest running request.

Doing this may address:
--------------------------------
"The operation has timed-out" and "The underlying connection was closed: An unexpected error occurred on a receive." exceptions

6. Ensure proxy settings are correct:
===========================

You can use the static method WebProxy.GetDefaultProxy as per https://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemnetwebproxyclassgetdefaultproxytopic.asp, or you can use https://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpgenref/html/gngrfproxyelement.asp for defining <proxy> element in .config files.

If your environment is configured for autoproxy, refer the following.
873199 How to use autoproxy in managed code
https://support.microsoft.com/?id=873199

Doing this may address:
--------------------------------
"The underlying connection was closed: Unable to connect to the remote server." exception

7. Ensure that the server (where client code is running) does not have protocol binding limitations issue: ================================================================================

826757 FIX: Socket Initialization Does Not Succeed if Your Computer Has More
https://support.microsoft.com/?id=826757

This fix is included in .Net framework 1.1 SP1 and also in Windows 2003 SP1. If this is the cause, any System.Net socket application will fail.

Doing this may address:
---------------------------------
"The operation has timed-out" and "The underlying connection was closed: Unable to connect to the remote server." exceptions

8. Problems in establishing SSL/TLS channels:
====================================
First verify that the server and client (if any) SSL certificates are valid, not expired and can be used from browser without any problems.

The following explain how to use SSL with client certificates with .Net framework.

901183 How to call a Web service by using a client certificate for
https://support.microsoft.com/?id=901183

https://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnnetsec/html/SecNetHT13.asp

https://msdn.microsoft.com/library/en-us/dnnetsec/html/SecNetHT14.asp?frame=true

895971 How to send a client certificate by using the HttpWebRequest and
https://support.microsoft.com/?id=895971

If the problem is caused by server SSL certificate (expired, untrusted CA or common name mismatch) and if the server is trusted, the errors can be ignored by using ICertificatePolicy.CheckValidationResult method (https://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemNetICertificatePolicyClassCheckValidationResultTopic.asp). Please note that ignoring SSL certificate warnings in code is a possible security threat, unless the server is completely trusted.

Doing this may address:
----------------------------------
"The underlying connection was closed: Could not establish secure channel for SSL/TLS." exception

9. Server response not conforming to HTTP RFC: ======================================
.Net framework 1.1 HTTP clients strictly follow HTTP 1.1 RFC guidelines in https://www.w3.org/Protocols/rfc2616/rfc2616.html. If the server response to client does not fully comply with HTTP 1.1 RFC, the client will close the connection with the "The underlying connection was closed: The server committed an HTTP protocol violation." exception. Usually this happens with incorrect headers and header delimiters (https://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html#sec6). The above exception is often seen after applying SP1 for .Net framework 1.1.

Any response not adhering with HTTP 1.1 RFC guideline, is considered as a security threat by .Net framework client, as allowing malformed headers may expose the client for "HTTP response splitting" attacks.

If the server is trusted completely, then the above exception can be worked around by following the KB article below, at the risk of compromising security. Please note that the best course of action for this exception would be to correct the server side application/webserver to send correct responses as per HTTP 1.1 RFC.

892467 HttpWebRequest.GetResponse() returns "HTTP protocol violation" error
https://support.microsoft.com/?id=892467

Doing this may address:
-----------------------------------
"The underlying connection was closed: The server committed an HTTP protocol violation." exception

The underlying connection was closed...

Additional resources