Upgrading machine to .Net framework 4.5, WCF service built on .Net Framework 4.0 using more than one NetTcpBinding fails with AddressAlreadyInUseException
Recently, we came across the scenario where customer had a WCF service built on .Net Framework 4.0. This WCF service has couple of endpoints using TCP bindings with configuration as follows,
<endpointaddress="CalcService" binding="netTcpBinding" bindingConfiguration="nettcp"contract="Service.ICalcService"/>
<endpointaddress="OrderService" binding="netTcpBinding" contract="Service.IOrderService"/>
Customer had requirement to upgrade (install) their server machines to .Net Framework 4.5. After installing .Net framework 4.5 on machines, customer noticed that WCF service failed to activate with following exception,
Unhandled Exception: System.ServiceModel.AddressAlreadyInUseException: There is already a listener on IP endpoint 0.0.0.0:8080. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service host with the same IP endpoint but with incompatible binding configurations. ---> System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted.
Cause: Above exception can occur if there’s mismatch in binding configuration for 2 different endpoints using TCP bindings and they cannot share the sample port (see Here). However, looking at the binding configuration applied on endpoints above, they look pretty similar! So, we enabled the WCF diagnostics tracing at verbose level and it indeed showed that transport manager is found incompatible on opening 2nd endpoint, which means 2 endpoints cannot share same port,
Looking at binding configuration for above 2 endpoints closely again, we see that one of the endpoint has nettcp binding configuration applied which sets parameter maxConnections = 10.
One of the changes in .Net framework 4.5 brings in change in MaxConnection knobs, where using the default bindings, some of the defaults are calculated using OSEnvironmentHelper.ProcessorCount instead of hard coded default values as they were in .Net 4.0.
In this case, when using a named netTcpBinding bindingConfiguration ("nettcp"), the value supplied for the MaxConnections property is 10. (Setting the MaxConnections property on the NetTcpBinding also sets it's TcpTransportBindingElement's MaxPendingConnections property and the TcpTransport's ConnectionPoolSettings.MaxOutboundConnectionsPerEndpoint property to the same value.)
In case of OrderService endpoint, it’s using NetTcpBinding with default values, so the MaxPendingConnections property was calculated by using the following algorithm: return (12 * OSEnvironmentHelper.ProcessorCount);
So, When using a named netTcpBinding bindingConfiguration for CalcService endpoint, the transport's MaxPendingConnections was 10, and for OrderService endpoint, transport's MaxPendingConnections was 12*CPU count. The difference in values for the MaxPendingConnections between these two endpoints causes incompatible transport manager and 2 endpoints cannot share the same port. Same issue could have also been seen if there’s a mismatch in ListenBacklog parameter. Also, in my situation, customer has 2 different endpoints using netTcpBindings. The issue can also happen if there's mex endpoint using maxTcpBinding with default values, which is very common setup.
Application worked in .Net Framework 4, because both bindings resolved to same MaxConnection values i.e. 10. nettcp binding configuration sets MexConnections = 10 and maxTcpBinding didn’t override this value which is by default 10. Thus, there’s no conflict and both endpoint could share the same port.
Solutions: There’re multiple solutions available to resolve the issue and choose per your requirement:
a) If you want to have a limit on MaxPendingConnections and ListenBacklog on first endpoint, then change the second endpoint to listen on a different base address by specifying the fully qualified URL using different port than used in based address, in endpoint address. For example, to resolve issue in my customer’s case, the changes would look as follows for second endpoint,
b) Alternatively, use the same binding configuration for Mex endpoint by changing is binding to NetTcpBinding. Again, this option should be followed when you want to have a limit on MaxPendingConnections and ListenBacklog now for both endpoints,
c) If you don’t want to have a limit on MaxConnections and ListenBacklog, then simply remove these parameters from binding configuration so that both named NetTcpBinding uses default values for maxConnections and ListenBacklog and there’s no mismatch.
I hope this helps to save someone’s time!