Requests to IIS with SSLCertNegotiate and TLS 1.3 is failing.

Question

We have an IIS server setup on Windows Server 2022, TLS 1.3 enabled and with the "SSL Certificate" setting set to SSLCertificateNegotiate. Our site supports optional certificate-based authentication.

What we have observed is that clients which communicate using TLS 1.3 but are NOT using the certificate-based authentication (as it is optional) fail to send requests to our site. In the "Client Hello" messages from the network traces, we do not observe any "post_handshake_auth" extension mentioned in the TCP packet.
We see "Request Cancelled" errors logged in the IIS logs and the network traces show "Connection Reset" from server.

This seems to be an issue exclusively with Windows Server 2022 and TLS1.3.

With TLS1.2 everything works fine.

Is this a known issue with Windows Server 2022 / IIS / TLS1.3?

Answer 1

@Kedar Sane

Microsoft implemented TLS 1.3 in most secure way by RFC. IIS wants to perform post-handshake authentication, but some browsers do not support it in default configuration, you can try using other browsers (Firefox is recommended), so, de facto IIS default configuration for two-way SSL with common browsers do not work with IIS when TLS 1.3 only is enabled.

You can enable IIS and TLS 1.3 only configuration by enabling in-handshake method for IIS instead on post-handshake method.

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.