SSL Handshake and HTTPS Bindings on IIS

Secure Socket Layer (SSL) also known as Transport Layer Security (TLS) is a cryptographic protocol which defines how 2 entities (client and server) communicate with each other securely. TLS is the successor of SSL. You can read more about it here:

These are the following protocols which are most commonly used:

  • SSL 2.0
  • SSL 3.0
  • TLS 1.0 (SSL 3.1)
  • TLS 1.1 (SSL 3.2)
  • TLS 1.2 (SSL 3.3)

SSL 2.0 had many security flaws which led to the development of its successor SSL 3.0. It is present only for backward compatibility. I have rarely seen anyone using this version and I would highly recommend against it.

As we know TLS/SSL is an application layer protocol. Below is a diagram depicting the TCP/IP model:

I am not going to discuss the SSL/TLS protocol in this post as it is beyond the scope of this topic. However I would be discussing SSL handshake in brief and relate it to IIS.

The above diagram makes it clear that TLS/SSL runs on top of TCP/IP like any other application layer protocol. Before we delve into SSL handshake we need to know something about TCP handshake too.

TCP/IP Handshake

Microsoft has published a support article explaining the 3-way TCP/IP handshake. Here is the link:

Below diagram should give you a gist of the TCP/IP handshake:

If we were to capture a network trace (or a TCP Dump) and look at the details available and analyze the details available; the IP Layer provides the TCP layer with IP Address of the client and server. The TCP layer contains the details about the source port and the destination port, TCP Flags and other details like checksum, Windows Size etc.

When the user launches a browser and punches in the web address, let's say, the client and the server would perform the TCP/IP handshake as seen below

So basically this is what is passed on from the TCP/IP layer to the application layer:

  • IP Address of the source and destination
  • Source Port and Destination Port

The host header is neither present in the IP or the TCP layer. This actually leads to a problem which was addressed via the introduction of SERVER NAME INDICATION (a TLS Extension).

Problem due to above Limitation

Before I describe the problem we need to understand a little about the server side bindings. When routing a HTTP request to a website the server determines which process the request to be routed based on the IP, PORT & the HOSTNAME. These 3 are always available to the server during a normal HTTP communication. So basically the combination of IP+PORT+HOSTNAME is used as a unique identity to route the site to a specific process. The server admin can have the same IP+PORT for all the HTTP websites and alter only the HOSTNAME and maintain the uniqueness throughout. Which also makes the server scalable.

However in case of SSL the server has access to IP & Port only. Since the HOSTNAME is not available, the server has to route the request to the process depending on IP+PORT. This limitation leaves the server handicapped, as it has to changes the design for websites running on HTTPS. Due to this, the uniqueness for the websites running on HTTPS is determined through combination of IP+PORT. In real world, having a separate IP for a website is not ideal due to hardware & monetary limitations. Also changing the port number for all SSL bindings may not be ideal as changing the port number to anything other than the default SSL port would require the client to specifically put out the port number in the request. As a result, the server is not scalable for HTTPS sites.

This was a protocol limitation and severely affected the scalability of the sites

This problem was addressed by introducing a TLS Extension called Server name Indication. The client sends the server the hostname it is requesting for as a part of the CLIENT HELLO in the form of TLS EXTENSIONS. You can read more about it here: RFC 3546 (Section 3.1)

TLS/SSL Handshake

Let's consider a scenario where the client launches the browser and punches in

  • Client will try to resolve the hostname to an IPAddress via DNS.

  • Once the client has the Destination IP, it will send a TCP SYN to the server.

  • The Server responds with ACK to this SYN.

  • The client responds with an ACK to the ACK it received from the server. Now a TCP connection has been established between the client and the server. The client will now forward the requests to the Destination IP on port 443 (Default TLS/SSL port)

  • The control is now transferred to the SSL Protocol in the application layer. It has the IP & the Port information handy from previous steps. However, it still has no clue whatsoever about the hostname.

  • The client creates a TLS Packet called as CLIENT HELLO. This contains the following details:

    • SSL Protocol version
    • Session ID
    • List of Cipher Suites supported by the client.
    • List of CLIENT HELLO Extensions

    The Client typically selects the most secure protocol version and sends it to the server. Below is a snippet from the RFC 3546:

Blake-Wilson, et. al.            Standards Track                       [Page 4]RFC 3546                         TLS Extensions                       June 2003

2.1. Extended Client Hello

Clients MAY request extended functionality from servers by sending the extended client hello message format in place of the client hello message format. The extended client hello message format is:

struct {        ProtocolVersion client_version;         Random random;        SessionID session_id;        CipherSuite cipher_suites<2..2^16-1>;        CompressionMethod compression_methods<1..2^8-1>;        Extension client_hello_extension_list<0..2^16-1>;} ClientHello;

Here the new "client_hello_extension_list" field contains a list of extensions. The actual "Extension" format is defined in Section 2.3.

In the event that a client requests additional functionality using  the extended client hello, and this functionality is not supplied by the server, the client MAY abort the handshake.

Note that [TLS], Section, allows additional information to be added to the client hello message.  Thus the use of the extended client hello defined above should not "break" existing TLS 1.0 servers.

 A server that supports the extensions mechanism MUST accept only client hello messages in either the original or extended ClientHello format, and (as for all other messages) MUST check that the amount of data in the message precisely matches one of these formats; if not then it MUST send a fatal "decode_error" alert.  This overrides the "Forward compatibility note" in [TLS].

 If you were to capture a network trace (or a TCP Dump) this is how the CLIENT HELLO would look like:

Frame 310: 187 bytes on wire (1496 bits), 187 bytes captured (1496 bits) on interface 0Ethernet II, Src: WistronI_86:74:54 (3c:97:0e:86:74:54), Dst: Cisco_e5:44:00 (10:bd:18:e5:44:00)Internet Protocol Version 4, Src: (, Dst: ( Control Protocol, Src Port: 42079 (42079), Dst Port: http (80), Seq: 226, Ack: 116, Len: 133Secure Sockets Layer    TLSv1 Record Layer: Handshake Protocol: Client Hello        Content Type: Handshake (22)        Version: TLS 1.0 (0x0301)        Length: 128 Handshake Protocol: Client Hello            Handshake Type: Client Hello (1)            Length: 124 Version: TLS 1.0 (0x0301)             Random                gmt_unix_time: Aug  3, 2013 06:45:04.000000000 India Standard Time                random_bytes: 894966609a64a0b0ba0b4cd5adcc431aad77f0ff6108590e...            Session ID Length: 0            Cipher Suites Length: 24            Cipher Suites (12 suites)                 Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)                Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)                Cipher Suite: TLS_RSA_WITH_RC4_128_SHA (0x0005)                Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)                Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA (0x0032)                Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA (0x0038)                Cipher Suite: TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA (0x0013)                Cipher Suite: TLS_RSA_WITH_RC4_128_MD5 (0x0004)            Compression Methods Length: 1            Compression Methods (1 method)            Extensions Length: 59            Extension: renegotiation_info            Extension: server_name            Extension: status_request            Extension: elliptic_curves            Extension: ec_point_formats            Extension: SessionTicket TLS
  • The client sends a CLIENT HELLO to the server on the IP & Port it obtained during TCP handshake.

  • For this scenario I will consider IIS 7.5 as the SERVER entity. Upon receiving the CLIENT HELLO, the server has access to the following information:

    • IP Address (
    • Port Number (443)
    • Protocol Version (TLS 1.0)
    • List of Cipher Suites
    • Session ID
    • List of CLIENT HELLO Extensions etc.

    The Server will first check if it supports the above protocol version and if any of the cipher suites in the provided list. If not, the handshake fails there itself.

    The Server will now try to determine if there is an end point listening on the IP and PORT. If it finds an endpoint and if it is IIS, then the TCPIP.SYS driver moves the packet to the HTTP.SYS layer.

    • HTTP.SYS moves the request into the generic SSL Queue.
    • Until IIS 7.5 the SSL bindings were IP based i.e., IP+ Port and were associated with a certificate hash.
    • The HTTP.SYS tries to determine the certificate has corresponding to this IP+Port combination. It does so by enumerating the following registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HTTP\Parameters\SslBindingInfo
    • From the above, the certificate hash corresponding to the IP+PORT combination is determined. Now the HTTP.SYS calls the CRYPTO API's by passing on the cert hash to retrieve certificate blob, which calls the certificate store, finds the certificate and sends it back to the HTTP.SYS.
  • The Server responds to the client with SERVER HELLO. RFC 3546 defines the format of the SERVER HELLO:

Blake-Wilson, et. al.            Standards Track                       [Page 5]RFC 3546                          TLS Extensions                      June 20032.2. Extended Server HelloThe extended server hello message format MAY be sent in place of the server hello message when the client has requested extended functionality via the extended client hello message specified in Section 2.1.  The extended server hello message format is:      struct {          ProtocolVersion server_version;          Random random;          SessionID session_id;          CipherSuite cipher_suite;          CompressionMethod compression_method;          Extension server_hello_extension_list<0..2^16-1>;      } ServerHello;Here the new "server_hello_extension_list" field contains a list of extensions.  The actual "Extension" format is defined in Section 2.3.Note that the extended server hello message is only sent in response to an extended client hello message.  This prevents the possibility that the extended server hello message could "break" existing TLS 1.0 clients.

The Server typically responds back with the following details:

  • SSL/TLS Protocol version.
  • One of the cipher suites from the list of cipher suites provided by client. (whichever is the most secure)
  • Certificate of the server (Without the private key of course)
  • List of SERVER HELLO Extensions.
  • (OPTIONAL)If the web app associated with this binding requires a Client Certificate for authentication then it would request the client to send the certificate. Here the IIS Sever would send the client the distinguished names of the list of TRUSTED ROOT CA it supports.

Below is a snippet of the network trace:

Frame 320: 257 bytes on wire (2056 bits), 257 bytes captured (2056 bits) on interface 0Ethernet II, Src: Cisco_e5:44:00 (10:bd:18:e5:44:00), Dst: WistronI_86:74:54 (3c:97:0e:86:74:54)Internet Protocol Version 4, Src: (, Dst: ( Control Protocol, Src Port: http (80), Dst Port: 42079 (42079), Seq: 1576, Ack: 359, Len: 203[2 Reassembled TCP Segments (1663 bytes): #319(1460), #320(203)]Secure Sockets Layer    TLSv1 Record Layer: Handshake Protocol: Multiple Handshake Messages        Content Type: Handshake (22)        Version: TLS 1.0 (0x0301)        Length: 1658        Handshake Protocol: Server Hello             Handshake Type: Server Hello (2)            Length: 81            Version: TLS 1.0 (0x0301)            Random            Session ID Length: 32            Session ID: 8d0a0000efffe1ad6a82edc6d6a8967bd759cd0f3bdf70e9...            Cipher Suite: TLS_RSA_WITH_RC4_128_SHA (0x0005)             Compression Method: null (0)            Extensions Length: 9            Extension: renegotiation_info            Extension: server_name        Handshake Protocol: Certificate            Handshake Type: Certificate (11)            Length: 1565            Certificates Length: 1562            Certificates (1562 bytes)                 Certificate Length: 1559                Certificate (,id-at-organizationalUnitName=Azure,id-at-organizationName=Microsoft,id-at-localityName=Bangalore,id-at-stateOrProvinceName=India,id-at-countryName=IN)                    signedCertificate                    algorithmIdentifier (shaWithRSAEncryption)                    Padding: 0                    encrypted: bcd1c6d0a5e548eea94749e950d9ed8d7b73a79ac63306f0...        Handshake Protocol: Server Hello Done            Handshake Type: Server Hello Done (14)            Length: 0
  • The Client uses the SERVER HELLO to perform SERVER AUTHENTICATION. This is described in detail here: the server cannot be authenticated, the user is warned and informed that an encrypted and authenticated connection cannot be established. If the server is successfully authenticated, the client proceeds to the next step.
NOTE: If you captured a network trace for a SSL Handshake you could see the details until SERVER HELLO, after that the encryption begins and nothing would be available and would make sense as the packets are encrypted.
  • The Client uses the data provided from the server to generate a pre-master secret for the session, encrypts it with the server's public key (obtained from the server's certificate), and then sends the encrypted pre-master secret to the server. If the server had requested for CLIENT CERTIFICATE, then client also signs another piece of data that is unique to this handshake and known by both the client and server. In this case, the client sends both the signed data and the client's own certificate to the server along with the encrypted pre-master secret.
  • If the server had requested for client authentication, the server attempts to authenticate the client. If the client cannot be authenticated, the session ends. If the client is successfully authenticated, the server uses its private key to decrypt the pre-master secret, and then performs a series of steps (which the client also performs, starting from the same pre-master secret) to generate the master secret.
  • Both the client and the server use the master secret to generate the session keys, which are symmetric keys used to encrypt and decrypt information exchanged during the SSL session and to verify its integrity (that is, to detect any changes in the data between the time it was sent and the time it is received over the SSL connection).
  • The CLIENT & the SERVER send each other a message informing that future messages from them will be encrypted with the session key. It then sends a separate (encrypted) message indicating that its portion of the handshake is finished.
  • The SSL Handshake is done. The Client and the Server send each other messages which are encrypted/decrypted using the session keys generated in the previous step.
  • It is now that the Client sends the actual HTTP Request packet to the Server in the encrypted form.
  • The Server decrypts the request via the symmetric key and generates a response, encrypts it and sends it back to the client.
  • This continues normally for the entire session of secure communication. However, at any time either the client or the server may renegotiate the connection. In this case the process repeats again.

Below is a diagrammatic representation of the SSL Handshake:

Identifying problems during SSL Handshake

Eventually, once the handshake completes and the data exchange has been done, either both or one of the entities will eventually close down the connection gracefully. If there was a problem during the SSL Handshake then you there would be an exception raised within the SSL Layer (SSL ALERT PROTOCOL) . These exceptions may or may not be fatal i.e. not all exceptions would cause the handshake to fail.

As we know there we can see details only until SERVER HELLO. Anything beyond this point is not visible. However, in case of a SSL ALERT we would see some notification which can be viewed in the network traces.

Servers also tend to propagate this information through some sort of server logging. On Windows we have SCHANNEL logging which throws a corresponding SCHANNEL event in the SYSTEM event logs. Watch out for these events. Below is a snippet of one such event:

Log Name:      SystemSource:        Schannel Date:          05-08-2013 20:16:02Event ID:      36888Task Category: NoneLevel:         ErrorKeywords:      User:          SYSTEMComputer:      My-ComputerDescription:The following fatal alert was generated: 40. The internal error state is 1205.


However, do remember that not all the alerts that you see are fatal. Try to reproduce the error and confirm that this was the error message that was logged due to the problem you were facing.

You could either use WireShark or Network Monitor (NETMON) . I using both depending on what scenario I am running into.

That's it for now folks, hopefully this would give you some idea on how SSL handshake works. Let me know if you have any queries/suggestions.


TCP/IP Handshake:

Description of SSL Handshake:

Description of Server Authentication during SSL Handshake:

SSL/TLS Alert protocol & the alert codes:

Server Name Indication: