Microsoft Windows Server 2008 R2: Certificates Drive RDS Security

Certificates play a huge role in securing connections between Remote Desktop Services hosts and clients.

Kristin Griffin

Asking which roles use certificates is kind of a trick question. The real answer is “all of them.” It’s important to understand where you should use certificates in Remote Desktop Services (RDS) deployments, and why you use them with each particular RDS role service.

Most roles you’ll need to configure, but some of them you won’t (such as the RD License Server). By default, you’ll need SSL (x.509) certificates at every stage of connecting to a session or hosted virtual machine (VM). You’ll use these for three main purposes: to secure communications between client and server, to confirm the identity of the server or Web site to which the client is connecting, and to sign Remote Desktop Protocol (RDP) files so your users know the RDP file comes from a trusted source and hasn’t been altered.

Here are some examples of how RDS uses certificates:       

• RD Session Host servers use certificates to prove their identity. This is called server authentication.
• RD Session Host servers and RD Virtualization Host servers use certificates to set up a secure link between client and server with TLS 1.0.
• RD Session Host Servers use certificates for client authentication required for Network Level Authentication (NLA), Single Sign-On (SSO) and implementing Web SSO.
• RD Session Host servers and RD Connection Broker both use an SSL certificate to sign RemoteApps and VM RDP files, assuring users they’re launching a trusted file.
• RD Gateway servers use certificates to encrypt communications with clients using TLS 1.0.
• You can secure the RD Web Access site with an SSL certificate to ensure that people are going to a trusted site (HTTPS).

Enabling RDS functionality relies on specific technologies to support the use of certificates (see Figure 1).

Figure 1 Enabling RDS functionality brings into play certain security technologies.

Secure the Channel

TLS is the Internet Engineering Task Force (IETF) standard based on SSL version 3, published by Netscape. Some of the enhancements to TLS include new message alerts, the ability to chain certificates to an intermediary Certificate Authority (CA) certificate instead of the root CA certificate, and slightly different encryption algorithms from SSL.

Although TLS is based on SSL, the two are incompatible. TLS can, however, implement a mechanism by which it can fall back to SSL version 3 if necessary. To establish a secure communication channel between a client and server using TLS, the client and server go through a process of messaging, response and encryption (see Figure 2).

Figure 2 The two-way encryption process for establishing a secure channel.

There are two requirements for this process to work properly.

• The client must trust the CA that signs the server’s SSL certificate.
• The connection between server and client must use high-level (by default) or Federal Information Processing Standard (FIPS) encryption. Low-level encryption only encrypts the traffic from client to server, not server to client, so it’s not a secure way to send security capabilities or shared secrets.

If the connection and encryption level meets those two requirements, the client and server establish communication as follows:

1. The client sends a hello message along with a random fixed-length value. The server responds with a random fixed-length value. During this exchange, the client tells the server the compression methods, ciphers and hashes it supports. It also sends its protocol version and a session ID to the server. The session ID identifies the communi­cation channel—this is not the Session ID on an RD Session Host server.
2. The server picks the highest encryption method they both support and a cipher and hash function from the client’s list. Then it tells the client which one it has cho­sen. If there’s a minimum level set for the server and the client can’t meet this minimum, the connection will fail.
3. The server sends its digital certificate to the client. This certificate contains the server’s name, the trusted CA that signed the certificate and the server’s public key.
4. The client verifies the certificate is valid and trusted. The certificate used to sign the server certificate will be in the client’s Trusted Root Certification Authorities store. Then it creates a pre-master secret, encrypts it with the server’s public key and sends it to the server.
5. The server receives and decrypts the pre-master secret with its private key. This server is the only one that can do this because it’s the only server with the matching private key.
6. Both the server and client have the pre-master secret and random numbers exchanged at the beginning of the process. They use these values to generate the 48-byte master secret (also known as the shared secret). After generating the master secret, they delete the pre-master secret.
7. Both client and server then hash the 48-byte master secret and use it to generate the MAC secret (the session key used for hashing) and the WRITE key (the session key used for encryption). These keys encrypt and decrypt communication for this session. After the session is over, the keys are discarded.

If any step of this sequence doesn’t work, the connection hasn’t been fully secured. What happens then depends on the Advanced tab settings on the Remote Desktop Connec­tion (RDC) client. In the case of authentication failure, a user can choose to do any one of the following:

• Connect anyway without notifying the client there was a problem authenticating the server.
• Warn the client, but still allow the connection (the default setting).
• Deny the connection if it can’t be verified.

The exception is if the server requires a minimum security level. If that’s the case and the client can’t meet the minimum level, the connection will fail. By default, the client and server will negotiate and use the most secure connection settings they both support.

Credential Caching with CredSSP

Credential caching was introduced with Windows Vista and Windows Server 2008. This enables two features—one that helps the user and one that helps protect the server.

Credential caching helps users by storing credentials for a particular connection so they don’t need to provide them every time they connect to that server (this is SSO). This speeds up the connection. Otherwise a brokered connection must be checked at each step.

On the server side, credential caching provides credentials to the server before it establishes a session. This avoids the overhead of a session if the user isn’t authorized (this is NLA).

The piece that makes credential caching work is the Credential Security Service Provider (CredSSP). This is supported by Windows 7, Windows Vista, Windows Server 2008 and Windows XP SP3. It isn’t linked to the version of RDC being used because CredSSP is part of the OS. CredSSP performs the following functions:

• For NLA, CredSSP provides the framework that authenticates a user to an RD Session Host server before fully establishing the connection.
• For reconnecting to a session within a farm, CredSSP speeds the process of passing the connection to the correct server by letting the RD Session Host server see who’s logging on without having to create an entire session. This uses NLA in a slightly different scenario.
• For SSO, CredSSP stores user credentials and passes them to the RD Session Host server to automate logon.

CredSSP enables mutual authentication of the server and client (see Figure 3).

Figure 3 Authenticating both the server and client requires a secure channel.

This authentication process takes the following steps.

1. The client initiates a secure channel with the server using TLS. The server passes back the certificate with its name, CA and public key. Only the server is identified. The client remains anonymous at this point.
2. When the session is established and a session key created, CredSSP uses the Simple and Protected GSS-API Negotiation (SPNEGO) protocol to mutually authenti­cate the server and client. Basically, this mechanism lets the client and server agree on an authentication mechanism they both support, such as Kerberos or Windows NT LAN Manager (NTLM).
3. After mutual authentication is finished, CredSSP on the client side encrypts the server’s certificate with the session key created during step 2, and sends it to the server. The server receives the encrypted certificate, decrypts it with its private key, and then adds one to the most significant bit of the certificate number. It then encrypts the result and sends it back to the client. The latter operation ensures that no one can intercept the exchange between client and server and spoof the server.
4. The client reviews the encrypted certificate received from the server and com­pares it to its own certificate.
5. Assuming the results match, CredSSP on the client side sends the user credentials to the server.

Authenticating Server Identity

One danger of communicating with a remote computer that requires you to supply your credentials is that the server might not be what you think. If it’s a rogue server impersonating a trusted one, you could inadvertently type your credentials into the wrong server. This would give attackers everything they need to connect to your domain or server.

RDP includes encryption, but the protocol doesn’t have any means to authenticate the server. That’s where TLS and CredSSP come in. Server Authentication checks the name you enter in the RDC client (or RDP file) against the name issued in the certificate specified in the RD Configuration Tool on the RD Session Host server to which it’s connected.

Signing RDP Files

You can use certificates to digitally sign RemoteApp files, as well as RDP files used to connect to a pooled or personal VM (VDI). Signing these files assures the user they were created by a trusted source. It also secures the RDP file from tampering.

Signing RemoteApp files is also required for implementing Web SSO. This lets users sign in once to the RD Web Access Web site. Then they can launch RemoteApps from any farm without having to provide their credentials again.

CredSSP can’t pass credentials to RD Web Access. The user must first log in to the Web site to store their credentials. Then they won’t need to authenticate again to start RemoteApp programs. For this to work, the RemoteApps must be signed and the user must trust the certificate used to sign the RemoteApp.

RDP files created when a user launches an RDP connection from the RD Web Access Desktops tab are created on the fly. The files aren’t signed. Therefore, Web SSO won’t work when connecting to desktops. The user will have to log in to the endpoint once the connection is established. Web SSO also won’t work for connections to pooled or personal VMs.

Securing Connections

RD Gateway uses TLS 1.0 to secure communications between RD Gateway and remote desktop clients, typically located outside your corporate network (over the Internet). TLS, as described earlier, requires an SSL certificate. You can also secure communications between clients and the RD Web Access server using a digital certificate to encrypt the Web site sessions (HTTPS).

Certificates are an essential piece of an RDS deployment. RDS uses certificates to secure communications and the features they enable. Next month, I’ll cover setting up RDS role services to use those certificates.

Kristin Griffin is a Remote Desktop Services MVP. She moderates a Microsoft forum dedicated to helping the server-based computing community (Remote Desktop Services) and maintains an RDS blog at blog.kristinlgriffin.com. She’s a contributor to Mark Minasi’s “Mastering Windows Server 2008” (Sybex, 2008) and “Mastering Windows Server 2008 R2” (Sybex, 2010). She also coauthored “Microsoft Windows Server 2008 Terminal Services Resource Kit” (Microsoft Press, 2008) and “Microsoft Windows Server 2008 R2 Remote Desktop Services Resource Kit” (Microsoft Press, 2010) with Christa Anderson.

Related Content

• Add Hyper-V to RDS for Inexpensive Virtual Desktops
• Digitally Signing RemoteApp Programs
• Implementing Web SSO