Configure VPN Remote Access with ISA Server 2006

Alan Maddison


At a Glance:

  • VPN protocols
  • Configuring a site-to-site VPN connection
  • Configuring a remote access VPN connection
  • New features in ISA Server 2006

Virtual Private Networks (VPNs) provide a high level of adaptability, scalability, and control over network connectivity. In addition to enhancing security, they can deliver significant cost savings

over traditional dedicated point-to-point connections. And they're flexible.

There are many types of VPNs. Some are used to connect mobile users to the corporate network; some are used to connect two networks that are geographically separated. The protocols these different VPNs use and the features they offer vary. Some will provide security capabilities, such as authentication and encryption, while others might offer no built-in security features at all. And some, obviously, are easier than others to set up and manage.

In this article, I will look at how you can implement a VPN using Internet Security and Acceleration (ISA) Server 2006. In particular, I will focus on how the ISA VPN functionality can be used to address two common scenarios: as a way to provide VPN remote access connections for users and as a way to provide site-to-site VPN connections. Both implementations include security features to ensure data privacy and to provide protection for internal network resources.

In the first scenario—a remote access VPN connection—a remote client initiates a VPN connection to your ISA Server over the Internet. ISA Server then connects the remote client to your internal network, providing the user with seamless access to internal network resources, including data and applications. The second scenario, a site-to-site VPN connection, is a little more complex because of its use of a router, which may be ISA itself. But this connection type provides the ability to seamlessly connect different office or branch locations to one another or to a central office.

There are several additional benefits gained by using ISA Server 2006 to implement a VPN. One of the most significant is due to the integrated nature of ISA Server, which means that the VPN functionality and the firewall functionality work hand in hand. In addition, ISA Server provides a VPN quarantine feature that utilizes the Network Access Quarantine Control feature of Windows Server® 2003 to quarantine a remote access computer until its configuration has been validated by a server-side script. This adds another layer of protection by providing a means to check such things as antivirus definition status and local firewall policy on the remote computer before you allow it access to your internal network resources.

The principal administrative benefits offered by ISA Server as a VPN solution include centralized management of policies, monitoring, logging, and reporting. For your day-to-day administrative responsibilities, these capabilities can prove to be invaluable. In particular, the real-time monitoring and log filtering capabilities give you key insight into network traffic and connections passing through ISA Server.

VPN Protocols

The core tunneling protocols used in an ISA VPN connection provide the first line of defense for securing connections. ISA Server supports three protocols—Layer 2 Tunneling Protocol (L2TP) over IPsec, Point-to-Point Tunneling Protocol (PPTP), and IPsec tunnel mode. This last protocol is supported only for site-to-site connections and is primarily used for interoperability with routers and other operating systems that do not support L2TP or PPTP.

L2TP is paired with IPsec because L2TP has no mechanism of its own to provide data privacy. When teamed with IPsec, which offers robust authentication and encryption methods, the combination provides a compelling solution. PPTP uses either Microsoft® Challenge Handshake Authentication Protocol (MS CHAP) version 2 or Extensible Authentication Protocol-Transport Layer Security (EAP-TLS) for authentication and Microsoft Point-to-Point Encryption (MPPE) for, as you can guess, encryption.

Whether an organization should choose L2TP over IPsec or PPTP often depends on factors that are specific to that organization. L2TP over IPsec will let you use more complex and sophisticated encryption and supports more types of networks (including IP, X.25, Frame Relay, and ATM). However, PPTP is easier to implement and typically has fewer requirements. That said, in situations where there are no organizational constraints, it's considered a best practice to implement L2TP over IPsec.

Site-to-Site VPN Connection

ISA Server 2006 has made some tremendous progress in simplifying the setup of a site-to-site VPN. In earlier versions, there were many more steps involved. For this example, I'll discuss a single remote site that is connecting to a central office using L2TP over IPsec with an ISA Server in both locations. The process requires that you configure the central office ISA Server before you configure the remote site.

The first step in the process is to configure the local user accounts that are required on both ISA Servers. This user identity will provide the security context under which the connection to the remote or main ISA Server will be made. The name of this account should match the name of the VPN connection you create within the ISA administration console. For instance, a good naming convention would be to set up the user name "Site VPN" on the ISA Server located at the headquarters (pointing to the remote site) and "HQ VPN" on the remote ISA Server. Once you've created these accounts, you need to access the account properties, open the Dial In tab, and make sure the Allow Access option is selected.

After you have created the local user account, the next step is to create the public key infrastructure (PKI) certificate that will be used for authentication between the two sites. You have the option of using a pre-shared key, but use of a certificate is always a preferable method. The simplest way to install a certificate for a VPN connection is to connect directly to your own enterprise certificate authority and request a certificate through the certificate server's Web site. But to do this, you may need to create an access rule to allow your ISA Server to connect to the certificate server via HTTP.

If you are familiar with earlier versions of ISA Server, you'll notice that the ISA Server 2006 interface, shown in Figure 1, is far more intuitive. When you select VPN in the left-hand pane, context-sensitive configuration and task options are displayed in the center and right panes.

Figure 1 ISA Server 2006 has a more intuitive interface

Figure 1** ISA Server 2006 has a more intuitive interface **(Click the image for a larger view)

Launch the site-to-site VPN wizard by clicking on the Create VPN Site-to-Site Connection task in the right-hand pane. When the wizard launches, it asks for a site-to-site network name; this name should match the name of the user account that you created earlier. After entering the name, press the Next button. On the next screen (see Figure 2), select the VPN protocol to be used—in my example, this is L2TP over IPsec. Press Next and the wizard helpfully warns you that a user account matching the network name must be available—since you were careful and did this, you can just press OK to get to the next screen.

Figure 2 Select the VPN protocol you want to use

Figure 2** Select the VPN protocol you want to use **(Click the image for a larger view)

Now you have to create an IP address pool. You have two choices here: you can either create a static address pool on the ISA Server or you can use your existing DHCP server to handle address assignment. Since both methods are acceptable, your decision should be guided by your company's procedures as they relate to this particular option. The screen now asks you to input the remote server name. Enter the fully qualified domain name (FQDN) of the remote server and then enter the user credentials for the connection. Remember that you have to enter the user account information you created on the remote ISA Server. In the example here, this would be the user account HQ VPN, as shown in Figure 3.

Figure 3 Enter the FQDN of the remote server

Figure 3** Enter the FQDN of the remote server **(Click the image for a larger view)

On the next screen, select the outgoing authentication method. (As I mentioned earlier, it is always preferable to use a certificate.) Then, input the IP address range being used on the internal network at the remote site.

After this, if you are using ISA Server 2006 Enterprise Edition, the wizard will allow you to configure the Network Load Balancing dedicated IP addresses of the remote site. If you are using ISA Server 2006 Standard Edition, skip the load-balancing step and go directly to the next screen (see Figure 4), which is where you create the network rule used to route traffic from the remote site to the local network.

Figure 4 Create the network rule that routes traffic

Figure 4** Create the network rule that routes traffic **(Click the image for a larger view)

You also have to create an access rule, which is the next step in the process. When you configure the access rule, there are three options for allowing protocols to be used—you can choose to allow all outbound traffic, allow all outbound traffic with the exception of certain protocols, or allow only certain protocols. In most situations you'll want to select the all outbound traffic option.

You're nearly finished. At this point, the wizard will present a summary of your configuration and once you press the Finish button, the Site-to-Site VPN connection will be created. You will receive a dialog indicating that a system policy rule needs to be enabled for Certificate Revocation List downloads—press Yes to enable this rule. The wizard will then provide information about any additional configuration steps that may be needed. After completing the configuration of ISA Server at the main site, you need to follow all the same steps to configure the remote site. Once both steps have been completed, users at both sites will be able to communicate across the VPN.

Remote Access VPN Connection

Configuring a remote access VPN connection for client access is far more straightforward (see Figure 5). The first step is to select VPN in the left-hand pane of the ISA Server administration console. Then select the Enable VPN Client Access task in the right-hand pane. You will receive a warning that this may cause the Routing and Remote Access service to restart. (Since this can cause connection issues, you may want to implement this change during a scheduled maintenance window.) Click OK to proceed past this warning; ISA Server enables a system policy that allows VPN client traffic to access the ISA Server. You can view this rule by selecting Firewall Policy in the ISA Server administration console and then selecting the Show System Policy Rules task.

Figure 5 Configure a remote access VPN connection

Figure 5** Configure a remote access VPN connection **(Click the image for a larger view)

A default network rule is also enabled to allow routing between the internal network and the two VPN networks (VPN Clients and Quarantined VPN Clients). This network rule can be viewed or edited by selecting Networks in the left pane and then opening the Network Rules tab in the center pane.

Now you have to configure VPN client access. There are three additional parameters that need to be configured: the Windows Security Groups that are permitted access, the protocols that may be used by the clients, and user mapping. The dialog for configuring these parameters is shown in Figure 6.

Figure 6 Configure VPN client access

Figure 6** Configure VPN client access **

The Groups tab simply requires that you select which Windows groups should be allowed remote access via VPN. From an administration perspective, I think it's a good idea to create a single group to which permission is given. This makes it very easy to manage remote access.

The Protocols tab shows that only PPTP is enabled by default. You should definitely enable L2TP over IPsec if this is at all feasible in your environment.

User Mapping lets you map user accounts from namespaces, such as Remote Authentication Dial-In User Service (RADIUS), to Windows accounts to ensure that access policies are applied correctly. Once you have completed these configuration options and applied the changes to ISA Server you are finished. Clients should now be able to seamlessly access your internal network data and applications using a VPN connection.

Enhancements to ISA Server 2006

ISA Server 2006 introduces a number of enhancements that increase its power and performance over previous versions. These features—which include HTTP compression, Background Intelligent Transfer Service (BITS) support, and Quality of Service (QoS)—provide various techniques for optimizing network usage. Of course, even with these techniques, you should also continue to use common bandwidth optimization technologies that are core to ISA Server, including caching.

HTTP compression has been supported in various Microsoft products for some time now—it's been in Internet Explorer® since version 4 and Windows Server since Windows® 2000. However, this is the first version of ISA Server to support HTTP compression as well as the industry standard GZIP and Deflate algorithms.

What does this mean? With support for HTTP compression, a Web browser that complies with HTTP 1.1 can request compressed content from any Web site. Remember, though, that only the responses are compressed, not the initial outgoing connections from the client.

HTTP compression is a global HTTP policy setting that applies to all HTTP traffic that passes through ISA Server, as opposed to being associated with a specific network rule. You do, however, have the option of implementing HTTP compression on a per-listener basis; this is configured through the Web Listener wizard.

HTTP compression, which can be accessed through the General option in the left-hand pane, is enabled by default. But as Figure 7 shows, you have to configure the network elements where compression should be used. Continuing with the site-to-site VPN example, you need to select the Return Compressed Data tab and add the Site VPN network element that you created earlier. At this point you also have the option of configuring which content types should be compressed. ISA Server 2006 comes with a list of standard content types, but you can add custom content types through the Firewall Policy task pane on the Toolbox tab. Remember that the Return Compressed Data tab refers to ISA Server compressing the data before it is returned to the client. In order to have ISA Server request that a Web server handle the compression of data before the data is sent to ISA Server, you need to use the Request Compressed Data tab.

Figure 7 Configure HTTP compression

Figure 7** Configure HTTP compression **(Click the image for a larger view)

Background Intelligent Transfer System is an asynchronous file transfer service for HTTP and HTTPS traffic. Windows Server 2003 includes BITS version 1.5, which supports downloads and uploads, though uploads also require Internet Information Services (IIS) version 5.0 or higher. As an intelligent file transfer service, BITS has the ability to examine network traffic and use only the idle portion of your network bandwidth. In addition, the file transfer is dynamic and BITS will respond to spikes in normal network traffic by throttling back its network usage. The advantage here is that BITS ensures that downloads and uploads of large files will not negatively impact other network usage. From an administrator's perspective, this means you should receive far fewer calls complaining about poor network performance. Good news!

BITS offers other benefits that can improve the user experience and make for better network performance. For instance, a file transfer that uses BITS is binary, meaning BITS can resume file transfers (without starting from scratch) that have been interrupted by such factors as a network outage. And ISA Server 2006 includes built-in support for a Microsoft Update caching feature that uses BITS caching to optimize updates.

The Differentiated Services (DiffServ) protocol enables packet prioritization (or QoS) for HTTP and HTTPS traffic. In other words, depending on your ISA Server configuration, certain traffic will receive priority. This is useful on networks that have limited bandwidth, either due to traffic volume or network connection speed. According to the Internet Engineering Task Force (IETF), DiffServ is a class-based traffic management mechanism. Thus, DiffServ can differentiate between types of traffic and manage them accordingly, ensuring that high-priority traffic gets preference.

To provide this feature, ISA Server works in conjunction with routers that support QoS. It is important to ensure that ISA Server DiffServ bits match the priorities on your routers if you want the packets to be routed with the same prioritization.

As implemented in ISA Server, this type of packet prioritization is an HTTP policy setting that is global and is applied to all HTTP traffic that passes through ISA Server 2006 using a DiffServ Web filter. This means that ISA Server does not support DiffServ for other protocols and may, in fact, drop existing DiffServ bits on non-HTTP traffic.

As shown in Figure 8, DiffServ is implemented as a Web filter and can be accessed by selecting Add Ins in the left pane. It is important that you not change the default settings or the priority order of the DiffServ filter due to the need for ISA Server to inspect the size of request/response at the time it is processed.

Figure 8 View the DiffServ Web filter

Figure 8** View the DiffServ Web filter **(Click the image for a larger view)

If you take a close look at Figure 8, you might notice that the DiffServ filter is not enabled by default. To enable the filter, simply select the Enable Selected Filters in the Tasks pane and then configure the filter. Since the DiffServ packet prioritization in ISA Server is based on either specific URLs or domains, you need to add this information to the DiffServ preferences. To do this, select General in the left pane of the management console and then, under Global HTTP Policy Settings, select Specify DiffServ Preferences. When you specify these preferences, you must define the priorities and the URLs and domains that should be prioritized. ISA Server 2006 provides powerful new capabilities for setting up remote access for VPN clients or creating site-to-site VPNs. It should be at the top of your list when considering which VPN solution to implement.

Alan Maddison is a Senior Consultant in the Microsoft technology practice of MTI Technology Corporation. His primary focus is on Active Directory, Exchange Server, and virtualization.

© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.