Using IPSec in Tunnel Mode

Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

Tunnel mode is primarily used for interoperability with gateways or end systems that do not support L2TP/IPSec or PPTP VPN site-to-site connections. Table 6.3 provides a summary of scenarios in which IPSec tunnel mode is appropriate to use.

Table 6.3   IPSec Tunnel Mode Usage

IPSec Scenario	IPSec Tunnel Mode Usage
Establish gateway-to-gateway tunnels between sites, when the gateways or end systems do not support L2TP/IPSec VPN connections.	Use only when necessary for interoperability with gateways or end systems that do not support L2TP/IPSec VPN connections. When L2TP/IPSec VPN connections are supported, use L2TP/IPSec (IPSec transport mode) instead. When IPSec in tunnel mode is used for gateway-to-gateway tunnels, static IP addresses are required for the IPSec tunnel endpoints. For an illustration of this scenario, see Figure 6.6.
Protect traffic end-to-end, but one endpoint of the communication does not support IPSec.	Use an IPSec tunnel from one endpoint to a firewall or IPSec-enabled router nearest to the endpoint that does not support IPSec. This scenario requires static IP addresses for both tunnel endpoints. For an illustration of this scenario, see Figure 6.7.
Encrypt traffic end-to-end between two computers, but a third-party firewall or network intrusion detection system requires that traffic be decrypted.	Use one IPSec tunnel to reach the third-party firewall or the network intrusion detection system. Use another tunnel from the third-party firewall or network intrusion detection system to reach the destination. This scenario requires static IP addresses for all four tunnel endpoints. For an illustration of this scenario, see Figure 6.8.

Gateway-to-Gateway Traffic Must be Secured

In this scenario, traffic is being sent between a client computer in a vendor site (Site A) and a File Transfer Protocol (FTP) server at the corporate headquarters site (Site B). Although an FTP server is used for this scenario, the traffic can be any unicast IP traffic. The vendor uses a third-party gateway, while corporate headquarters uses a gateway running Windows Server 2003. An IPSec tunnel is used to secure traffic between the third-party gateway and the gateway running Windows Server 2003. Figure 6.6 shows IPSec in tunnel mode for gateway-to-gateway tunneling between sites.

Figure 6.6   Gateway-to-Gateway Tunneling Between Sites

Note

• For ease of deployment, it is recommended that the gateway running Windows Server 2003 be a stand-alone computer and that you remotely manage this computer by using Remote Desktop Connection. If this computer is joined to a domain, the computer must be able to access Active Directory locally, not through an IPSec tunnel.

One Endpoint Does Not Support IPSec

In this scenario, an IPSec tunnel is used to secure traffic between a computer running Windows Server 2003 in a perimeter network and an IPSec-enabled router (for example, a Cisco IOS router). Traffic on the path between the router and the third-party server in the internal corporate network is not secured, because the third-party server does not support IPSec. Note that this is not a VPN remote access scenario, because there is no dynamic address assignment, nor is user authentication used to establish the tunnel. The tunnel rules can be defined to protect traffic between the computer running Windows Server 2003 and either the internal network subnet or the specific IP address of the third-party server. Figure 6.7 shows IPSec in tunnel mode to provide end-to-end security of traffic when one endpoint of the communication does not support IPSec.

Figure 6.7   One Endpoint Does Not Support IPSec

Traffic Must Be Decrypted for Third-Party Firewall Inspection

In many cases, IPSec transport mode can provide secure end-to-end communication through firewalls. If traffic must be inspected at the firewall, you can use IPSec AH or ESP with null encryption in transport mode to maintain end-to-end security. However, if the traffic must be encrypted and inspected, you can use IPSec tunnel mode to secure traffic to the inspection point.

Figure 6.8 shows IPSec in tunnel mode encrypting traffic end-to-end between two computers: a line of business (LOB) application server running Windows Server 2003 in a perimeter network, and a computer running SQL Server that functions as a data store for the application server in an internal corporate network.

Figure 6.8   Traffic Must Be Decrypted for Third-Party Firewall Inspection

In this scenario, IPSec tunnels are used to secure traffic between the application server and the third-party firewall’s external interface and between the third-party firewall’s internal interface and the computer running SQL Server. Traffic is decrypted for firewall inspection and then reencrypted when it is forwarded to the IPSec peer. The tunnel rules can be defined to protect traffic between the application server and either the internal network subnet or the specific IP address of the computer running SQL Server.

Note

• This scenario cannot work if a computer running Microsoft® Internet Security and Acceleration (ISA) Server is used as the firewall. Use IPsec transport mode on the side where the ISA interface IP address is being used as the source or destination IP address.

For more information about using IPSec in tunnel mode, including detailed configuration procedures, see article Q252735, "How to Configure IPSec Tunneling in Windows 2000," in the Microsoft Knowledge Base. To find this article, see the Microsoft Knowledge Base link on the Web Resources page at https://www.microsoft.com/windows/reskits/webresources.