A Guide to Securing ISA Server 2006

Alan Maddison


At a Glance:

  • Best practices for securing your servers
  • Setting up the Security Configuration Wizard
  • A Security Configuration Wizard walkthrough
  • Assigning Administrative roles


Securing Your Servers
Setting Up the Security Configuration Wizard
Running the SCW
Administrative Roles

While many IT pros rely on ISA Server 2006 (Internet Security and Acceleration Server 2006) to secure their technology assets, few take the extra step of securing ISA Server itself. If you

have recently installed ISA Server 2006, you may recall that there was a reminder to do this immediately after installation is complete. Unfortunately, many of us IT pros rarely take (or have) the time to perform this important step, and so it ends up on that list of things to do that never get done.

In today's constantly changing security landscape, this failure to secure ISA Server is no longer acceptable. Fortunately, the tools used to secure ISA Server have made tremendous advances. No longer do you have to face many of the challenges in the Security Hardening Wizards found in versions prior to ISA Server 2004. Instead, you can rely on well-defined steps and tools, such as the Security Configuration Wizard (SCW) that ships with Windows Server® 2003.

In this article I will provide a brief recap of general best practices for securing servers. Then I'll take a step-by-step look at hardening strategies for ISA Server itself, using the Security Configuration Wizard to reduce the attack surface area of the ISA Server and Administrative roles to restrict access to the ISA Server.

Securing Your Servers

There are many elements and best practices involved in securing servers, whether they are located in a datacenter or in the server room next to your office. As an administrator, it is your responsibility to understand what these best practices are and do your best to implement these requirements in a manner that is practical for your organization. As the emphasis on security has become more prevalent in recent years, many of us have become quite familiar with the tasks that form the core of these requirements, so I won't belabor the points but instead will provide only a brief overview.

The first step you must take to secure your environment is to ensure that your servers are physically secure. What this means in practical terms is that you must restrict physical access to your servers. In smaller environments this means making sure the server room door remains locked and the list of individuals with access to the room remains very small. In larger environments this basic requirement remains very much the same, but it can be implemented in a more sophisticated manner. Many organizations use electronic monitoring, for example. This allows them to audit entry to server locations and even restrict access to individual racks or cages depending on how job responsibilities are structured.

There are some unique factors to consider when dealing with the theft or physical compromise of an ISA Server or an ISA Configuration Storage Server. The nature of the information that can be obtained from the stolen server can potentially compromise all the ISA Servers and traffic (including encrypted traffic) in your environment.

If you suspect a server has been compromised, stolen, or otherwise, remove the affected server immediately (if it's still on site) and follow standard procedures for securing evidence. After you have taken these necessary steps, you need to begin the process of changing all confidential information—all certificates installed on the server should be revoked and all preshared keys and shared secrets should be modified. In addition, if you maintain a replica Configuration Storage Server, make sure that any data relating to the compromised server is removed.

Once your servers are physically secure, the next step is to ensure that there is a structured methodology for patching all the software, including the virtualization layer, the OS, and applications. Patches, upgrades, and hotfixes should be reviewed and applied regularly. But don't forget to test these updates before applying them to production systems. A patch won't do you any good if it ends up causing a problem that compromises application or data integrity.

If data integrity is compromised, you will need to rely on your backups. This brings me to another key element in securing your infrastructure. If you can't quickly and completely restore data if the need ever arises, the downtime can have a significant impact on your operations and, as a result, increase the cost of an intrusion.

Two other elements to consider are monitoring and auditing. Monitoring your applications and systems is a critical piece of any good security plan. If you don't take the time to review logs, particularly security-related logs, you are unlikely to ever find intrusion attempts before the damage is already done.

Likewise, auditing is critical. For many organizations, especially large environments, this is often formalized and even required by legislation. Regardless, it is important in any environment to review on a regular basis the controls and methodology used to secure your assets for your efforts to be effective.

Finally, since you're running ISA Server 2006 on Windows Server® 2003, it is important that you review the Windows Server 2003 Security Guide and implement the necessary recommendations. You can find the Windows Server 2003 Security Guide at microsoft.com/technet/security/prodtech/win­dow­sserver2003/w2003hg/sgch00.mspx.

Microsoft currently recommends implementing the Baseline Security Policy template, but you should not implement any Internet Protocol Security (IPsec) filters.

Setting Up the Security Configuration Wizard

So how can you make ISA Server itself more secure? The primary tool for securing ISA is the SCW. This is an attack surface reduction tool. It creates security policies that target a server's services, network security, registry, and audit policy, configuring the system for just the services and features it requires. It is important to note that you should only configure ISA Server services that you intend to use. For example, the Web Proxy service is enabled by default, but you should disable this functionality if you don't intend to use it. Thus, you need to pay close attention to the configuration options the SCW presents to you.

The SCW isn't installed by default in Windows Server 2003, so the first step is to install it yourself using the Add/Remove Windows® Components applet in the Add or Remove Programs control panel. (Note that the SCW is installed by default in Windows Server 2008.) Once the Windows Components screen has loaded, scroll down and check the box for the SCW.

Once the installation is complete, the application can be found under Administrative Tools. Before you begin using the wizard you must update the SCW by downloading an update for ISA Server 2006 (available at go.microsoft.com/fwlink/?LinkId=122532). This update adds the roles for ISA Server 2006 Standard Edition, ISA Server 2006 Enterprise Edition, and ISA Server Configuration Storage Server.

After you download the update, you need to run the package and extract the files from it. After extracting these files, copy the two .xml files (isa.xml and isaloc.xml) to the SCW kbs folder—in a default installation of Windows Server, this will be c:\windows\­security\msscw\kbs.

When you copy the files, you will be prompted to overwrite two existing files with the same name. These two files are for ISA Server 2004, so you should back them up before overwriting them. The final step is to copy the isascwhlp.dll file to the bin folder, which is typically found in c:\windows\­security\msscw\bin. Once you have completed the addition of the ISA roles to the SCW, you will be ready to begin the installation.

Microsoft generally recommends that you only run the SCW once you have completed the configuration of ISA Server. If you are running Enterprise Edition, this includes the configuration of all the arrays and all the array members.

Running the SCW

The first step is to launch the SCW from Administrative Tools—remember that you need Administrative permissions to successfully complete the SCW process.

Figure 1 shows the first screen of the wizard. If you read through the text on this screen, particularly the warning that says "this wizard detects the inbound ports that are listened to by this server," you can understand why it is important to have your ISA Server and arrays fully configured prior to beginning this process.


Figure 1 Starting the Security Configuration Wizard (Click the image for a larger view)

If you have not fully configured your environment, there is a good chance that you will need to revise the SCW configuration after completing the ISA configuration. The next screen, shown in Figure 2, asks which action you want to perform. You should select the option to Create a new security policy.


Figure 2 Creating a new security policy (Click the image for a larger view)

You are then instructed to select the server that will serve as the baseline for the policy. Since you are creating a new policy, the default selection is to use the computer on which you are running the SCW. This behavior changes, however, depending on the action you choose to perform on the previous screen. Regardless, it is recommended as a best practice to have the SCW installed on the server you wish to use as the baseline. If the SCW is not installed on the target server, information used to complete the policy will be missing. Thus, to make your life easier, install and run the SCW from the server you want to serve as the baseline.

When you press Next, the SCW will begin analysis of your ISA Server. This analysis includes determining roles that are installed on the server, roles that are likely installed on the server, services that are installed, and basic networking information. When processing is complete, you can view the database by selecting View Configuration Database. The configuration database contains a lot of information, including all supported server roles, client features, and ports.

Then the SCW begins the process of walking you through Role-Based Service Configuration. Pressing Next takes you to the next screen, where you are prompted to select the server roles, as shown in Figure 3. The initial scan performed by the SCW is reliable, and you should find that the correct server roles have already been identified. However, it is very important that you double-check and remove any unnecessary roles. And if the server fulfills multiple roles, make sure all of the appropriate roles are selected.


Figure 3 Specifying server roles (Click the image for a larger view)

One important point to remember if you are running ISA Server 2006 Enterprise Edition is that you have to consider the Configuration Storage Server. If the Configuration Storage Server is installed on a server that also acts as an ISA Server (this, by the way, does not follow recommended best practices but is often done nonetheless), you will need to make sure the Configuration Storage Server role is also selected. You should not use a baseline scan of a server that hosts both roles for servers that actually have just the ISA Server 2006 role.

Next you are prompted to select the client features of the server. In other words, you need to specify the services required by the server. For instance, nearly all servers will require the DNS client, and if a server is a member of a domain, then it will require the Domain member feature.

After you've done that, you are shown the Administration and Other Options screen. This is where you indicate the application, administration, and operating system options that use services or rely on network connectivity. Any services not selected by this point will be disabled. But after you make your selections and press Next, you are given the option to select any additional services you want to allow.

Then you move on to configure how unspecified services should be handled. This lets you define what should happen when services that are not included in the main database or installed on the baseline server are found when the policy is being applied. As a general best practice you should choose to disable unspecified services because this will limit any unforeseen attack vectors. Unfortunately, this option can have negative consequences if your servers are unalike in any significant ways; you also need to remember this setting if you add any applications or network services in the future.

The next section of the wizard, which is shown in Figure 4, allows you to review the services that are being modified by the SCW. This confirmation screen provides you with a comparative view of both the current status and the modified status of services after the policy is applied.


Figure 4 Review and confirm service changes (Click the image for a larger view)

After you confirm the services that are going to be changed, you move onto the Network Security section. This is where the SCW would typically let you modify Windows Firewall and IPsec settings. But since you are configuring ISA Server 2006, you have no choice but to skip this section, as shown in Figure 5.


Figure 5 Skipping past the Network Security settings (Click the image for a larger view)

The wizard then moves on to configuring Registry settings that focus on network authentication methods and security. The first screen in this section involves Server Message Block (SMB) signing. The SMB protocol is a core Microsoft networking protocol, and these settings allow for signed communications in order to reduce the likelihood of man-in-the-middle attacks.

The default settings, as shown in Figure 6, provide a good level of security for your ISA Server SMB communications. But you must take into account the impact that will be caused by signing all communications. If you don't have the spare CPU cycles, you should uncheck the second option. And don't forget to think about all the servers to which this policy will be applied—if you have servers with different workloads, you need to use the server with the highest CPU utilization as the guideline for whether or not you select this option.


Figure 6 Specifying whether to require signed communications (Click the image for a larger view)

The next set of screens deal with the LMCompatibility level that the ISA Server should use. The first of these screens presents three choices. Unless you have legacy Windows clients (such as Windows 95 or Windows 98) or you use local accounts for access control, you should leave the default choice of Domain Accounts selected.

On the second screen dealing with LMCompatibility level, you provide information about your domain controllers. If you don't have any Windows NT® 4.0 domains, you can leave the default choice (Windows NT 4.0 Service Pack 6a or later OSs) selected. On this dialog, you should also select that the Clocks be synchronized with the selected server's clock option. Selecting Next will lead to some additional configuration options for inbound LAN Manager (LM) communication, as shown in Figure 7.


Figure 7 Indicating inbound authentication methods (Click the image for a larger view)

This third screen pertaining to LM­Compatibility level will determine whether the Windows NT LAN Manager (NTLM) version 2 is required and whether LM hashes are stored. You should make sure that neither of these options are selected, provided your environment will support this configuration, as deselecting these two options will improve security significantly. You are then presented with a Registry Settings Summary. Review each entry and confirm that the policy settings are correct.

Next, the wizard takes you to the final section—Auditing. One important point to note about any of the configuration options you make in this section is that they cannot be rolled back. But since auditing won't affect system functionality, you should not skip this section.

The default selection of "Audit successful activities" will not provide you with event log entries for logon failures. However, information regarding logon failures can offer valuable information about intrusion attempts. Thus, as a general best practice, you should select "Audit successful and unsuccessful activities".

You can then review your audit configuration and press Next twice to save your security policy. All that is required on this screen, shown in Figure 8, is a file name; however, you can also provide a brief description,. This description can prove useful in large environments where different Administrators share security responsibilities.


Figure 8 Providing a name and description for your security policy (Click the image for a larger view)

Once the file has been saved, you are given the option of applying the policy immediately or later. If you choose to apply the policy later, the process is complete. If you discover that you made any errors in the policy configuration, you can roll back the policy, with the exception of auditing settings.

Administrative Roles

Reducing the attack surface of your ISA Server is a critical step in reducing the potential for breaches from external sources. However, it's also important to review the assignment of Administrative roles within ISA Server to limit the potential for compromise from internal sources. The Administrative roles and a partial list of common associated tasks are shown in Figures 9 and 10.

Figure 9 Roles and tasks associated with Standard Edition

Task Monitoring Auditor Auditor Full Administrator
View Dashboard, alerts, connectivity, sessions, and services Allowed Allowed Allowed
Acknowledge alerts Allowed Allowed Allowed
View log information Allowed Allowed
Create alert definitions Allowed
Create reports Allowed Allowed
Stop and start sessions and services Allowed Allowed
View firewall policy Allowed Allowed
Configure firewall policy Allowed
Configure cache Allowed
Configure a virtual private network (VPN) Allowed

Figure 10 Roles and tasks associated with Enterprise Edition

Activity Array Monitoring Auditor Array Auditor Array Administrator
View Dashboard, alerts, connectivity, and sessions Allowed Allowed Allowed
Acknowledge and reset alerts Allowed Allowed Allowed
View log information Allowed Allowed
Create alert definitions - Allowed
Create reports Allowed Allowed
Stop and start sessions and services Allowed Allowed
View firewall policy Allowed Allowed
Configure firewall policy Allowed
Configure cache Allowed
Configure a virtual private network (VPN) Allowed
Perform NLB Drain/Stop Allowed Allowed
View local configuration (in registry of array member) Allowed Allowed
Change local configuration (in registry of array member)

As you can see, there is a high degree of segmentation in the administrative tasks associated with ISA Server. This, in turn, should make it easy for you to assign the correct roles to users within your organization.

Beyond this, what you need to remember is that the best approach to role assignment is to employ the concept of least privilege. Any given user should have only the least amount of privileges necessary to allow him to do his job.

It is also important to remember that members of the local Administrators group on ISA Server 2006 Standard Edition have the same rights as an ISA Server Full Administrator. With Enterprise Edition, members of the local Administrators group on the server with the Configuration Storage Server role have complete control over the Enterprise configuration. This means that you need to carefully review the membership of the Domain Admins group, assuming your ISA Server is a member of a domain, as well as any other group that is a member of the ISA Server's local Administrators group.

Alan Maddison is a Senior Consultant, specializing in Microsoft technologies, with Strategic Business Systems, a division of Brocade.