Serving The WebGet It Right The First Time With IIS 6.0
Server administrators are always on a quest for the secret formula that will solve their performance woes. But no matter what you may have heard, there is no magic pixie dust. Yet the promises keep rolling in, even though few results are delivered. There is no bulletproof mechanism for enhancing server performance.
IIS 6.0 and Windows Server™ 2003 comprise the Microsoft® Web serving platform. While IIS 6.0 has powerful features that improve security, architecture, manageability, and even troubleshooting, it takes administrator or developer intervention to pump up performance. Out of the box, it is tuned to serve many thousands of static requests, but dynamic content is what drives the Web. So high-performance dynamic content is a serious goal.
Simply put, IIS 6.0 can't do performance alone. It's up to you, the IIS 6.0 user, to understand the significant architectural and platform improvements Microsoft made when redesigning the Web server for IIS 6.0 and Windows Server 2003, as well as understand how to exploit these changes to enhance performance.
The IIS 6.0 architecture consists of three interdependent components: IIS Admin Service, W3SVC service, and the worker processes (W3WP.exe). Unlike IIS 5.x (and earlier versions), IIS 6.0 eliminates any confusion about which component is not performing optimally. In IIS 5.x, it was possible that code not written by Microsoft could be executed inside the Web server's core. In the IIS 5.x architecture, the core Web service (W3SVC) and user code in the form of ISAPI filters and extensions would run within the main process called Inetinfo.exe (see Figure 1). Besides the obvious security concerns, this also posed a dramatic risk to the overall health of the server by allowing code not properly tested in the IIS 5.x core to cause exceptions and fail.
Figure 1** IIS 5.0 Architecture **
In IIS 6.0, the W3SVC service is separated from any user code (see Figure 2). W3SVC now executes at all times in a process that is hosted by SVCHOST.exe, which does not allow any custom code to be loaded or injected into it. This process is responsible for two tasks: configuration and process management. It does not serve any requests.
Figure 2** IIS 6.0 Architecture **
IIS 6.0 improves reliability by using W3SVC to monitor the health of worker processes that host Web applications. Unlike IIS 5.0, which depended on external sources for monitoring such as Application Center 2000, W3SVC either maintains a certain level of health or else restarts the worker process.
Application Pools: The Building Blocks
The fundamental activities for maintaining the health of Web applications are the responsibility of the W3SVC service. In IIS 6.0, this service owns and manages a collection of "application pools." These application pools create a boundary between groups of applications or even two applications. In a nutshell, an application pool contains a single application or a collection of applications with the goal of isolating them.
The Web server governs apps in an app pool with a single set of criteria and determines overall health. Specifically, it determines the health of the host processes, which has the secondary effect of helping to ensure good application health. Application pools can be restarted or recycled based on different criteria. The criterion chosen is based on overall health parameters or performance characteristics of the hosted apps.
For example, DefaultAppPool is the single application pool installed on Windows Server 2003 when IIS 6.0 is configured. This application pool is fairly "reserved" by being set to recycle at 1740 minutes (every 29 hours). It is also set to ensure maximum resource use by shutting down after 20 minutes of no new requests (and no currently running requests), thus ensuring that resources are freed for the system or other services to use. This is important, because application pools should group apps with respect to recycling, health, usage, and performance characteristics.
Application Pools and Web Administrators
Performance tuning with IIS 6.0 takes planning and work; there is no one formula that will work in all circumstances. Developers can program for IIS using a variety of supported technologies such as ASP, ASP.NET, and other ISAPIs (for Perl, PHP, or Cold Fusion, for example). No single solution can remedy Web server performance problems because there is no single cause.
For example, tweaking the performance of a server running mostly Cold Fusion applications is much different than tweaking an ASP.NET application written in C#. To apply one practice to all situations is to risk unexpected and undesirable behavior.
A framework, however, can and should be applied to ensure maximum reliability and overall better performance. The framework depends almost entirely on the core Web server and very little on the actual code being executed.
It is very important to understand that frameworks can be applied at two independent layers: the core Web server layer or the application layer. IIS 6.0 helps you with your reliability and performance efforts by letting you tweak, optimize, and customize the framework at the core Web server layer.
The first step in performance tuning is to understand the different avenues by which an application pool might be restarted (recycled). The three possible options are: request-based, time-based, and resource-based.
Request-based recycling means applications are recycled according to the number of requests the server receives (often called "load"). This typically produces a seemingly random recycling behavior.
Two varieties of time-based recycling are available; the administrator must determine which is best for a given environment. The first is based on total minutes of run time (the default) while the second is schedule-based over a 24-hour period. A good example of when scheduled-based recycling is useful is an application pool serving a timecard application that requires all employees to enter their work hours for the previous day into a Web application. In this case, it might be beneficial to set the recycle to occur every morning at 3:00 A.M., thereby releasing any used resources and starting fresh for the new day. This would maintain the health of the application by ensuring that it is fresh when it goes into service. Note, though, that recycling this hypothetical timecard application would not be a good idea if it made extensive use of in-process caching. In many scenarios, caching is critical for good performance and the cache takes time to build up and use. The in-process cache is lost when recycling is done, however, so the cache built up from previous days would never be used.
For this reason resource-based recycling, which uses memory consumption as the basis for restarting, was included in IIS 6.0. If the timecard application were written in ASP.NET, for example, it would be a good move to disable any request-based or time-based recycling and choose resource-based recycling. ASP.NET, specifically the common language runtime's garbage collector, maintains its own health by ensuring that unused memory is released. ASP.NET is often capped at a memory limit, thereby allowing applications to run as long as their health is maintained within the individual application domains.
When too much memory is being used, IIS 6.0 can restart the process. A good starting point for application pools hosting ASP.NET apps is to set the PeriodicRestartPrivateMemory limit to the minimum of 800MB or 60 percent of physical RAM, whichever is smaller. This helps to ensure that the worker processes are recycled before ASP.NET applications experience out-of-memory exceptions.
Before choosing the appropriate recycling strategy, you should first decide exactly what applications will be installed on the server and then map out a strategy for implementing application pools. This often entails breaking down applications according to criteria such as:
- Business units (HR, IT, Marketing, for example)
- Application types (ASP.NET, PHP, CGI, or static files only)
- Application demands (such as CPU or memory)
- Application behavior (putting misbehaving applications in one group or pool and well-behaved applications in another)
By default, IIS 6.0 starts with a single application pool designed to own all applications, and these applications will execute within the same worker process. This can lead to failures or bottlenecks as applications are forced to share resources.
Above and beyond the health of applications, the benefit of the application pool architecture is code isolation. Sandboxing apps by isolating code has long been a best practice, but IIS didn't do this until the current version. IIS 6.0 is capable of running many concurrent and completely isolated app pools. In fact, no memory is shared between worker processes running on the same IIS 6.0 server, not even ISAPI filters, extensions, session data, or connections.
The W3SVC service is responsible for configuring the HTTP.sys kernel driver and starting the worker process. It does not host any Web applications or serve any requests.
W3SVC first creates application pool queues in HTTP.sys. It then instructs HTTP.sys to listen for the URLs configured in the metabase and tells it how to route the URLs and which app pool to send them to. This all occurs before the first request to IIS 6.0 is served, thus starting at the connection layer (HTTP.sys), no application data is shared. The request queues are unique and tied to pools. At the time of the first request, HTTP.sys notifies the W3SVC service that a new request is pending for an application pool. The W3SVC service then starts a new worker process, which then pulls the request from HTTP.sys, executes it, and returns the response to HTTP.sys. To recap, at run time and only on the first request the following occurs:
- HTTP.sys notifies the W3SVC service that a pending request has been received for a given application pool and that there are currently no processes serving that application pool.
- W3SVC creates the new worker pro-cess for that pool.
- W3WP.exe (worker process) opens the application pool in HTTP.sys, pulls the request for that pool from HTTP.sys, and executes it.
- W3WP.exe returns the response to HTTP.sys.
This provides a fail-safe architecture for processing requests that are presented to IIS 6.0 by removing the single-process bottleneck. In versions of IIS prior to 6.0, all new requests, regardless of the URL, had to be routed through the Inetinfo.exe process prior to execution.
Ping: Are You There?
Every administrator at some time has used the ping.exe utility to verify that communication was established between two machines. This is a tried–and-tested method for ensuring that each machine is configured for network communication. Why mess with what works? In IIS 6.0, a simple communication channel that allows status data to be communicated is opened between the W3SVC service and each worker process. This status data describes the overall health of the worker process. This kind communication is known as "pinging" in the IIS 6.0 IIS Manager. Pinging, enabled by default, allows the W3SVC to check that the worker process is able to do work on requests, and health reporting piggybacks on top of this mechanism. This feature is configurable, but Microsoft strongly recommends that administrators do not disable this feature.
IIS 6.0 has two options for recycling: overlapping (the default) and non-overlapping. To maximize availability of Web applications, the W3SVC service by default starts a new worker process while the unhealthy worker process continues to churn along. When the new worker process is fully started, W3SVC notifies HTTP.sys to start routing the new requests to the new process. When this notification is complete, the old process is allotted a time to shut down (90 seconds by default), otherwise the W3SVC service terminates it. This overlapping process ensures that the client's experience with the application is maximized by removing a potential bottleneck and starting a new process.
Building Reliability from the Ground Up
The ability to recycle is useful for various reasons, but many applications are simply sick from the start. An application, or a set of applications, that repeatedly perform poorly are detrimental to the server's overall health. With IIS 6.0, an application pool that often causes worker process crashes is marked as unhealthy. Multiple failures over a short period of time could be exceptionally bad for your overall Web server health due to the repeated starting of new processes. IIS 6.0 protects against multiple failures by including a special feature called "rapid-fail protection."
This feature eliminates the application pools that crash but are not dealt with by administrators and developers. Plus, it also prevents infinite crash and restart loops. An application whose worker processes fail five times in a five-minute time span (these are the default values, as shown in Figure 3) is considered unhealthy and marked as unusable until an administrator has manually restarted the application pool. IIS 6.0 cannot always detect the reason for failures, but it does attempt to minimize the amount of CPU or memory these applications take from other well-behaved applications and application pools. The beauty of rapid-fail protection is that it allows the administrator to configure both the number of failures and the time period in which the failures can occur. You can do this in the IIS Manager or by using your favorite command-line script.
Figure 3** Application Pool Properties **
Rapid-fail protection is unique in that it doesn't apply to recycles based on requests and resources. Instead, an application pool is placed into rapid-fail protection if the IIS 6.0 W3SVC fails to ping the worker process, if the worker process crashes, or if its startup or shutdown time limit is exceeded. This is a sign to the service that the worker process is no longer available and for health reasons should be shut down. When a pool is placed into rapid-fail protection, W3SVC actually stops the app pool, and HTTP.sys returns 503 responses to all queued requests and to all new requests to that pool.
The easiest way to see IIS 6.0 go into rapid-fail protection is to use the Windows® kill.exe utility to terminate the process (without notifying W3SVC) and restart it. After five consecutive failures, IIS 6.0 no longer creates a new worker process and returns a 503 Service Unavailable error to the unfortunate user. To see rapid-fail protection in action, follow these steps:
- Open a browser and make a request to the IIS 6.0 server using http://localhost.
- In Task Manager, click the Processes tab and highlight W3WP.exe. Then click Kill.
- If you repeat these steps four more times, W3WP.exe will fail to load again.
It's a Wrap
IIS 6.0 moved the management overhead from the customers to the W3SVC service while also offering a high level of flexibility. It offers administrators the ability to use a variety of solutions for maintaining the health of their services and offerings, and can be expanded or contracted as needed to suit a particular implementation. The key to performance and reliability is the ability to remove unused resources and recycle the resources that are affecting Windows Server 2003 overall health.
The ability to recycle allows customers to approach a true "shared" hosting environment. Whether you're an intranet site host or a service provider, IIS 6.0 can effectively host thousands of Web sites with many hundreds of applications while continuously maintaining effective management and performance capabilities. The architecture of IIS 6.0 is designed to use resources efficiently while also allowing for maximum client satisfaction by maintaining the internal health of all currently running processes.
Chris Adams is the Web Platform Supportability Lead at Microsoft. Chris is currently certified as a MCP, MCSA, and MCSE on Windows NT 4.0, Windows 2000, and Windows Server 2003. Reach him at email@example.com.
© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.