Understanding Thread Pool Enhancements
This article describes the new thread pool architecture in Windows Server® 2008. A detailed discussion on developing scalable applications using this feature can be found in the companion topic, Developing with Thread Pool Enhancements.
Thread Pool Enhancements
A thread pool is a collection of worker threads managed by the operating system to schedule and execute asynchronous operations on behalf of an application. Because the operating system is responsible for the worker threads and assigning queued work items to the threads, the developer is freed from handling the complexities of creating and destroying threads to optimize application throughput.
Windows Server 2008 includes new objects and APIs that implement the new thread pool architecture. This release continues to support the legacy thread pool architecture for backward compatibility, but developers will find that the new thread pool architecture is both easier to use and highly customizable. In earlier releases of the operating system, applications had access to a single thread pool per process. Windows Server 2008 allows applications to allocate multiple thread pools per process, with each pool scheduled independently. In previous releases of the operating system, there were two types of worker threads: I/O and non-I/O. In Windows Server 2008, this distinction no longer exists; there is a single worker thread type. The new thread pool APIs also support canceling pending work items, and simplified cleanup of resources.
The new thread pool architecture is object-based. It includes the following object types:
Thread pools
Callback environments
Work callback objects
Wait callback objects
Timer callback objects
IO completion callback objects
Cleanup groups
Thread Pools
A thread pool object is used to manage and schedule a pool of worker threads and a queue of asynchronous callbacks to be executed on behalf of an application. An application can create multiple thread pools. The number of threads in a thread pool is customizable; both a maximum and minimum number of worker threads can be specified. When the application executes, the number of threads in the thread pool at any given time will be within these limits and will depend on the actual workload being managed by the thread pool and the number of available processors in the system.
Callback Environments
A callback environment binds a thread pool to the objects that perform callbacks using the thread pool. A callback environment can also associate a cleanup group (discussed later in this article) with a thread pool's callback objects. A callback environment can be associated with a thread pool created by an application or with a process’ default thread pool. Using the callback environment, an application can specify characteristics of the callbacks that execute in the environment. For example, an application can indicated that the callback functions do not return quickly. The thread pool can use this information to determine when a new thread should be created.
Work Callback Objects
Applications use work objects to queue asynchronous callbacks to be executed in FIFO (first in, first out) order. When a worker thread becomes available, the thread pool assigns the work object’s callback to the worker thread for execution. A work object can supply application-specific data to be passed to the callback when it is executed on the worker thread. Work objects can be associated with a callback environment to bind the work object to a thread pool and its optional cleanup group. Work objects are reusable; the same work object can be used to queue multiple executions of the work object’s callback function. Each work object has a unique identifier, and each instance of the callback to be executed also has a unique identifier. An application can block until all pending callbacks for a work object have completed and cancel any pending callbacks for the work object that remain in the queue. When using work objects, the application controls the life cycle of the work object. The new thread pool APIs also provide a lightweight alternative for submitting work items to the thread pool where the thread pool infrastructure manages creating and destroying the work object internally.
Wait Callback Objects
A wait object executes asynchronous callbacks when a kernel object is signaled or a specified time has elapsed. The time interval can be specified as an absolute time or a relative time. When the callback executes, it receives a value that indicates whether the callback was executed because the kernel object was signaled or because the wait object timed out. Like work objects, wait objects can supply application-specific data to the callback, can be bound to a callback environment, and are reusable. Also, an application can block until a wait object's callbacks have completed and cancel pending callbacks that have not been scheduled.
Timer Callback Objects
A timer object queues asynchronous callbacks at a specified absolute or relative time. Using the timer object, an application can queue the callback once after a specified time interval, and an additional callback can be queued each time an optional time period has elapsed. An application can disable this periodic scheduling functionality without destroying the timer object. Timer objects also support an optional time window setting that informs the system of the accuracy requirements for the expiration time of the timer object. This window can be useful when an application is using a large number of timer objects and there is no requirement for exact accuracy in their due time. The time window setting allows the system to coalesce expiring timer objects that fall into the window together so they can be processed more efficiently.
Timer objects have the same capabilities as work and wait objects in terms of reusability, binding to callback environments, and support for blocking until timer callbacks complete and cancellation of pending timer callbacks.
IO Completion Callback Objects
An IO object executes asynchronous callbacks when an asynchronous I/O operation completes. An IO object associates a file handle with a thread pool so asynchronous I/O completion notifications are processed by the thread pool’s worker threads. The callback receives the overlapped structure supplied with the initiation of the asynchronous I/O operation, the outcome of the operation, and the number of bytes transferred. IO objects have the same capabilities as the other callback objects in terms of reusability, binding to callback environments, and support for blocking and cancellation of pending IO callbacks.
Cleanup Groups
A cleanup group is responsible for freeing resources used by the callback objects that are using a thread pool. If an application does not associate a cleanup group with a thread pool's callback environment, the application is responsible for freeing each callback object that uses the thread pool. By associating a cleanup group with a callback environment, an application can free all of the callback objects bound to the callback environment using a single function call.