Ten Principles of Microsoft Patch Management
By Christopher Budd, Security Program Manager, Microsoft Corporation
See other Viewpoint articles.
Patch management is a critical part of maintaining the security of your systems and network. The patch management system that you build and maintain is, among other things, the channel through which you deploy security updates from Microsoft and other vendors. Although patch management is sometimes viewed as a systems management discipline rather than a security discipline, its role in addressing vulnerabilities through the deployment of updates makes it a vital component in an organization’s security operations. Because the timely application of security updates is one of the most important and effective things you can do to protect your systems and network, your patch management system must be as efficient as possible.
To help customers develop and maintain efficient patch management strategies, Microsoft provides information about tools and strategies on our patch management page on the TechNet Security site (http://www.microsoft.com/technet/security/guidance/patchmanagement.mspx). There, you will find a wealth of important information on the nuts-and-bolts aspects of building and maintaining a patch management system to support Microsoft products. This site is an excellent and valuable resource, but in helping customers with questions and concerns around security updates for some years I have found that although the resources provide excellent guidance from Microsoft on how we recommend you do patch management in your environment, they don’t always make clear why we make particular recommendations. We have provided good resources concerning the practice of Microsoft patch management, but we haven’t outlined as fully as we could the principles of Microsoft patch management.
To help address that shortcoming, in this article I will outline ten principles of Microsoft patch management. With a better understanding of these principles, you can better align your patch management strategy with Microsoft and thus improve the efficiency of your patch management system. You can also prevent unpleasant surprises that can result from pursuing a strategy or tactics that Microsoft doesn’t recommend or support. Finally, with an understanding of the why behind how we recommend customers implement Microsoft patch management, often you will be able to answer questions that may arise in your day-to-day work managing security updates for Microsoft products.
1. Service packs should form the foundation of your patch management strategy
At Microsoft, we have consistently recommended that you view service packs as the primary vehicle for security maintenance and look at security updates as something that augments service packs.
It’s true that security updates are comprehensive for the vulnerabilities they address, and they are only released when they reach an appropriate level of quality. But service packs are a broader vehicle, both in the scope of the updates they contain and the testing process they undergo. A service pack includes as much as possible, all the security updates made available for that product before its release. The service pack also contains other updates and improvements from the ongoing work of code maintenance that individual security updates may not contain, so it always offers the overall protections you may need for you environment.
Your patch management strategy should focus first on service packs and then on security updates, rather than the other way around.
2. Make Product Support Lifecycle a key element in your strategy
Directly related to the central role of service packs should be the role of the Microsoft Product Support Lifecycle in your patch management strategy
Like all technology companies, Microsoft products follow a timeline of life cycle support. For several years now, Microsoft has worked to make this process as predictable and transparent as possible by developing and posting information about our Product Support Lifecycle at this site: www.microsoft.com/lifecycle.
From a security point of view, one of the most important things Product Support Lifecycle governs is the timeline for how long security updates for a particular product will be made publicly available. When a product is no longer publicly supported under the Product Support Lifecycle, Microsoft no longer publicly provides security updates for that product.
The Product Support Lifecycle is also relevant to our first principle regarding service packs. Security update support for products is specific to particular service packs for that product. This means that for as long as a product is publicly supported, security updates are made available only for those specific service packs of that product; updates are not made available for service packs of products that are no longer publicly supported. For example, at the time of this writing, Security Updates are publicly released for Windows XP Service Pack 1 and Service Pack 2. Security updates are not publicly released for Windows XP Gold. The Product Support Lifecycle Web site calls out timelines for specific service pack support for publicly supported products.
By keeping up to date with service packs, you ensure that your environment is always in conformity with the Product Support Lifecycle. Doing so means that you are never in a situation where security updates are released and you have no information about the vulnerable state of your environment in the security bulletin because you’re on an unsupported version.
To make your patch management strategy most effective, then, you should integrate the timelines from the Product Support Lifecycle into your patch management strategy.
3. Perform risk assessment using the Severity Rating System as a starting point
Microsoft Security Bulletins serve to make you aware that security updates are available for specific Microsoft products. Another goal of the bulletins is to help you with understanding issues in the security bulletin so that you can perform a risk assessment of the issues in your environment in accordance with your organization’s security policies. Risk assessment is an important step in the practice of patch management because it helps to answer questions relevant to the prioritization, testing, and deployment of security updates.
To help customers with risk assessment, Microsoft Security Bulletins use a Severity Rating System. With it, we evaluate each issue and quantify the issue’s impact objectively on a technical level using criteria that we have publicly posted on this site: http://www.microsoft.com/technet/security/bulletin/rating.mspx.
In the “Executive Summary” of each Security Bulletin, you will find a table that lists each vulnerability addressed in the bulletin and a severity rating for that vulnerability for each product listed as affected. For bulletins that address multiple vulnerabilities, a maximum severity for all vulnerabilities is provided for each product listed as affected in the bulletin. Finally, for ease of reference, a maximum severity for all vulnerabilities for all products is provided in the Summary at the top.
In addition, we provide more technical information about the vulnerabilities addressed in the bulletin in the Vulnerability Details section.
This information in the security bulletins is intended to support the risk assessment processes and procedures that customers have implemented as part of their patch management strategy. We provide our assessment and guidance regarding the issue that you can use as a starting point to make your own risk assessment of the issue in your individual environment. Because the Severity Rating System evaluates the issue solely on technical grounds, it cannot account for specific aspects in customer environments. Therefore, it’s important to use the Severity Rating System as a starting point for your own risk assessment that can evaluate those elements specific to your environment.
4. Use mitigating factors to determine applicability and priority
One of the pieces of technical information in the Security Bulletin pertains to mitigating factors for each vulnerability addressed in that bulletin. Provided for each specific vulnerability in the “Vulnerability Details” section of the bulletin, the information about mitigating factors explains how the impact of the vulnerability may be lessened or mitigated and is important for your own risk assessment in understanding the applicability, scope, and threat of the issue in your environment.
Mitigating factors are used in the Security Bulletins as part of the criteria in determining the severity under the Severity Rating System. For example, a mitigating factor where a particular vulnerable component is disabled by default will result in a lesser severity rating than if that component were enabled by default. One of the goals in explicitly calling out these mitigating factors is to help you understand why an issue has been rated with the severity that it has to help you with your own risk assessment process. Continuing the example, if in your environment you’ve chosen to enable the particular component by default, you will want your risk assessment to reflect that fact.
Ultimately, you should use information about mitigating factors first to determine the applicability of an issue for your specific environment. If it is applicable, you then should incorporate the information about mitigating factors into your risk assessment for prioritization of the deployment of security updates.
A very important point is that information about mitigating factors is never meant to justify not deploying a security update. In looking at mitigating factors, if you determine that a vulnerability is applicable to your system, Microsoft recommends that you always apply the relevant security update. You should look at mitigating factors as data to answer the question of when you apply the security update not if you should apply the security update.
5. Only use workarounds in conjunction with deployment
As part of the process of investigating reports of vulnerabilities, Microsoft tries to identify workarounds that are applicable to help protect against attempts to exploit the vulnerabilities that are being addressed by the security update. When we are able to identify viable workarounds to address a specific vulnerability, we make this information available in the Vulnerability Details section of the bulletin. When we are unable to identify a viable workaround, we make note of that instead.
The goal in providing workaround information for specific vulnerabilities is to give you an option that you can use to protect your environment right away, while you put the security updates through the appropriate testing before deploying them broadly. Just as mitigating factor information is never intended to justify not applying security updates, workaround information is provided as an interim measure until you apply the relevant security updates. In this way, in your patch management strategy, workarounds should be viewed as being closely tied both to your risk assessment and your deployment processes and procedures. You should consider implementing workarounds immediately for issues that you identify as posing a high risk to your environment in order to provide protections while you are deploying the updates. Issues for which there are no workarounds available may merit increased priority for deployment.
6. Issues with Security Updates are documented in the Security Bulletin Master Knowledge Base Article
A very common question from customers after the release of security updates is, “Are there any known issues with the security update?” All security updates go through an extensive testing process and are only released when they meet an appropriate level of quality, but as part of the risk assessment process administrators often want to identify any known issues.
On those rare occasions when an issue occurs after the application of a security update, that issue is documented in a Knowledge Base article by our Product Support Services. Any Knowledge Base article about a security update is also documented in the “Master Knowledge Base Article” for that Security Bulletin. Each bulletin has a Master Knowledge Base Article associated with it that is referenced in parenthesis after the bulletin’s title. Also, the “Caveats” section in the Summary is updated to call attention to the inclusion of information in the Master Knowledge Base Article.
To see whether there are any known issues with a security update, look at the “Caveats” section of the bulletin. If there is no listing, take the Master Knowledge Base Article number from the security bulletin and view the Master Knowledge Base Article on the Microsoft Product Support Site (http://support.microsoft.com). If there is no information there about known issues, it means that no issues have been identified with that security update.
7. Test updates before deployment
Testing security updates before deployment is an industry-recognized best practice. Proper testing of security updates allows you to understand the possible impact of the security update on your specific environment. You can factor that understanding into your risk assessment to prioritize the deployment of the security update.
Just as your testing process should influence your risk assessment, your risk assessment should influence your testing process. Security updates that you identify as being particularly critical to your environment may merit abbreviated or expedited testing. A security update that addresses an issue that you assess as having low risk but updates critical systems instead may merit an extended testing phase to allow for broader and more through testing.
A common question is, What should testing entail? Unfortunately, the correct answer is always going to be specific to your circumstances. The appropriate testing environment and procedures will depend on the production applications that you run, the type and mixture of system types, and the resources that you can afford to dedicate to testing. In general, though, you want to balance your costs and resources devoted to testing against the costs and resources potentially incurred by deploying an untested update in your environment.
8. Contact Microsoft Product Support Services if you encounter problems in testing or deployment
An important piece of your testing process and procedures is what to do when you think you’ve identified a possible issue with a security update in your environment. We strongly encourage that your procedures state that you get in touch with our Product Support Services organization before you do anything else.
Working with our Product Support Services is easily the most efficient way to identify genuine issues on those rare occasions when they do occur. Only when we work with you directly are we able to gather the level of technical detail that we need to understand what is actually happening. We can use that information to help you address any genuine issues that are present in your environment. Also, only by working with customers can our Product Support Services identify and document the known issues in the Master Knowledge Base Article for a security bulletin. In fact, I have gone so far as to tell customers that taking any steps to resolve a potential issue in their environment other than calling our Product Support Services only delays the resolution of their issue.
An important thing to remember is that Microsoft provides no-charge support for issues related to security updates. You can get in touch with Microsoft for security bulletin support through the Security Support Site at http://support.microsoft.com/securityitpro
9. Use only methods and information recommended for detection and deployment
Detection and deployment of security updates is a critical part of the patch management process. It’s the application of the security update, where needed, that addresses the vulnerability discussed in the Security Bulletin, and so should always be the final step in your patch management process. It’s also the piece of your patch management strategy that falls most fully on the systems management side. This means that detection and deployment of security updates are sometimes handled by a group that does not read Microsoft Security Bulletins and therefore aren’t aware of the specific information provided there. This can lead organizations to use methods of detection and deployment of security updates that have not been recommended in the Microsoft Security Bulletin.
Part of the process of building and releasing security updates is developing and verifying information for each security bulletin around detection and deployment. With that information, customers can identify systems that the security updates apply to and verify that they have been installed properly. This information around detection and deployment is included in the Security Bulletin under the “Frequently asked questions (FAQ) related to this security update” and “Security Update Information” sections. This critically important material represents the methods and information around detection and deployment that Microsoft has tested and verified to be accurate. Because any other methods or information have not been tested, Microsoft cannot guarantee that they will be accurate or reliable.
Problems concerning detection and deployment directly affect your ability to successfully apply and manage security updates, so a critical piece of your patch management strategy needs to be to use only methods and information recommended in a bulletin for detection and deployment. If you are not currently using information from the security bulletins, you should immediately investigate implanting something like Microsoft Security Baseline Analyzer 2.0, Windows Software Update Services (WSUS) or Systems Management Server 2003 to support detection and deployment. If you are using another patch management system, ensure that it uses methods and information around detection and deployment included in the Security Bulletin.
10. The Security Bulletin is always authoritative
Any time there is any question about information related to a Microsoft Security Update, the place you should go for an answer is the Microsoft Security Bulletin that accompanies that security update. Information is entered into the bulletin after it has been verified for accuracy, so you know that information in a security bulletin is authoritative. In addition, Microsoft will always put important information that is relevant to a security update in the Microsoft Security Bulletin, so you know that if you need to find information that is important about a security update, it will always be present in the security bulletin.
Sometimes we will get new information that is relevant to a security update after it has been released. When we get new information like this, we go ahead and add that to the bulletin. All changes to the security bulletins are documented at the bottom of the bulletin and include the date and the nature of the change. Anytime a bulletin is updated, the change is announced to the Security Notification Service, Comprehensive Edition (SNSCE). This list will notify you of all changes to security bulletins. If you want to keep up to date on changes to Microsoft Security Bulletins, you can register for SNSCE on the notification page: http://www.microsoft.com/technet/security/bulletin/notify.mspx.
Microsoft Security Bulletins are your single, authoritative source for the important information you need about Microsoft Security Updates.
By incorporating these ten principles of Microsoft patch management into your patch management strategy you can help ensure that your processes and procedures are working in harmony with Microsoft. That way, you help ensure that we are all working together efficiently as team with one simple and important goal in mind: helping you to better protect your systems.