The Making of Web Services Security Guide
The Web Service Security Guide has been released!
It seems like ages ago when I thought about how this whole Web services security guidance thing got started. In fact, it was ages ago…
It all began the winter of 2003. At the request of my colleague Chris Keyser, I started working on the design of a security infrastructure that an ISV partner needed to secure their Web service-based applications. This was during the dark and dreary days when WS-Security only existed as a spec and WSE 2.0 was still brewing in building 42 of the Microsoft Redmond campus.
In addition to WS-Security, WS-SecureConversation and WS-Trust have just been released as specifications. As we all know, WS-* is great from the modularity point of view. The problem was, there wasn’t a lot of knowledge that lay people can find to re-compose the specs so that they can implement Web services security infrastructures that provide features similar to what the Kerberos protocol offers for single sign-on, mutual authentication, etc.
4 months later, Chris and I ended up with a design and prototype of a Web services security infrastructure that provided single sign-on services and secure communications between Web service client and services. Unlike passive client cookie-based schemes, it has all the good stuff such as session key exchange handshake, session key derivation and renewal, stateless service support for security sessions etc. At a very high level, our architecture relied on a third party STS in the manner as shown below:
The web service client (WSC) can authenticate to the STS using username/password or X509 certificate. Once authenticated, the STS issues a token granting token (TGT) with purpose similar to the Kerberos TGT. Subsequently, the WSC can use this TGT as a proof of authentication to request for service tokens (ST) for specific Web services. We have designed the ST to carry an encrypted key that the WSC and WS can use to establish a session secret. In addition, we use the WS- SecureConversation spec to specify transition to a secure conversation token (SCT) to further optimize security performance. Once the SCT is established, the client and Web service derives encryption and signing keys to encrypt and sign data transmitted over the secure channel.
Furthermore, to improve Web service scalability in a Web farm, it would be wise not to store security state at the Web services. Chris Keyser had written an MSDN article to show how the Web services can avoid storing state for the security session.
Shortly after we concluded our experiment, I had the opportunities to meet and work with more customers to help them design and review their Web services security infrastructures. Through numerous customer interactions, I learned that most of our customers had very identical needs that could be met through the infrastructure that Chris and I had designed. With the experience I gained from designing the above infrastructure, I was able to offer each of them reasonable level of technical advice. However, what I had was a prototype, not a production quality infrastructure that comes with the complete security and administration modules that they can take with them to a production environment.
(For those wondering, you can be rest assured that the product teams at Microsoft is currently working to address this area of customer needs.)
This is when the lights bulb for Web service security patterns came on for me. Before any kind of commercial infrastructure is available, what our customers can use is better security guidance.
I’ve always been interested in security patterns and I’ve read a number of those papers on the Web. Some of them are somewhat theoretical and I’ve not seen many that have been applied to the Web services context. So I thought here laid an opportunity for Microsoft to demonstrate some thought leadership…
The Microsoft team that naturally comes to my mind to collaborate on such a pattern guidance project is none other than the Patterns and Practices group at Microsoft. My first conversation was with Ron Jacobs (who now works in my team). Together, we presented to multiple internal groups on the needs and opportunities for Web service security guidance. We knew from the very beginning that we had to be very careful with the scope of the guidance - as you already know, security is a very broad area.
We agreed at that time that we would focus on the primary patterns that we could derive from the Web services security infrastructure Chris and I had designed – which are primarily on authentication and the attributes of secure communications such as data integrity and encryption.
Shortly after, Ron took on new areas of responsibilities and the most excellent program manager I’ve worked with, Jason Hogg took on the baton to manage the delivery for this guide.
By now, it is Jan 2005. (For those who have worked in large organizations, you will appreciate why things sometimes take so long).
Although I have been working on protocols and security for a long time, generalizing the knowledge I have into software patterns still proves to be a very challenging task. This is especially so for an area like security, where the space is not just broad but the domain concepts are very much intertwined and it is not always clear if there are clean ways to define the relationships between various security patterns. For instance, signing is often used to prove knowledge of secret for authentication purposes, but the process of authentication also enables a secret to be exchanged and used for data integrity and signing purposes. Because of relationships like this, it was difficult to mine and categorize security patterns that will always please the software pattern purists. We enlisted the help of colleagues at Microsoft, external security experts and interested customers to help arrive at the pattern organization that we thought would be practical and at the same time, can withstand the intellectual rigor that pattern purists would want to impose on. We also spent a lot of time making sure that we use our terms and definitions consistently - concepts like identity provider, subject, requestor that are often used very loosely had to be tightened and crisp when used to describe patterns.
Another area that we thought we could originally pull off but had to regroup with a different strategy was the notion of a security calculator or decision tree. Based on our research with our customers, we learned that the process of arriving at a security solution for Web services is not always easy. Factors such as requirements to use existing credential store (that may be SQL, LDAP or Active Directory etc), existence of Web services security gateway/processor, need to flow original caller identities, performance, interoperability, encryption and data integrity all interplayed together and must each be considered for the desired security solution. Our project team toyed with the idea of shipping a “security consultant in the box” to help ease the customer pain. We had envision our customers choosing from a set of application scenarios and answering a few questions with multiple choice answers and then auto-generating the recommended security solution for the customers. This goal proved to be extremely challenging. After a few iterations, we concluded that the number of variables and dependencies is so great that we are better off demonstrating the usages of the patterns through a set of commonly encountered application scenarios. The readers may then peruse through the set of application scenarios and even if they cannot relate entirely with the canned scenarios, portion of the pattern-based solution may still be adopted if there is sufficient similarity in part between the example and the real world. For instance, in our Internet B2B scenario, the reader’s organization may not be able to deploy an Active Directory solution to secure the first part of the Web service interaction within the organization. Nevertheless, he may still choose to adopt the solution for the Internet interaction between the procurement Web service and the supplier’s ordering Web service, if that part of the scenario and solution plays well with his problem at hand.
In the guide, we had organized the patterns at 3 tiers consisting of architecture, design and implementation patterns. At the architecture level, the pattern captures the core spirit and primary motivation for the pattern. At the design level, there may be various design specifications that are based off the same architecture model. For instance, brokered authentication defines an authentication model using a third party to broker the trust relationships. At the design level, Kerberos, X509 certificates and an STS are based off the same model. At the implementation level, we show how the solution can be implemented using development frameworks and shipping products. For instance, showing how to use WSE 3.0 with Kerberos tokens to authenticate and secure Web service messages. The rationale behind this 3 related tiered approach is quite simple. At the architecture level, the pattern is conceptual, the context and solution should remain true independent of technology evolution. New design pattern can be added to be a child of an architecture pattern over time, as new ways of realizing the model is made possible. The implementation level is where new implementation patterns can replace old ones that are not so interesting anymore due to commercial product availability. For example, a year from now, I imagine that most people will be more interested in using WCF for securing messages rather than using WSE 3.0. This example shows the tiered pattern organization for brokered authentication:
After we shipped our first CTP in July 2005, we also added to our scope a few more patterns that address other common security design issues encountered in the Web services environment, such as when to use the trusted subsystem model vs. the impersonation\delegation model. At first glance, there may not seem to be a whole lot to the two models. It turns out that there are significant security assumptions and goals that can be accomplished with one but not the other. I am currently working on a paper that will provide more information on designing trusted subsystems. I will also blog more about the trusted subsystem topic in the near future.
Now that we have shipped the Web Services Security Guide, we didn’t forget about what got us here in the first place – the Web services security infrastructure prototype. We have been working on a SAML STS Quickstart that demonstrates how to use WSE 3.0 to implement the security profile previously described. The quickstart is a sample implementation of the STS brokered authentication pattern. Folks who are willing to invest some effort to stay in the bleeding edge should find the quickstart useful.
Finally, Jason and I will be presenting a session at the RSA Security Conference this year.
Hope to see you there!