The Secret of LINQ Design

A team within Microsoft ran an "app week" recently to build applications that implement customer scenarios using a variety of LINQ technologies.  The feedback on LINQ to XML was uniformly positive. The participants were not XML geeks, but more like our target audience: developers who come across XML now and then and need to deal with it without undue fuss. This group found that they were immediately productive without having to read the documentation, because the APIs worked the way one intuitively thought they should.  This set off an interesting internal thread speculating on why LINQ to XML was successful in achieving its design objectives in a world (and to be honest, a company) where this is not the norm.

I saw The Secret of Apple Design and was struck by some similarities with how LINQ to XML came about. The article quotes extensively from design guru and former Apple exec Donald Norman.  For example:

"[Jobs] had a single, cohesive image of the final product and would not allow any deviation, no matter how promising a new proposed feature appeared to be, no matter how much the team complained. Other companies are more democratic, listening to everyone's opinions, and the result is bloat and a lack of cohesion.

Thie quote points to the first similarity -- the existence of a designer with the skill to make hard decisions and the authority to make them stick. In LINQ, the Steve Jobs role is played by Anders Hejlsberg. Superficially, the LINQ design teams  look a lot like W3C working groups, with regular meetings where requirements are interpreted and fine-tinued, and proposals for alternative ways of achieving the requirements are presented and discussed, sometimes for months.  In sharp contrast to the designed-by-committee XML technologies such as DOM, however, LINQ has an architect who makes decisions, not a chair who drives consensus.  In the LINQ design meetings, all opinions were considered, and previous decisions open for reconsideration, but there is no pretense of democracy.  When it came time end a discussion, there wasn't a vote, or weasel-wording to paper over disagreements, there was a decision.

The quote also highlights the importance of conceptual integrity. Anders began what is now called LINQ to XML with a clear vision for how a modern XML API could work, with C# functional constructors and LINQ queries providing an essentially functional approach to working with XML.  Lots of ideas were proposed, and lots of people complained when they were removed or rejected, but the goal of internal cohesion and alignment with LINQ was paramount.  Of course, XML reality intervened in various unavoidably ugly ways, but much design effort was put into minimizing and isolating the ugliness.

The third similarity I noted is an insistence on simplicity over feature richness.  Quoting again from the article:

"The hardest part of design keeping features out." Simplicity, [Norman] says, is in itself a product differentiator, and pursuing it can lead to innovation.

Rolston agrees. "The most fundamental thing about Apple that's interesting to me," he says, "is that they're just as smart about what they don't do. Great products can be made more beautiful by omitting things."

 A year ago, I wrote a post on what LINQ to XML would not do, such as guarantee that a tree in memory is well-formed, support XML 1.1, offer more than bare minimum support for DTDs, round-trip entity references, and so forth.  Likewise, the design team decided to remove the streaming input API that had been extensively discussed and prototyped.  In many such cases, Anders decided that the functionality benefit wasn't worth the complexity cost.

Of course, the real secret to successful design is the skill and tenacity of the designers and the implementers; giving a chief designer the authority to purse cohesiveness and simplicity just enables success, it doesn't assure it. Likewise, LINQ's technical success doesn't guarantee real world success. For example, for all  its aesthetic flaws, DOM has been an undeniable success story in helping make XML and dynamic HTML reasonably usable by developers, and has helped the world move from mostly proprietary to interoperable data formats over the last 10 years.   It is possible that the same ugly compromises that promote committee consensus actually help overall adoption.  Likewise, just as Apple's design aesthetic pays off because it appeals to an upscale market willing to pay a premium for form and fashion on top of functionality, perhaps LINQ to XML will appeal mostly to a niche in the developer community.  We shall see ... but I bet a couple of years on this and I'm convinced that this technology will be widely adopted on the Microsoft platform and will spark a round of competitive innovation on other platforms.