Carrying around the weight of the past

I've been having an incredibly interesting email conversation with Michail Arkhipov (a Dev lead on the Web.Net) team and I'm pretty torn on how I feel about it.  A little background might be a little helpful to give some context to the discussion.

There is a whole heck of a lot of lines of source code inside VS and MS as a whole.  Millions upon millions of them.  It's taken as a given that for the company to be successful, developers need to be productive, and for developers to be productive, managing all that source and the process of producing the binaries from the source must happen very smoothly (in a perfect world transparently).  In order to facilitate this there are full time employees and teams out there that work to make that happen, and to make sure that the “build” is always getting produced day in and day out.  This is not an easy job to accomplish.  There are teams who are actively working on one part of VS (like our team and the C# component) and there are teams that need to use the work we produce in order to do their own work.  For example, we code some of work in C++ and so we need a working C++ compiler in order to function.  A lot of the Team System code is written in C# and so they need our code to work.  Then there are large teams like the runtime whose work ends up impacting everyone.  When those changes are finally batched up and applied to our branch it ends up conflicting with the changes we've been making.  Those conflicts need to be identified, resolved, and propagated out so that the build can proceed.  

In order to make this possible all (or at least much) of MS is standardized on a common build system we call NTBuild.  I don't know the history of it, but judging by the name, I assume it came about when they were building NT (so potentially nearly 10 years old).  The system has many flaws and faults, but it works and there's a lot to be said for that.  Unfortunately, there's an issue for our team when it comes to this.  Specifically, C# projects now work with the next-gen build system that MS has (appropriately called MSBuild).  In order for things like C# intellisense to work you need to have one of these projects set up.  This is a concern for other teams inside MS because it means they now have to support two build systems.  The NTBuild system that the entire organization uses (and which is supported by a full time staff), and the the MSBuild system which they would need to take care of themselves. 

Because of this Mikhail was requesting that our team do work to support source files that aren't in a project.  He raised many very good points (and hopefully I'll be able to present them here).  The primary issue was that if we did not do this we would end up with teams inside MS not being able to dogfood our product because they would lose so much productivity trying to support this sort of dual system.  I can appreciate that concern.  However, it raised several issues for me, including:

  • Are we solving a purely Microsoft problem here?  Do our customers benefit from this?  Would they benefit more from us working on something else?
  • If we can't get teams internal to switch over to this new terrific build system which we want our customers to use, then how can we be sure that it really is going to be good enough?
  • Are we doing ourselves more harm by sticking with the existing ancient system?  Should we be moving to newer and better ways to handle this process?

There's a lot of information about MSBuild out there, so I won't repeat it.  However, there is one really nice thing about it's integration in VS2005.  Specifically, when you do a build from within the IDE you get the same build system as when you run MSBuild externally on the project file.  So we have a system now that will allow you to build internally and run from your existing build system without any changes, and which will produce the exact same results.

So what do you think?  Is it worth adding that feature that will allow C# to work outside of a project?  In your own organizations (or on your own) do you find yourself wishing for this feature?  Or are you ok with the support we currently have.  If you're not ok, would the new MSBuild system solve your problems, or do you need support that goes far beyond that.  Do you think that this is just an MS issue and that we should fix our internal processes rather than adding more complexity to an already complex tool?

If you are looking for more then let us know what your build systems are like so that if we do this we can try to make sure that this won't be solving a purely MS problem but also yours as well.