Goals for Community Tech Previews

Mark, my boss, posted a long article that explains the intent and direction of the CTP program.  Here are some choice quotes and some of my own comments...

Our ideal is that every build is publicly available. Another way to say it is that we want to enable a scenario where a customer finds a bug, reports it, the team/developer gets the report essentially immediately, sees it’s an easy fix, fixes it, resolves the bug, the customer gets notified, and with no extra effort from anyone (beyond the customer installing) the customer can install the fixed version within a few days.

First, we need to solve the distribution problem. We’re looking at several different options – shipping media, VPC images, the p2p solution suggested earlier.

Another solution is obviously looking to reduce the size of the download a few different ways.

Second, we have to expose what we know about the build’s quality. Internally we survive by running build verification tests on every build.

Third, we need to be smart about letting the community help in communicating build quality.

Coming from the test world; what amuses me about these statements is that, if we pull it off well, the external page would actually become the preferred location for internal people trying to figure out what build to install.  Yes, we do run several very basic build verification tests and some more intense “nightly“ automation if the first round passes, but the internal problem is that a lot of this data currently stays within the separate teams that ran the tests.  I'd been pushing for a while that if we thought about having to make our tests and results publicly consumable that it would actually lead to some much needed improvements to how we think about and report our collective testing output.  Its sort of a lesson for anyone thinking about the complications of an internal process to pretend your customers had to make use of the process.  I think it will make us more successful.  It won't just be externals leaving their comments on the builds there either. :-)

Fourth, we need to be more transparent about where we are in the dev cycle, what features are coming in now, etc... In the coming months expect us to do things like cover the project status on Channel 9 and be even more explicit in blogs.

We've thought about posting bug tallies, feature completion status, specific bug communications, etc.  What type of Channel9 information would you like to see? 

Lastly, we need to give customers a dependable, effective mechanism for submitting feedback and finding out what we’ve done with it.

• We will have shifted much of the control/decision-making over how involved customers can be from us to our customers.  
• It will enable every person within our division to be much more responsive to customer feedback, without a lot of incremental extra effort.

I'll add that, personally, I'd like to see us untie the tools from the framework so that you could use stable builds from Orcas with it's new IDE features for working with the Whidbey framework.  This, IMO, would also have a customer benefit in addition to enabling more people to test each build.