tasting fresh dogfood

As I mentioned before, we’re in the process of updating our internal version of VSTS with updated bits. We made good progress last week. For instance, Monday night we resolved all known dogfood blocking issues. I emphasize the word “known” because this really means that we’ve cleared out the backlog of issues but doesn’t mean we’re ready to go. Instead it signals that we’re ready for important next steps including branching, a mini-test pass and dry runs all of which are currently underway. 

Branching: we've created a separate source branch for the dogfood refresh sources. This allows us to minimize the source churn which in turn helps limit inadvertent regressions. In this branch, we only permit fixes that address dogfood blockers. After we go live, we have a place to fix showstoppers without worrying about other destabilizing changes.

Mini-test pass: we’ve started a two week mini-test pass. While it’s not specifically focused on finding dogfood blocking issues, they’re certainly found from time to time. Anyone that finds a dogfood blocker is instructed to enter the bug, mark it against the “dogfood refresh” iteration and to send mail to our triage team explaining why they think it should be fixed in the dogfood branch. 

Dry runs: this is an activity solely focused on determining readiness for going live with this updated version of VSTS. In addition to using the latest good build, we migrate all of our existing data into the system. This helps expose issues in the migration scripts as well as issues only apparent when we have large amounts of data. And we’re talking a lot of data…36,000+ work items, 298,000+ files, representing 700+ workspaces and 1,400 shelvesets. We usually iterate through the bug bashing / dry run process. Typically it takes 3 dry runs to get things ready to go live and we’re wrapping up round #1. 

I spent an hour or so on Friday afternoon using the results of the first dry run. Compared to other first dry runs from the past (this is our 4th major update), this dogfood tasted pretty good. It’s certainly not ready for prime time yet but I was able to do many of my most common operations with the system without too many problems. Of course before we roll this out to hundreds of folks, we need to clear up these and other blocking issues so that we don’t impede productivity more than absolutely necessary. 

So, we've got so more work to do but we're certainly getting closer. After we go live with this refresh, we'll share the final dogfood build with you as a CTP so that you can let us know how you think it tastes.