Didn’t you get the memo about the TPS reports?

Today my inbox was flooded with opinions about the VS Data team status post that was inspired by the MSBuild status post. These come to me because a large portion of my job currently revolves around figuring out and encouraging how Devdiv participates in the community. I spent too much time addressing concerns today not to share my responses publicly.

Opening Remarks

What keen observers are witnessing is a division in the midst of a transition towards a more transparent and customer connected future. Transitions can be painful, and a lot of people fear change like the guy who refuses pants in the old Quiznos ads. This transition has been one with a lot of experiments. Some stick, some don’t, and some are refined into something different but better. The old adage says crawl, walk, and then run. We are heading into the walking phase of our “community” efforts.

Sharing Status

The concept of sharing status is something that happened organically. No one was asked to do it, but people have been trying it for themselves and for their teams. Like Mike Gunderloy I do see the potential upside and ability to refine this concept. Of course there are downsides and risks if not done right. Most of the negative customer comments I’ve seen suggest that there is a need for more education to help with the translation and the warnings that “once execs get wind of what you guys are blogging about heads will roll”.

Since I’m one of the people leading teams towards transparency I suppose that would be my head. I might even be a little worried if I hadn’t been told on several occasions that there is executive support for what we are encouraging and I wasn’t being internally transparent about what we are working on. Following are some of my responses to the concerns I’ve heard internally.

Don’t we need to provide some sort of education to go with these reports?

I agree that unknown meanings and numbers without context could scare customers and potentially do more harm than good if not properly explained. The great example on the VSData team post is a term that was unknown to an internal person until another internal person responded. How is that for cross group collaboration? :)

Customer education will be a big part of what we do. For example: I don’t expect to throw up a number like “we shipped with 60k known bugs” without also providing the context for users about what this number means. Trust me, customers know we ship with bugs and with more public feedback mechanisms available they will also be able to tell exactly how many of their own reported bugs we ship with. What I want to do is help people understand what it means to ship a large product and humanize the process a bit more.

Shouldn’t disclosed information only come from one source?

I expect us to provide both a single source and multiple sources in the future. Everyone will continue to have their own blogs and outlets where they can share and have been sharing their personal accounts of working on their product. We are also hoping to unify these together into one feed/voice/location at some point. It’s possible that statusreports.microsoft.com is not too far off.

I don’t see the customer or Microsoft benefit for all this transparency.

I’ve been told that neither the internal teams nor the customers would benefit from the release of such information. This may be partially correct. To some (probably large) percentage of our customers this information will not be immediately useful. However… It will exist as a backlog for the “Why did you make this choice?” questions after we ship, and before we ship it will be useful for that set of customers that live for future products and do a lot of influencing the everyday customers on what tools they should be using. There’s two, here are some more:

Better (Real-time) Expectations Setting: Having this information become more public will help continually set customer expectations better than “surprise” announcements like “Whidbey won’t ship until 2005” that only lead to disappointment. It can help explain delays more in a fashion that developers will understand and appreciate better.

Competitive Advantage: Eclipse is a competitor in our marketplace. Just about every bit of their schedule information and planned feature work is readily available for us and the community to read. They aren’t losing ground because of it. Customers tell us that this type of openness makes them trust the platform and the team delivering the platform better. We need to “keep pace with the jones” in the developer world and maybe do them one better. This does not mean giving away all of our plans. Status and bug metrics being published does not imply all planned feature work. For example: No one mentioned VSTS work until it was announced.

Humanization: All part of the general need for “lifting the covers” so that customers can see what really happens here and letting people see how excited teams get around large milestones about the awesome products they are building.

We’re going to have to explain stuff anyway: We will have better feedback mechanisms and issue reporting in the near future. Customers will be able to query that database and see for themselves what is happening to their requests. We absolutely need to set expectations and be open about the what, how, and why questions that will come from their access to this information.

What if there is conflicting information from different Microsoft sources?

We are working to make sure some things go through some sort of central channel, but you have this problem today regardless of the community medium. The problem exists today in newsgroups where one MS person replies, then an MVP replies, then another MS person replies all saying three different things. Alex has a great comment here that can help with this: “People *should not* blog about things that they don't own.” His post also does a great job of dispelling a few other fears and I would recommend anyone read it. Individual voice is still a necessity. It does not; however replace entirely the need for press releases and central marketing messages. They can co-exist just fine. I might also suggest that someone with this fear read more from D. Weinberger who was recently in town. He is really the expert on this. 

Won’t customers be disappointed if they see more of our early plans and we don’t deliver on everything?

You own your voice. Don’t set expectations to high. Don’t make promises you know you won’t be able to keep. Look at what Cyrus is doing for the C# team. He is simply holding a conversation with people that love C# about what features they would like to see in the next version. He is not sharing a vision, complete product plan, or making promises publicly he is just having the conversation with people that share his passion for C#.

I do agree that this is something everyone should be careful with when it comes to any customer interaction. Part of our general community strategy that I’m encouraging teams to follow is to keep the list of people you work most intimately with small when it comes to sharing the most intimate secrets. Public expectations play a part in that choice to keep things small.

If you think of the degree to which we are transparent as a black box that customers can’t see inside of then we are indeed trying to shrink the size of the box. Publicly we intend to shrink it by X, with MVPs we intend to shrink it X + Y, and for some partners we intend to make what they don’t see very small indeed. Becoming more transparent does not imply we therefore must turn over the product source code to the community or be 100% transparent.

And if you are having conversations with your customers as you go, then they won’t be totally upset in their disappointment because they will understand the thought that went into a choice. Most likely you also learned what their priorities were through the conversations and implemented what was most important.

And just what are OGF Reports?

Mike, if you are still wondering… you can think of OGF as “Overall Goodness Factor”. Typically it is the thumbs up/down from the QA teams where they report on product quality beyond the mere test results. It’s a more “IP required” report from test team members.