Characteristics of great Beta Testers

Things have been particularly busy here for the Encarta Test team as we roll out the Beta for our next version, Encarta 2005. This seems like a good time to talk about characteristics of a good Beta tester, and is something that I would like to elaborate more on in the coming weeks.

I would categorize great Beta testers into one of two groups.

1. Power User Testers. These are the beta testers who could certainly excel if they were to choose a full-time career in Beta testing. They're naturally driven by curiosity about new technology, and enjoy pushing the latest software to it's limits. These are the same testers that tend to be very active in the Beta discussion newsgroups, and enjoy the problem solving aspects of a interesting software bug. Testers in this category are the  “power users“ that we all know who have an aptitude for learning new technology quickly.

2. Less-Technical testers. These are testers who don't care much technology for the sake of technology. Instead, they're focus is on applying technology when it's needed and useful. They've got some great suggestions about how to improve products for common people. Picture our Moms and Dads that didn't grow up using computers, but still find interesting uses for them. Some of the most compelling suggestions for feature improvements come out of this group.

As you might guess, most Test teams at Microsoft are stacked with Power Users. We can script and code our way through our most frequent tasks, and know every keyboard shortcut of our heavily used applications so we don't need to waste time grabbing the mouse. We share these traits with our Power User beta testers. While the “core“ of our Beta tests rely on the feedback and bugs entered by our technically savvy Power Users, some of the greatest feedback comes through “suggestion“ bugs entered by our Less-Technical testers. These are the people who help us see the other side of the coin- that not all of our users find it clear when we intersperse phrases like “GB” and “MB” within the same document.

Perhaps the greatest tester of all is one that is a Power User, but still maintains the ability to see software as it's used by less-technical people. I often have problems “remembering“ what it was like to first begin using computers, but I think I share that problem with most of the software industry. In the mean time, we'll be using some of the non-technical feedback from our Beta testers to help us remember.