The obligatory end of the year summary from OneNote Testing
I've been in OneNote for just over a year now. Since the purpose of this blog is describing the day to day life of OneNote testers, I want to sum up the work we have completed in the past year, how well we have performed and give an idea of where I think we are.
As far as pure testing goes, we (the test team) have completed service pack testing for SP1 for OneNote 2007. "SP" testing is not necessarily balanced evenly across the team: if you are the owner of an area which needs more work than others, you may have a larger workload. Of course, other work will get moved off your plate to make room so the work expectations on each person are still fair. The test team did very well here!
The other big testing task has been working on the next version of OneNote. We are busy trying to get the new features up and running. This is a large amount of work, and the team has handled it very well. Testing here includes what most people would generally think of testing the software, but also providing feedback on the designs, developing test plans, working with teams with whom we have heavy interactions (some examples from the past are SharePoint and Outlook), and the general overhead of any project at any company (meetings, emails, etc…).
Automation has been a big push for us. We overhauled the old UI based system to go with a white box automation model. All new work is done with our new automation system, and we have an ongoing task of converting all the old automation scripts and writing new ones for new features as well. This is the reason for overhauling the napkin math scripts. We're doing a good job here. This task will never be done. I consider it ongoing for all future releases, so it's a little tough to say how far down the road we are with it. Our automation is much more stable and we are spending far less time investigating script failures, so the system has been a big win for us.
The test team has created at least three internal tools which we all use daily to help define and track bugs in OneNote. Tool creation is incredibly valuable. While I expect everyone on the test team to be able to attach a debugger and get a callstack and memory dump when you hit a problem, other people around Office may not have the savvy to get the information in the format we need to locate the root problem. If we can create a tool that is something like "If ON hangs, click this button and the tool will email the testers a bug report" then we can give it to the nontechnical users of OneNote (e.g., "managers" like me :-) ) to file bugs.
I wish I could find out how many classes the test team has taken to learn new technologies. I'll guess around 60-100 days total for the team. I take this as a sign of some great chances for personal development and career growth. In the midst of designing software, testing it, debugging it and so on, it's sometimes easy to lose track of the big picture of "my career." It is always good to see people invest time in themselves.
We moved offices. Jokingly, I think the Microsoft average is about 18 months in any given office, then you get moved to a new location. But moving whole teams around is a big domino-like process that takes a large amount of effort and scheduling, and we usually wind up missing work for a day or two. This year was no exception.
We have continued our customer connection commitments. Each tester has chosen a role in which to meet customers of OneNote and get feedback on how testing can be improved to make OneNote even more usable. This blog is one such commitment. I've heard some great feedback about OneNote from it, especially from teachers around the world. Thanks! There are other testers who pitch in on the discussion groups, internal support aliases, work with PSS to track the most frequently reported tech support calls and similar tasks. We use all this information to ensure our testing strategy is covering the real world use of OneNote.
We've continued our side projects. Some testers have invested time into powertoys (by my count, the test team has been involved in more than 20 addins this past year - I'll post a link to all of them next), some are working on our shared source object model over at www.codeplex.com/onom, some are evangelizing OneNote internally at Microsoft and others are working on getting in depth with new technologies.
It's been a great year overall. I can't wait for
Questions, comments, concerns and criticisms always welcome,