Testers Are Little More Than Accountants In A Factory

  • Article

The sum of these factors makes testers little more than factories churning out test case after test case, each of which is different only in some small detail (e.g., how a particular operation is invoked) than the others. Another way to say this is that testers are little more than accountants, keeping track of large numbers of myriad small details. These test cases and details together form a test matrix through which the testers march, ticking off boxes one by one. Tested that. Tested that. Tested that. Tested that. Tested that. This isn’t what gets testers excited.

Finding bugs is what excites testers. Yes, bugs will be found by marching through that test matrix. The test cases in that matrix and the test infrastructure upon which those test cases depend are only a means to an end, though, and that end is finding gnarly bugs. Testers don’t want to be accountants in a factory churning out test cases. We want to be explorers discovering bug after bug after bug. We want to be researchers developing ways to test faster, smarter, and more efficiently. We want to be scientists cooking up new ways to change the world of testing.

The feature development process we have implemented enables us to front-load our testing. The automation stack we have developed lets us write test cases in terms of features rather than UI (nicely insulating our tests from UI changes), write fewer tests while still achieving a higher-than-average coverage of execution path and test data, and decouple verification from test case execution in an intelligent fashion such that test cases can focus on execution, not verification. All of which leaves us exactly where we want to be: focused on finding lots and lots of bugs.

Finding bugs is just one aspect of testing, however. The overarching goal of testing is to produce a high quality product. The easiest, simplest, fastest, and most effective way to ensure quality is to build it in from the start. This means enabling testing of every aspect of the product development process. The techniques and technologies we have developed allow us to modify our feature definition, development, and testing process such that we can:

  • Test our specifications more thoroughly, helping the feature definition to be more solid sooner.
  • Write test cases as soon as (before, even) the specification is complete, providing a clear statement to Dev as to what the code they write should do.
  • Isolate test cases from changes to the UI, giving our PMs and designers the freedom to explore different ideas through the development cycle and allowing the UI to be locked down much later than would otherwise be possible.
  • Spend less time writing each test case, giving us time to write larger quantities of test cases as well as more detailed and comprehensive test cases.