Testing with Real Users - User Interaction and Beyond, with Online Experimentation

My talk at the Better Software Conference was a lot of fun. It was well received. If you want to check out the slides, take a look here.

Evidence shows than more than half of the ideas that we think will improve the user experience actually fail to do so—and some actually make it worse. Instead of guessing, why not measure what your real users like and don’t like? Controlled, online experiments (A/B tests being the simplest version) are a proven way to make data-driven decisions about what works and what doesn't. Seth Eliot shares numerous examples of online experimentation within Microsoft to test new user interfaces with their customers. Seth shows how special frameworks, such as Microsoft's ExP (Experimentation Platform), can also move testing into the high-value realm of testing-in-production. In addition to new features and designs, Microsoft tests the impact of new code in production. By employing online experimentation, you can control how and when new, potentially dangerous code is exposed to users. Exposure control enables you to reap the benefits of testing in production while limiting the potential negative impact on your customers and users