Guidance on Microsoft Test Manager usage
VSTS | TFS 2018 | TFS 2017
Visual Studio Team Services (VSTS) and Team Foundation Server (TFS) offer both web-based and client-based solutions for manual testing:
- The Test Center in Microsoft Test Manager (MTM) client is a desktop-based manual testing solution, which has traditionally been used by testers for their Manual testing needs.
- The Test hub in VSTS and TFS is a web-based manual testing solution, which works across all platforms and with all browsers. We have invested in Test hub over past two years to provide you better experiences across Plan, Author, Execute and Track phases of Manual testing.
Because the Test hub is a fully featured Test management solution which works across all platforms and with all browsers, we recommend you use the Test hub over Microsoft Test Manager for all your test management requirements. You can use Microsoft Test Manager to test your desktop applications by launching the Microsoft Test Runner (client) from the Test hub.
This topic will help you understand why the Test hub is a more comprehensive solution for manual testing compared to Microsoft Test Manager.
Manual Testing with the Test hub
The Test hub in VSTS and TFS is a fully-featured test management solution spanning all phases of the testing lifecycle. The Test hub works on all platforms (such as Windows, MacOS, Linux, and others) and all browsers (such as Edge, Chrome, Firefox, and others). You can easily get started with using manual testing features right from your Kanban board, and use the Test hub for more advanced manual testing capabilities. This topic shows new capabilities introduced in the Test hub.
Create and manage test plans and test suites for your teams with ease with Test hub. Export and share the test plans and test suites with your team or assign multiple testers to execute tests. See below the comparison matrix to know more about the features introduced in Test hub.
|Test planning capability||Web-based Test hub||Client-based MTM|
|Create test plan|
|Add/remove tests from test suite|
|Assign individual testers to test plan/test suite|
|Clone test plan/test suite*|
|Add tests from other test suites*|
|Order manual tests within suites (RBS, QBS, Static)|
|Export test plans and test suites|
|View test case references across test suites|
|Assign multiple testers to test plans and test suites for user acceptance testing|
You can easily get started creating test cases right from your Kanban board in the Work hub. Easily add, view, interact with, and execute tests from your Kanban cards, and create multiple test cases using a grid in the Test hub. Create shared parameters and use them in multiple test cases for data driven testing.
|Test authoring capability||Web-based Test hub||Client-based MTM|
|Create and edit test cases using WIT form|
|Create and edit shared steps|
|Bulk author and edit test cases|
|Inline tests on Kanban board|
|Create and edit shared parameters|
Test your web apps and your desktop apps.
The Test hub provides a browser based Test Runner you can use to test your web apps; for example, by marking test steps and test outcomes as pass or fail, and collecting diagnostic information such as system information, image action logs, screen recordings and screen captures during your tests.
You can use the browser based Test Runner for running tests on desktop apps if you only want to mark test steps and test outcomes as pass or fail, or collect screen recordings during your tests. If you need other data collection capabilities when testing desktop apps, you can use the Microsoft Test Runner client that is part of Microsoft Test Manager. You can launch Microsoft Test Runner client directly from the Test hub.
|Test execution capability||Web-based Test Runner for testing web apps||Client-based Test Runner for testing desktop apps|
|Bulk mark tests without opening in Test Runner|
|Pass or fail tests or test steps using Test Runner|
|Inline changes to tests during execution|
|Pause and resume tests|
|File bugs during test execution|
|Capture screenshots, image action log, and screen recording during test execution|
|Update existing bugs during test execution|
|Assign a Build for the test run|
|Assign test settings|
|Fast-forward steps using action recording*|
|Collect advanced diagnostic data during manual testing*|
|Connect to a machine in an environment|
* The web-based Test Runner currently does not support Action Recording (fast-forward test steps) or Advanced Data collection (code coverage, IntelliTrace, and test impact) during your tests. You can use the Microsoft Test Runner client, launched from the Test hub, for these requirements.
You can easily track your manual testing results using your chosen light-weight chart types, and pin them to your dashboard to quickly analyze the test results. View test case result history across test suites and test plans easily by using the right-hand pane in the Test plans tab of the Test hub. You can also select a retention policy to specify how long you want to keep your manual testing results.
|Test tracking capability||Web-based Test hub||Client-based MTM|
|Test run and results analysis|
|Create, configure, and pin light-weight test result charts|
|Test run and results retention policy|
|View test results history across test suites and test plans|
Use the light weight Test & Feedback browser extension to perform exploratory testing on your web applications. You can collect rich diagnostic data such as screen captures, screen recording, and image action logs using this extension. The extension also has the capability to capture page load data for your web applications. In the Test hub you can view completed exploratory testing sessions and derive meaningful insights at team or individual level, and for a specific period.
To explore your desktop applications, you can use the Exploratory Test Runner client in Microsoft Test Manager by launching it from the Test hub.
|Exploratory testing capability||Web-based extension for web apps||Client-based Exploratory runner for desktop apps|
|Explore user stories|
|File bugs using screen capture and recording, and image action log|
|Create test cases and tasks|
|Exploratory testing session insights|
|Capture page load performance data|
Why the Test hub over Microsoft Test Manager?
As clearly shown above, the Test hub is a much richer, faster, and easier-to-use solution for manual testing compared to the Test Center in MTM. The Test hub works on all platforms and all browsers, and has a rich and modern web UI that improves your testing experience across all phases of manual testing.
All the test plans, test suites, test cases, and other test management data you create using MTM are stored in your VSTS account or TFS, so existing MTM users can easily get started using the Test hub.