Managing New Testing Efforts

  • Article

You can use the Testing Center in Microsoft Test Manager to help you plan your testing effort, based on your approach. Microsoft Test Manager lets you use to create a test plan that is associated with your team project. In your test plan, you can define which tests you plan to run for a specific iteration. Also, you can create test configurations that specify the test set up that you want to use to run your tests. By defining what tests you want to run on which test configurations, you can now use the test plan to measure your progress as soon as you begin to run your tests.

Requirements

  • Visual Studio Ultimate, Visual Studio Premium, Visual Studio Test Professional

You can create test cases for your test plan that can be used for both manual and automated tests. You can add all the information that you need in order to run a test manually by adding test steps with both actions and expected results. In addition, you can share common test steps with other tests to reduce the overhead of maintaining your manual test steps.

When you run these manual tests using Microsoft Test Manager, you can collect details of the actions that you perform both in a log and as a recording that you can use the next time to fast forward your manual testing. You can capture video of your desktop, create a snapshot, and add comments to your test results. You can also gather other diagnostic information when you run a test and save it as part of the test result. You can create bugs as necessary when you run these tests, and automatically add any data that you collect to a bug.

Using Visual Studio you can create different types of automated tests. You can create unit tests to test individual methods in your code, coded UI tests to test your UI interface, generic tests that call API methods, and load tests to check the performance of your application under different levels of stress. You can associate automated tests with your test cases to run these tests as part of your test plan.

After you have run your tests, you can report on your progress. When you use test plans to structure your testing approach, you can answer the following questions:

  • How many tests have passed or failed?

  • How many tests still have to be run for this iteration?

  • When will the testing be complete?

  • Which areas of the product have high test failure rates?

  • Which test configurations have high test failure rates?

  • Who has the most tests left to be run?

  • Can resources be reallocated to better balance the remaining testing?

  • Which build should the testers be using based on code changes and bug fixes?

For more information about strategies for testing, see Test Early and Often.

Getting Started With Test Case Management

To test software, you plan your testing strategy first and then you run your tests and submit any bugs that you find. Then you can review your progress and decide whether you may want to rerun tests, verify bug fixes, add more test cases, add test configurations, or decide whether your testing is complete for your current iteration. The following steps will help you start to use Microsoft Test Manager.

  • Connect to Your Team Project

  • Plan Your Testing Effort

  • Run Your Tests

  • Track Your Progress

Connect

You must first connect to Team Foundation Server and your team project by using Microsoft Test Manager. The team project is the same one that is used to add requirements for your application, maintain source code, and build the application that you want to test. Your testing artifacts are created and managed in this same team project. Your test results are also associated with this team project. When you have connected to this project, you can create a test plan in that project to use for your test planning. 

The following illustration shows you how to connect to your team project.

Connect to Your Team Project

Note

Only team projects that you have permission to access will be displayed. For more information about permissions, see Team Foundation Server Permissions.

For more information, see How to: Connect to a Team Project For the First Time from Microsoft Test Manager.

Plan

You have to define your testing effort using a test plan in Microsoft Test Manager. This plan can be as simple or complex as you require for your project. This test plan lets you select which tests to run and measure the testing progress.

You can specify test configurations to define the software or hardware that you want to use to run your tests as part of your plan. Then you create a test suite hierarchy in the plan. This can just be one test suite that contains all your test cases, or you can use the test suite hierarchy to provide structure to group test cases together. This structure can be grouping based on requirements or user stories in your team project. Finally, you can add manual test cases, with both action and validation steps, or automated tests into a test suite.

This following illustration shows the testing artifacts in your test plan.

Components of a Test Plan

Use the following topics to help plan your testing effort:

Tasks

Associated Topics

Planning the Configuration Matrix for How You Plan to Run Your Tests: You can create test configurations to define the software or hardware that you want to use to run your tests. You can specify default configurations for your plan and which tests you plan to run on which configurations.

Creating Your Plan: You must create a plan for your tests and add the test configurations that you want to use as your default configurations.

Adding Test Suites and Test Cases to Your Plan: You can create test suites to group your test cases together. You can create test suites based on requirements or user stories. You can also create suites by selecting existing test cases or adding new test cases. You can then add manual test steps to these test cases. You can also associate automated tests with your test cases so that you can run them from a plan.

Import Test Suites from Another Test Plan: You can import test suites from an existing test plan if you need the same test suites in another test plan.

Assigning Who Will Run the Tests: You can assign the tests in your test plan to specific testers on your team. By default, the tests are assigned to the owners of the test cases. But you can change this assignment.

Planning the Setup You Need to Use for Your Testing: You can plan what lab environments and test settings that you might need to run your tests. The environments can be standard or SCVMM environments.

Test

When you have created your plan and there is a build of your application to test, you are ready to run your tests. You can select that build as the build you are using for your testing. The combinations of test configurations and test suites that you have created in your test plan are available to be run as shown in the following illustration.

Running Manual Tests in Different Test Suites

Manual tests are run locally using the Test Runner that lets you record the outcome of each test step and save the results of your test every time that you run it.

You can use Microsoft Test Manager to set up test settings to determine how you run your tests and select what data and diagnostics you collect when you are running your tests. You can collect data and diagnostic information locally, or remotely by using test environments. When you run your tests, you can save this data and diagnostics with your results, and if you need to you can use it to create detailed bugs.

Typically, you would use an environment when you are testing a more complex application. An environment consists of a set of roles. A role specifies the purpose of a computer in the environment. For example, a specific role could be called 'Web Site for Customer Data Store'.

The environment enables you to run tests, gather data or perform system actions on machines for each specific role. A machine can be a physical computer or a virtual machine. For example, you could run your tests on one machine and gather system information about a machine that has the Web server installed for your application. Alternatively, you could run your tests on an environment that uses multiple machines and collect test impact data on those machines, and then you can also perform network emulation on the machine that is running the Web client for your application.

Three examples of scenarios for how you can set your test settings with a test plan to run your tests are shown in the following illustration.

Test Settings Concepts

Use the following topics to help you run your tests:

Tasks

Associated Topics

Setting up test machines to run tests or collect data: You can create test settings to define the roles that you require for your application under test and how to collect data and diagnostics for each role. You can use a lab environment that contains the roles in your test settings to assign the actual machines that will be used when you run your tests. You require a test controller to create lab environments. You can only create lab environments using Visual Studio Lab Management.

Running manual tests from a test plan: You can run manual tests from your test plan using Test Runner to record if each step passes or fails. The test outcome and any data that is collected when you run the test can be saved.

Speeding up manual testing: You can record the UI actions that you take when you run a manual test. When you run the test again, you can use this action recording to fast forward by playing back the action recording to the test step that you need to perform to verify a bug.

Running automated tests: You can run tests directly from Visual Studio, from Team Build, or from the command line. You can associate an automated test with a test case from Visual Studio and run that test case as part of your test plan using Microsoft Test Manager so the test results from your automated tests can be tracked with any manual tests.

Analyzing test results: You can analyze the test results for your automated tests based on a specific test run. You can file bugs based on your results. You can also review the code coverage results to check that your tests are really testing as much of your application as possible.

Performing exploratory testing: You can implement exploratory testing from Microsoft Test Manager. Exploratory testing is the testing of an application without a set of tests defined in advance. It can also be described as simultaneous test planning and test execution. During an exploratory test session, you are not restricted to a script or a set of predetermined steps as you are with a manual test.

Track

You can now track your testing effort defined in your test plan. You can check whether there are more builds for your plan, and view the recommended tests to run for these builds based on code changes. You can run standard reports, or run your own customized queries, to track the quality of the application under test.

Use the following topics to help you track your testing effort:

Key Tasks

Associated Topics

View reports to help you track your testing progress: you can view reports on your test case readiness and the testing progress for your test plan.

Finding tests to rerun based on code changes: You can compare builds to view which tests are recommended to be rerun, based on changes to your application under test.

Triaging BUGS: You can review your bugs and decide the next steps for the bug. You can also reassign bugs as necessary.

Use standard and custom queries for reporting: You can use existing queries to report on test cases and bugs. You can also create your own custom queries for reporting.

See Also

Concepts

Testing the Application