Requirement Test Status Excel Report (CMMI)

  • Article

You can use the Requirement Test Status report to help determine gaps in test coverage and monitor test progress for each requirement. This report indicates how many tests are passing or failing for each requirement. This report is available only when the team creates test plans and starts to run tests by using Microsoft Test Manager. For information about how to define test suites and test plans, see Organizing Test Cases Using Test Suites.

For information about how to access this report, see Excel Reports (CMMI).

Note

You can view the Requirement Test Status report from the Test dashboard. You can access this dashboard only if your team project portal has been enabled and is provisioned to use Microsoft Office SharePoint Server 2007. For more information, see Access a Team Project Portal and Process Guidance.

In this topic

  • Data in the Report

  • Interpreting the Report

  • Updating and Customizing the Report

Related topics

Note

A team uses test cases to define both manual and automated tests that it can run and manage by using Test Runner and Microsoft Test Manager. By using Microsoft Test Manager, you can create not only test cases but also test suites and test configurations that support testing your project. You can use test configurations to define how you want to run your test cases and test suites. You can group your test cases together by organizing them into a hierarchy of test suites in your test plan. By creating test suites, you can run sets of test cases as a group. For more information, see Defining Your Testing Effort Using Test Plans.

Required Permissions

To view the report, you must be assigned or belong to a group that has been assigned the Read permissions in SharePoint Products for the team project.

To modify or customize the report, you must be a member of the TfsWarehouseDataReaders security role in SQL Server Analysis Services. You must also be assigned or belong to a group that has been assigned the Members permissions in SharePoint Products for the team project. For more information, see Grant Access to the Databases of the Data Warehouse for Visual Studio ALM and Managing Permissions.

Data in the Report

The Requirement Test Status report provides a horizontal bar chart that shows the count of results for each test case and test configuration combination that the team has defined for each requirement. The chart groups the results according to their most recent test runs, where the options are Passed (green), Failed (red), Blocked (purple), or Not Run (gray).

Requirement Test Status Excel report

The report is based on a PivotChart report that shows the most recent data that is stored in the data warehouse and that is captured for the results for the test cases that are linked to specific requirements.

Required Activities for Monitoring Test Activity

For the Requirement Test Status report to be useful and accurate, the team must perform the following activities:

  • Define test cases and requirements, and link test cases to requirements by using the Tested By link type.

  • Define test plans, and assign test cases to them.

  • For manual tests, mark the results of each validation step in the test case as passed or failed.

    Important

    Testers must mark a test step with a status if it is a validation test step. The overall result for a test case reflects the status of all the test steps that the tester marked. Therefore, the test case will have a status of failed if the tester marked any test step as failed or not marked.

    For automated tests, each test is automatically marked as passed or failed.

  • (Optional) To support filtering, assign Iteration and Area paths to each Test Case.

Interpreting the Report

You should expect the Requirement Test Status report to vary depending on where you are in your product development cycle. Early iterations will show few tests that are passing. However, you should expect most tests to pass toward the end of an iteration or product development cycle.

You can review the report to find answers to the following questions:

  • Which requirements have a low overall count of tests?

  • Which requirements have a high overall count of tests that are blocked or have never been run?

  • Does the test coverage for each requirement meet expectations?

  • Which requirements have a high rate of test failures?

  • What is the average number of tests that are defined per requirement?

  • Which requirements have a high rate of test failures?

  • Is the team running tests for each requirement?

  • If tests are blocked or not being run, does the team understand the blocking issues, and is the team addressing them?

Updating and Customizing the Report

You can update the Requirement Test Status report by opening it in Office Excel and changing the filter options for the PivotTable report. You can customize this report to support other views, as the following table describes.

View

Action

Requirement test status for an iteration

Change the filter for Iteration (default=All)

Requirement test status for a product area

Change the filter for Area (default=All)

For more information about how to work with and customize PivotTables and PivotChart reports, see the following pages on the Microsoft Web site:

See Also

Concepts

Test Dashboard (CMMI)

Requirement (CMMI)

Create and Modify Areas and Iterations

Other Resources

Excel Reports (CMMI)