Create a query based on build and test integration fields

  • Article

TFS 2018

Work item fields that support build and test integration support the following actions:

  • Associate bugs with the builds where they were found or fixed
  • Query for bugs associated with a build
  • Mark test cases as either manual or automated, and store information to support automated test cases
  • For test cases and shared steps, define the action and validation steps and the data that are used to run tests.

Supported operators and macros

Most build and test integration fields have a data type of String, PlainText, or HTML. Query clauses that specify a text or rich-text field can use the operators and macros listed in the following table.

Data type

Supported operators and macros

Rich-text (HTML) and
Multi-line text strings (PlainText)

Contains Words, Does Not Contain Words, Is Empty, Is Not Empty.
The Is Empty and Is Not Empty operators are supported for Azure DevOps Server 2019 RC2 and later versions.

Single text (String)

= , <> , > , < , >= , <= , =[Field], <>[Field], >[Field], <[Field], >=[Field], <=[Field], Contains, Does Not Contain, In, Not In, In Group, Not In Group, Was Ever
Macros: [Any], valid with the Work Item Type field; and @Project, valid with the Team Project field. The system automatically defaults to filtering based on the current project. For more information, see Query across projects.

Useful filters

Filter for

Include these query clauses

Automated test cases

        Work Item Type = Test Case And Automation Status = Automated

Query-based test suites

        Work Item Type = Test Suite And Test Suite Type = Query Based

Requirement-based test suites

        Work Item Type = Test Suite And Test Suite Type = Requirement Based

List bugs and the test cases that test them

Open a new query, set the query type to Work items and direct links. Filter for bugs in the top level and add the filter for Test Cases in the linked work items filter.

List bugs and the test cases that test them

Note

You can't construct a query that shows a hierarchical view of Test Plans, Test Suites, and Test Cases. These items aren't linked together using parent-child link types. You can view the hierarchy through the Test>Test Plans page.

Build and test data fields

The following table describes the fields that are defined in one or more of the test WITs. For information about data types and field attributes, see Work item fields and attributes.

To customize a field or picklist, see Add or modify a field to support queries, reports, and workflow.

Field name

Description

Work item type

Automation Status 1

The status of a test case. You can specify the following values:

Test Case

Found In 2

Product build number, also known as a revision, in which a bug was found.
Reference name=Microsoft.VSTS.Build.FoundIn, Data type=String

Note

You can also use the Found in build link type to link a work item to a build. This link type is available from Azure DevOps and only works with the current build processes (not XAML builds).

Bug

Integration Build 2

Product build number that incorporates the code or fixes a bug.
Reference name=Microsoft.VSTS.Build.IntegrationBuild, Data type=String

Note

You can also use the Integrated in build link type to link a work item to a build. This link type is available from Azure DevOps and only works with the current build processes (not XAML builds).

All

Issue

Indicates that the Shared Steps are associated with an expected result. Allowed values are Yes and No. Reference name=Microsoft.VSTS.Common.Issue, Data type=String

Shared Steps

Parameters

Contains the parameters to use when running a manual test.
Microsoft.VSTS.TCM.Parameters, Data type=HTML

Shared Parameters, Shared Steps, Test Case

Steps

The action and validation steps that are required to run the test. Microsoft.VSTS.TCM.Steps, Data type=HTML

Shared Steps, Test Case

System Info

Information about the software and system configuration that is relevant to the test.
Microsoft.VSTS.TCM.SystemInfo, Data type=HTML

Bug, Feedback Response

Repro Steps (or Steps to reproduce)

The steps that are required to reproduce unexpected behavior. Capture enough information so that other team members can understand the full impact of the problem and whether they've fixed the bug. This includes actions taken to find or reproduce the bug and expected behavior. Reference name=Microsoft.VSTS.TCM.ReproSteps, Data type=HTML

Bug

Test Suite Type 1

The test suite category. Allowed values are:

  • Query Based: Use to group together test cases that have a particular characteristic - for example, all the tests that have Priority=1. The suite automatically includes every test case that is returned by the query that you define.
  • Requirement Based: Use to group together test cases designed to track the test status of backlog items. Each test case that you add to a requirement-based test suite is automatically linked to the backlog item.
  • Static: Use to group together test cases with any characteristics or test suites.
    For more information, see Create a test plan.
    Reference name=Microsoft.VSTS.TCM.TestSuiteType, Data type=String

Test Suite

Note

  1. Do not customize the pick list for these fields. The system accepts only those values listed.
  2. By adding a GLOBALLIST element to the FIELD definition, you can provide a drop-down menu of builds that users can choose from. To learn how, see Builds and global list auto-population later in this article.

Other fields

The following fields don't appear on work item forms, but these fields are tracked for test cases or test suites. You can use some of these fields to filter queries and create reports. (None of these fields are added to the data warehouse nor indexed.)

Field name

Description

Work item type

Automated Test Storage

The assembly that contains the test that automates the test case.

Reference name=Microsoft.VSTS.TCM.AutomatedTestStorage, Data type=String

Test Case

Automated Test Type

The type of test that automates the test case.

Reference name=Microsoft.VSTS.TCM.AutomatedTestType, Data type=String

Test Case

AutomatedTestId

The ID of the test that automates the test case.

Reference name=Microsoft.VSTS.TCM.AutomatedTestId, Data type=String

Test Case

AutomatedTestName

The name of the test that is used to automate the test case.

Reference name=Microsoft.VSTS.TCM.AutomatedTestName, Data type=String

Test Case

LocalDataSource

The local data source that supports the test.

Reference name=Microsoft.VSTS.TCM.LocalDataSource, Data type=HTML

Test Case

Query Text

Field used to capture the query defined for a Query-based suite type.

Reference name=Microsoft.VSTS.TCM.QueryText, Data type=PlainText

Test Suite

Test Suite Audit

Tracks other operations run when modifying a test suite, for example: adding tests to a test suite or changing configurations. This field can be viewed through the History tab or through a separate query. There's a combined history view, including changes done to work items field and changes resulting from related artifacts such as test points and configurations.

Reference name=Microsoft.VSTS.TCM.TestSuiteAudit, Data type=PlainText

Test Suite

Test Suite Type ID 1

A system assigned value that corresponds to the test suite category and only applicable to test suites. Assigned values are:

  • 1 (Static)

  • 2 (Query-based)

  • 3 (Requirement- based)

Reference name=Microsoft.VSTS.TCM.TestSuiteTypeId, Data type=Integer

Test Suite

Note

  1. Do not customize the pick list for these fields. The system accepts only those values listed.

Fields that integrate with Team Foundation Build

Team Foundation Build is the on-premises build system you can use with Azure DevOps Server and TFS. You can configure your build process by using Team Foundation Build, and Team Foundation Build can generate work items when a build fails. It can also add build information to work items that were resolved in a particular build. For this to work, Team Foundation Build requires that the following two fields be added to the work item type definition: Found In and Integration Build.

Found In and Integrated in Build fields are defined for Bugs in the default processes. These fields associate bugs with the builds where they were found or fixed.

You can use the following code snippet to add these fields to a WIT definition.

<FIELD name="Found In" refname="Microsoft.VSTS.Build.FoundIn" type="String" reportable="dimension">
    <HELPTEXT>Product build number (revision) in which this item was found</HELPTEXT>
        <SUGGESTEDVALUES>
          <LISTITEM value="&lt;None&gt;" />
        </SUGGESTEDVALUES>
</FIELD>
<FIELD name="Integration Build" refname="Microsoft.VSTS.Build.IntegrationBuild" type="String" reportable="dimension">
    <HELPTEXT>Product build number this bug was fixed in</HELPTEXT>
        <SUGGESTEDVALUES>
          <LISTITEM value="&lt;None&gt;" />
        </SUGGESTEDVALUES>
</FIELD>

When the Found In field is present in a WIT definition, Team Foundation Build creates a work item when a build fails, and sets the Found In field to the build number of the build that failed. If the Found In field is missing, Team Foundation Build doesn't create a work item for the failed build, and everything else works as expected.

When the Integration Build field is present in the WIT definition, Team Foundation Build identifies work items that were resolved with each build and then updates those work items to set the build number in which they were resolved in the Integration Build field. If the Integration Build field is missing, Team Foundation Build doesn't store the build number in the work items, and everything else works as expected.

Builds and global list autopopulation

The first time you queue a build for a project using Team Foundation Build, TFS automatically adds a global list labeled Build - ProjectName. Each time a build is run, a LISTITEM is added to this global list with the name of the build.

By adding a GLOBALLIST element to the FIELD definition, you can provide a drop-down menu of builds that users can choose from. For example:

<FIELD name="Found In" refname="Microsoft.VSTS.Build.FoundIn" type="String" reportable="dimension">
    <HELPTEXT>Product build number (revision) in which this item was found</HELPTEXT>
        <SUGGESTEDVALUES>
          <LISTITEM value="&lt;None&gt;" />
        </SUGGESTEDVALUES>
        <SUGGESTEDVALUES expanditems="true" filteritems="excludegroups">
          <GLOBALLIST name="Builds - TeamProjectName" />
        </SUGGESTEDVALUES>
</FIELD>

Fields that Integrate with Test Plans

With Test Plans, you can automate the creation of a bug or other type of work item when a test fails. For more information, see Add findings to existing bugs with exploratory testing.

When a work item has been created in this manner, information about the system and the steps to reproduce the bug are captured in the System Info and Repro Steps fields.

You can add these fields to work item types that you create for tracking defects using the following code snippet.

<FIELD name="System Info" refname="Microsoft.VSTS.TCM.SystemInfo" type="HTML" />
<FIELD name="Repro Steps" refname="Microsoft.VSTS.TCM.ReproSteps" type="HTML" />

Fields that integrate with Team Foundation Version Control

One of the features available in Team Foundation version control (TFVC) enables you to associate or resolve work items when you check in code. You might have worked on a particular work item when you make a code change and you can set that association from within the source-control check-in window when you're finished working on the code.

The ability of Team Foundation version control to resolve a work item requires that work items contain a particular action. The source control system then queries work item tracking to determine whether the work item supports that action, and if it does support that action, it also queries for the source and destination states of the transition. If the action is found, the source control system can transition the work item according to the set transition when it checks in the code.

Note

When you use the Checkin action, you must set appropriate from and to states to reflect the state transition that you want.

For more information about Actions, see Automate field assignments based on State, Transition, or Reason.