VSTS | TFS 2018 | TFS 2017 | TFS 2015 | TFS 2013
How do you track and manage defects in your code? How do you make sure software problems and customer feedback get addressed in a timely manner to support high-quality software deployments? And, how do you do this while making good progress on new features?
At a minimum, you need a way to capture your software issues, prioritize them, assign them, and track progress. Moreover, you'll want to manage your bugs in ways that align with your Agile practices.
In a nutshell, you manage bugs through the following tasks:
- Capture information using the bug work item type
- Triage bugs by assigning a priority
- Update bug status throughout the bug lifecycle
- Monitor bug assignments and trends
Depending on the process chosen to create your project—Agile, Scrum, or CMMI— the items in your backlog may be called product backlog items (PBIs), user stories, or requirements. All three are similar: they describe the customer value to be delivered and the work to be performed.
By default, product backlog items (PBIs) and bugs appear on Scrum backlogs, user stories on Agile backlogs, and requirements on CMMI backlogs. Each team can choose how bugs show up on their backlogs and boards.
- You must be a member of the Contributors group or be granted Stakeholder access to add, modify, or remove work items
- Or, you must have your View work items in this node, and your Edit work items in this node permissions set to Allow.
To learn more, see Set permissions and access for work tracking.
You can track bugs in much the same way that you track product backlog items (PBIs) or user stories. Using the bug work item form, you capture the code defect in the Title, Steps to Reproduce, and other fields.
Bug work item form
The bug work item form tracks similar information to the one shown for the Scrum process.
The images you see from your web portal may differ from the images you see in this topic. These differences result from updates made to your web app, options that you or your admin have enabled, and which process was chosen when creating your project—Agile, Scrum, or CMMI.
The new web form provides a number of experiences not provided with the old web form. To learn more, see New work item experience.
Use the Discussion section to add and review comments made about the work being performed to resolve the bug.
The new web form is only available from TFS 2017 and later versions.
Fields specific to bugs
When defining a bug, use these fields to capture both the initial issue and ongoing discoveries made when triaging, investigating, fixing, and closing the bug.
|Steps to Reproduce (friendly name=Repro Steps)||
Capture enough information so that other team members can understand the full impact of the problem as well as whether they have fixed the bug. This includes actions taken to find or reproduce the bug and expected behavior.
Describe the criteria that the team should use to verify whether the code defect is fixed.
Information about the software and system configuration that is relevant to the test.
Provide the criteria to be met before the bug or user story can be closed. Before work begins, describe the customer acceptance criteria as clearly as possible. The acceptance criteria can be used as the basis for acceptance tests so that you can more effectively evaluate whether an item has been satisfactorily completed.
When Test Manager creates bugs, it automatically populates System Info and Found in Build with information about the software environment and build where the bug occurred. To learn more about defining the software environments, see Test different configurations.
When you resolve the bug, use Integrated in Build to indicate the name of the build that incorporates the code that fixes the bug.
For on-premises TFS, to access a drop-down menu of all builds that have been run, you can update the
For information about how to define build names, see build number format options.
A subjective rating of the bug as it relates to the business or customer requirements. Priority indicates the order in which code defects should be fixed. You can specify the following values:
A subjective rating of the impact of a bug on the project or software system. For example: If clicking a remote link (a rare event) causes an application or web page to crash (a severe customer experience), you might specify Severity = 2 - High and Priority = 3. Allowed values and suggested guidelines are:
1 To change the menu selection or pick list, see Customize the work tracking experience. The customization method depends on the process model used by your project.
Add and review comments made about the work being performed by going to the discussion section.
Capture bugs using test tools
You can create bugs during test sessions using one of the following tools:
- Test & Feedback extension: see Exploratory testing with the Test & Feedback extension
- Test Runner: see Update an existing bug while using Test Runner.
Once you've started coding and testing, you'll want to hold periodic triage meetings to review and prioritize your bugs. How frequently you meet and for how long depends on your situation. Typically, the project owner runs the bug triage meetings, and team leads, business analysts and other stakeholders who can speak about specific project risks attend them.
The project owner can create or open a shared query for new and reopened bugs to generate a list of bugs to be triaged.
Open a shared query or use the query editor to create useful bug queries, such as the following:
- Active bugs by priority (
State <> Doneor
State <> Closed)
- In Progress bugs (
State = Committedor
State = Active)
- Bugs to fix for a target release (
Tags Contains RTM)
- Recent bugs - bugs opened within the last 3 weeks (
Created Date > @Today-21)
Triage mode in query results
From the query results page, you can quickly move up and down within the list of bug work items using the up and down arrows. As you review each bug, you can assign it, add details, or set priority.
To learn more, see Triage work items.
Assign bugs to a sprint
Once bugs have been triaged, it's time to assign them to a sprint to get fixed. By addressing a set of bugs to get fixed every sprint, your team can keep the total number of bugs to a reasonable size.
When bugs appear on the product backlog, you can assign bugs to sprints in the same way you do PBIs and user stories during your sprint planning sessions.
When bugs are treated as tasks, they're often automatically linked to a PBI or user story. So, assigning their parent PBI or user story to a sprint will assign the linked bugs to the same sprint as the parent PBI or user story during your sprint planning sessions.
Your team should consider fixing all bugs found during a sprint when testing a feature in development.
Fix, resolve and close bugs (update status)
Bug fixes that involve more than a single section of code may require significant regression testing and may involve other team members. Record any conversations that relate to assessing the risk of bug fixes in the bug work item history.
Bug workflow lifecycle
Once you fix a bug, you should update its workflow State. State choices vary depending on the process you use—Scrum, Agile, or CMMI. The following images illustrate the workflow lifecycle defined for the default bug workflow for the Agile, Scrum, and CMMI processes.
For Scrum bugs, you simply change the State from Committed (similar to Active) to Done. For Agile and CMMI, you first resolve the bug, indicating that the bug has been fixed. Typically, the person who created the bug then verifies the fix and updates the State from Resolved to Closed. If more work has been found after a bug has been resolved or closed, it can be reactivated by setting the State to Committed or Active.
Verify a fix
To verify a fix, a developer or tester should attempt to reproduce the bug and look for additional unexpected behavior. If necessary, they should reactivate the bug.
When verifying a bug resolution, you may find that the bug was not completely fixed or you may disagree with the resolution. In this case, discuss the bug with the person who resolved it, come to an agreement, and possibly reactivate the bug. If you reactivate a bug, include the reasons for reactivating the bug in the bug description.
Choose the Verify option to re-run tests which identified the bug. You can invoke the Verify option from the bug work item form context menu to launch the relevant test case in the web runner. Perform your validation using the web runner and update the bug work item directly within the web runner.
Choose the Verify option to re-run tests which identified the bug. (Requires TFS 2017.1 or later version.) You can invoke the Verify option from the bug work item form context menu to launch the relevant test case in the web runner. Perform your validation using the web runner and update the bug work item directly within the web runner.
To learn more about running test from the web portal, see Run tests for web apps.
Close a bug
You close a bug once it's verified as fixed. However, you may also close a bug for one of these reasons:
- Deferred - deferring a fix until the next product release
- Duplicate - bug has already been reported, you can link each bug with the Duplicate/Duplicate of link type and close one of the bugs
- As Designed - feature works as designed
- Cannot Reproduce - tests prove that the bug can't be reproduced
- Obsolete - the bug's feature is no longer in the product
- Copied to Backlog - a PBI or user story has been opened to track the bug
It's always a good idea to describe any additional details for closing a bug in the Discussion field (new web form) or the History field (old web form) to avoid future confusion as to why the bug was closed.
Monitor bug status, assignments, and trends
You can track the bug status, assignments, and trends using queries which you can then chart and add to a dashboard.
For example, here are two examples showing active bugs by priority trend and a snapshot of bugs by priority.
Customize the bug form
You can add fields, change the workflow, add custom rules, and add custom pages to the work item form of most work item types. You can also add custom work item types. For details, see Customize an inheritance process.
You can add fields, change the workflow, add custom rules, and add custom pages to the work item form of most work item types. You can also add custom work item types. For details, see Customize the On-premises XML process model.
Try this next
To track your bugs and integrate with other resources available to you, see these topics:
- Scrum and working with sprints best practices
- Follow a work item or pull request
- Move, change type, or delete work items
- Pre-populate fields using a template
- Copy or clone a work item
Integrate & Test resources
Use the Analytics service to create bug reports
You can use Power BI to create more complex reports than what you can get from a query. To learn more, see Connect to VSTS with Power BI Data Connector.
Pre-defined SQL Server bug reports
If you work from an on-premises TFS and you have SQL Server Analysis Services and SQL Server Reporting Services configured for your project, you have access to the following reports (Agile and CMMI processes only).
To learn how to add SQL Server reports for a project, see Add reports to a project.
Use SonarQube to help manage technical debt
SonarQube provides a way of automatically measuring some technical debt. SonarQube finds important violations of best coding practices. You implement Sonar to ensure that developers follow important code metrics like appropriate class and method size or low cyclomatic complexity (a quantitative measure of the number of linearly independent paths through a program's source code).
By integrating your on-premises TFS with a SonarQube server, you can get the following data:
- Code clone analysis
- Code coverage data from tests