Publish Test Results task

• 09/30/2020
• 14 minutes to read
• • s
  • m
  • a
  • m
  • D
  • +18

Azure Pipelines | Azure DevOps Server 2020 | Azure DevOps Server 2019 | TFS 2018 - TFS 2015

This task publishes test results to Azure Pipelines or TFS when tests are executed to provide a comprehensive test reporting and analytics experience. You can use the test runner of your choice that supports the results format you require. Supported results formats include CTest, JUnit (including PHPUnit), NUnit 2, NUnit 3, Visual Studio Test (TRX), and xUnit 2.

Other built-in tasks such as Visual Studio Test task and Dot NetCore CLI task automatically publish test results to the pipeline, while tasks such as Ant, Maven, Gulp, Grunt, .NET Core and Xcode provide publishing results as an option within the task, or build libraries such as Cobertura and JaCoCo. If you are using any of these tasks, you do not need a separate Publish Test Results task in the pipeline.

The published test results are displayed in the Tests tab in the pipeline summary and help you to measure pipeline quality, review traceability, troubleshoot failures, and drive failure ownership.

The following example shows the task configured to publish test results.

You can also use this task in a build pipeline to publish code coverage results produced when running tests to Azure Pipelines or TFS in order to obtain coverage reporting.

Check prerequisites

If you're using a Windows self-hosted agent, be sure that your machine has this prerequisite installed:

• .NET Framework 4.6.2 or a later version

Demands

[none]

# Publish Test Results
# Publish test results to Azure Pipelines
- task: PublishTestResults@2
  inputs:
    #testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit, cTest
    #testResultsFiles: '**/TEST-*.xml' 
    #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
    #mergeTestResults: false # Optional
    #failTaskOnFailedTests: false # Optional
    #testRunTitle: # Optional
    #buildPlatform: # Optional
    #buildConfiguration: # Optional
    #publishRunAttachments: true # Optional

Arguments

Result formats mapping

This table lists the fields reported in the Tests tab in a build or release summary, and the corresponding mapping with the attributes in the supported test result formats.

¹ Duration is used only when Date started and Date completed are not available.

² The fully Qualified name format is Namespace.Testclass.Methodname with a character limit of 512. If the test is data driven and has parameters, the character limit will include the parameters.

Docker

For Docker based apps there are many ways to build your application and run tests:

• Build and test in a build pipeline: build and tests execute in the pipeline and test results are published using the Publish Test Results task.
• Build and test with a multi-stage Dockerfile: build and tests execute inside the container using a multi-stage Docker file, as such test results are not published back to the pipeline.
• Build, test, and publish results with a Dockerfile: build and tests execute inside the container and results are published back to the pipeline. See the example below.

Build, test, and publish results with a Docker file

In this approach, you build your code and run tests inside the container using a Docker file. The test results are then copied to the host to be published to the pipeline. To publish the test results to Azure Pipelines, you can use the Publish Test Results task. The final image will be published to Docker or Azure Container Registry

Get the code

1. Import into Azure DevOps or fork into GitHub the following repository. This sample code includes a Dockerfile file at the root of the repository along with .vsts-ci.docker.yml file.

   https://github.com/MicrosoftDocs/pipelines-dotnet-core
   

2. Create a Dockerfile.build file at the root of the directory with the following:

   # Build and run tests inside the docker container
   FROM microsoft/dotnet:2.1-sdk
   WORKDIR /app
   # copy the contents of agent working directory on host to workdir in container
   COPY . ./
   # dotnet commands to build, test, and publish
   RUN dotnet restore
   RUN dotnet build -c Release
   RUN dotnet test dotnetcore-tests/dotnetcore-tests.csproj -c Release --logger "trx;LogFileName=testresults.trx"
   RUN dotnet publish -c Release -o out
   ENTRYPOINT dotnet dotnetcore-sample/out/dotnetcore-sample.dll
   

   This file contains the instructions to build code and run tests. The tests are then copied to a file testresults.trx inside the container.

3. To make the final image as small as possible, containing only the runtime and deployment artifacts, replace the contents of the existing Dockerfile with the following:

   # This Dockerfile creates the final image to be published to Docker or
   # Azure Container Registry
   # Create a container with the compiled asp.net core app
   FROM microsoft/aspnetcore:2.0
   # Create app directory
   WORKDIR /app
   # Copy only the deployment artifacts
   COPY /out .
   ENTRYPOINT ["dotnet", "dotnetcore-sample.dll"]
   

Define the build pipeline

Attachments support

The Publish Test Results task provides support for attachments for both test run and test results for the following formats. For public projects, we support 2GB of total attachments.

Visual Studio Test (TRX)

NUnit 3

Related tasks

• Visual Studio Test
• Publish Code Coverage Results

FAQ

What is the maximum permissible limit of FQN?

The maximum FQN limit is 512 characters.

Does the FQN Character limit also include properties and their values in case of data driven tests?

Yes, the FQN character limit includes properties and their values.

Will the FQN be different for sub-results?

Currently, sub-results from the data driven tests will not show up in the corresponding data.

Example: I have a Test case: Add product to cart Data 1: Product = Apparel Data 2: Product = Footwear

All test sub-result published will only have the test case name and the data of the first row.

Open source

This task is open source on GitHub. Feedback and contributions are welcome.

Help and support

• See our troubleshooting page
• Get advice on Stack Overflow, and get support via the Developer Community