Set up continuous testing for your builds

VSTS | TFS 2018 | TFS 2017 | TFS 2015

Find problems early after changes are checked in and built by running continuous tests using Visual Studio Team Services (VSTS) or Team Foundation Server (TFS) and test automation.

Note

Build and release pipelines are called definitions in TFS 2018 and in older versions. Service connections are called service endpoints in TFS 2018 and in older versions.

NOTE: You can use version 2.x or higher of the Visual Studio Test task to deploy and run unit and functional tests without requiring the Deploy Test Agent and Run Functional Tests tasks, and run tests on platforms that don't have Visual Studio installed by using the Visual Studio Test Platform.

For more details, see Testing with unified agents and phases.

Before you start

Set up app deployment for your build

  1. In VSTS or TFS, go to your project.

  2. Go to your build pipeline and open it for editing.

    Edit build pipeline

  3. Choose the + icon for the process phase and add tasks to deploy your app for testing. For example, if you're testing a Visual Studio solution and want to use the Window Machine File Copy and PowerShell on Target Machines tasks:

    Add File Copy and PowerShell tasks

  4. Add the details to copy your app from its drop location to its test environment:

    • Source folder for your app

    • List of machines where you want to deploy your app

    • Credentials to connect to the target machines

    • Target folder where to put your app

    Build pipeline, copy app details

    If you use the list of machines in more than one task, consider defining a variable that contains the list of machines. For example, a build or release pipeline variable or a variable defined within a project-wide variable group. Using a variable means that you can change the list of machines in one place and have the change apply to all the tasks that use the variable.

  5. Add the details to deploy your app using PowerShell.

    • List of machines where you want to deploy your app

    • Credentials to connect to the target machines

    • Path to the PowerShell script used to deploy your app

    Add task to deploy app with PowerShell

  6. Save your build pipeline.

    Build pipeline: save

Set up test deployment for your build

NOTE: You can use version 2.x of the Visual Studio Test task to deploy and run unit and functional tests without requiring the Deploy Test Agent and Run Functional Tests tasks, and run tests on platforms that don't have Visual Studio installed by using the Visual Studio Test Platform.

If you want to use the Deploy Test Agent and Run Functional Tests tasks:

  1. In your build pipeline, add a Window Machine File Copy task with these details to deploy your tests:

    • Source folder for your tests

    • List of machines where you want to run your tests

    • Credentials to connect to the target machines

    • Target folder where to put your tests

    Build pipeline, copy tests

  2. Add a task with these details to deploy the test agent for running your tests:

    • List of machines where you want to deploy the test agent

    • Credentials to connect to the target machines

    • Credentials for the deployment agents, so the test agent can run in your test environment

    Build pipeline: deploy test agent details

  3. Save your build pipeline.

    Build pipeline: save

Set up your tests to run with your build

  1. In your build pipeline, add a task with these details to run your tests with the test agent:

    • List of machines where you want to run your tests

    • Folder where you put your tests

    Build pipeline: Run tests with test agent

  2. Save your build pipeline and queue a build to check your test run.

    Build pipeline: queue build

  3. After your build is done, review your test results.

See also

Help and support

Report any problems on Developer Community, make suggestions on UserVoice, get advice on Stack Overflow, and get support via our Support page.