Configure unit tests by using a .runsettings file

Unit tests in Visual Studio can be configured by using a .runsettings file. For example, you can change the .NET Framework version on which the tests are run, the directory for the test results, or the data that's collected during a test run.

Run settings files are optional. If you don't require any special configuration, you don't need a .runsettings file. The most common use of a .runsettings file is to customize code coverage analysis.

Specify a run settings file

Run settings files can be used to configure tests that are run from the command line, in the IDE, or in a build workflow using Azure Test Plans or Team Foundation Server (TFS).

Specify a run settings file in the IDE

Select Test > Test Settings > Select Test Settings File and then select the .runsettings file. The file appears on the Test Settings menu, and you can select or deselect it. While selected, the run settings file applies whenever you select Analyze Code Coverage.

Select test settings file menu in Visual Studio

Specify a run settings file at the command line

To run tests from the command line, use vstest.console.exe and specify the settings file by using the /Settings parameter.

  1. Launch the Visual Studio Developer Command Prompt:

    On the Windows Start menu, choose Visual Studio 2017 > Developer Command Prompt for VS 2017.

  2. Enter a command similar to:

    vstest.console.exe MyTestAssembly.dll /EnableCodeCoverage /Settings:CodeCoverage.runsettings
    

For more information, see VSTest.Console.exe command-line options.

Customize tests

To customize your tests using a .runsettings file, follow these steps:

  1. Add an XML file to your Visual Studio solution and save it as test.runsettings.

    Tip

    The file name doesn't matter, as long as you use the extension .runsettings.

  2. Replace the file contents with the XML from the example that follows, and customize it as needed.

  3. On the Test menu, choose Test Settings > Select Test Settings File. Browse to the .runsettings file you created, and then select OK.

    Tip

    You can create more than one .runsettings file in your solution and select one as the active test settings file as needed.

Example .runsettings file

The following XML shows the contents of a typical .runsettings file. Each element of the file is optional because it has a default value.

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <!-- Configurations that affect the Test Framework -->
  <RunConfiguration>
    <MaxCpuCount>1</MaxCpuCount>
    <!-- Path relative to solution directory -->
    <ResultsDirectory>.\TestResults</ResultsDirectory>

    <!-- x86 or x64 -->
    <!-- You can also change it from menu Test > Test Settings > Default Processor Architecture -->
    <TargetPlatform>x86</TargetPlatform>

    <!-- Framework35 | [Framework40] | Framework45 -->
    <TargetFrameworkVersion>Framework40</TargetFrameworkVersion>

    <!-- Path to Test Adapters -->
    <TestAdaptersPaths>%SystemDrive%\Temp\foo;%SystemDrive%\Temp\bar</TestAdaptersPaths>

    <!-- TestSessionTimeout is only available with Visual Studio 2017 version 15.5 and higher -->
    <!-- Specify timeout in milliseconds. A valid value should be greater than 0 -->
    <TestSessionTimeout>10000</TestSessionTimeout>
  </RunConfiguration>

  <!-- Configurations for data collectors -->
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>
              <Exclude>
                <ModulePath>.*CPPUnitTestFramework.*</ModulePath>
              </Exclude>
            </ModulePaths>

            <!-- We recommend you do not change the following values: -->
            <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
            <CollectFromChildProcesses>True</CollectFromChildProcesses>
            <CollectAspDotNet>False</CollectAspDotNet>

          </CodeCoverage>
        </Configuration>
      </DataCollector>

      <!--Video data collector is only available with Visual Studio 2017 version 15.5 and higher -->
      <DataCollector uri="datacollector://microsoft/VideoRecorder/1.0" assemblyQualifiedName="Microsoft.VisualStudio.TestTools.DataCollection.VideoRecorder.VideoRecorderDataCollector, Microsoft.VisualStudio.TestTools.DataCollection.VideoRecorder, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" friendlyName="Screen and Voice Recorder">
      </DataCollector>

    </DataCollectors>
  </DataCollectionRunSettings>

  <!-- Parameters used by tests at runtime -->
  <TestRunParameters>
    <Parameter name="webAppUrl" value="http://localhost" />
    <Parameter name="webAppUserName" value="Admin" />
    <Parameter name="webAppPassword" value="Password" />
  </TestRunParameters>

  <!-- Adapter Specific sections -->

  <!-- MSTest adapter -->
  <MSTest>
    <MapInconclusiveToFailed>True</MapInconclusiveToFailed>
    <CaptureTraceOutput>false</CaptureTraceOutput>
    <DeleteDeploymentDirectoryAfterTestRunIsComplete>False</DeleteDeploymentDirectoryAfterTestRunIsComplete>
    <DeploymentEnabled>False</DeploymentEnabled>
    <AssemblyResolution>
      <Directory Path="D:\myfolder\bin\" includeSubDirectories="false"/>
    </AssemblyResolution>
  </MSTest>

</RunSettings>

Elements of a .runsettings file

The sections that follow detail the elements of a .runsettings file.

Run configuration

<RunConfiguration>
    <MaxCpuCount>1</MaxCpuCount>
    <ResultsDirectory>.\TestResults</ResultsDirectory>
    <TargetPlatform>x86</TargetPlatform>
    <TargetFrameworkVersion>Framework40</TargetFrameworkVersion>
    <TestAdaptersPaths>%SystemDrive%\Temp\foo;%SystemDrive%\Temp\bar</TestAdaptersPaths>
    <TestSessionTimeout>10000</TestSessionTimeout>
</RunConfiguration>

The RunConfiguration element can include the following elements:

Node Default Values
ResultsDirectory The directory where test results are placed.
TargetFrameworkVersion Framework40 Framework35, Framework40, Framework45

This setting specifies the version of the unit test framework used to discover and execute the tests. It can be different from the version of the .NET platform that you specify in the build properties of the unit test project.
TargetPlatform x86 x86, x64
TreatTestAdapterErrorsAsWarnings false false, true
TestAdaptersPaths One or more paths to the directory where the TestAdapters are located
MaxCpuCount 1 This setting controls the degree of parallel test execution when running unit tests, using available cores on the machine. The test execution engine starts as a distinct process on each available core, and gives each core a container with tests to run. A container can be an assembly, DLL, or relevant artifact. The test container is the scheduling unit. In each container, the tests are run according to the test framework. If there are many containers, then as processes finish executing the tests in a container, they're given the next available container.

MaxCpuCount can be:

n, where 1 <= n <= number of cores: up to n processes are launched

n, where n = any other value: the number of processes launched can be up to the number of available cores
TestSessionTimeout Allows users to terminate a test session when it exceeds a given timeout. Setting a timeout ensures that resources are well consumed and test sessions are constrained to a set time. The setting is available in Visual Studio 2017 version 15.5 and later.

Diagnostic data adapters (data collectors)

The DataCollectors element specifies settings of diagnostic data adapters. Diagnostic data adapters gather additional information about the environment and the application under test. Each adapter has default settings, and you only have to provide settings if you don't want to use the defaults.

Code coverage adapter

<CodeCoverage>
    <ModulePaths>
        <Exclude>
            <ModulePath>.*CPPUnitTestFramework.*</ModulePath>
        </Exclude>
    </ModulePaths>

    <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
    <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
    <CollectFromChildProcesses>True</CollectFromChildProcesses>
    <CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>

The code coverage data collector creates a log of which parts of the application code have been exercised in the test. For more information about customizing the settings for code coverage, see Customize code coverage analysis.

Video data collector

The video data collector captures a screen recording when tests are run. This recording is useful for troubleshooting UI tests. The video data collector is available in Visual Studio 2017 version 15.5 and later.

To customize any other type of diagnostic data adapters, use a test settings file.

TestRunParameters

<TestRunParameters>
    <Parameter name="webAppUrl" value="http://localhost" />
    <Parameter name="webAppUserName" value="Admin" />
    <Parameter name="webAppPassword" value="Password" />
</TestRunParameters>

Test run parameters provide a way to define variables and values that are available to the tests at runtime. Access the parameters using the TestContext.Properties property:

[TestMethod]
public void HomePageTest()
{
    string appURL = TestContext.Properties["webAppUrl"];
}

To use test run parameters, add a private TestContext field and a public TestContext property to your test class.

MSTest run settings

<MSTest>
    <MapInconclusiveToFailed>True</MapInconclusiveToFailed>
    <CaptureTraceOutput>false</CaptureTraceOutput>
    <DeleteDeploymentDirectoryAfterTestRunIsComplete>False</DeleteDeploymentDirectoryAfterTestRunIsComplete>
    <DeploymentEnabled>False</DeploymentEnabled>
    <AssemblyResolution>
      <Directory Path="D:\myfolder\bin\" includeSubDirectories="false"/>
    </AssemblyResolution>
</MSTest>

These settings are specific to the test adapter that runs test methods that have the TestMethodAttribute attribute.

Configuration Default Values
ForcedLegacyMode false In Visual Studio 2012, the MSTest adapter was optimized to make it faster and more scalable. Some behavior, such as the order in which tests are run, might not be exactly as it was in previous editions of Visual Studio. Set this value to true to use the older test adapter.

For example, you might use this setting if you have an app.config file specified for a unit test.

We recommend that you consider refactoring your tests to allow you to use the newer adapter.
IgnoreTestImpact false The test impact feature prioritizes tests that are affected by recent changes, when run in MSTest or from Microsoft Test Manager. This setting deactivates the feature. For more information, see Which tests should be run since a previous build.
SettingsFile You can specify a test settings file to use with the MSTest adapter here. You can also specify a test settings file by selecting Test > Test Settings > Select Test Settings File.

If you specify this value, you must also set the ForcedlegacyMode to true.

<ForcedLegacyMode>true</ForcedLegacyMode>
KeepExecutorAliveAfterLegacyRun false After a test run is completed, MSTest is shut down. Any process that is launched as part of the test is also killed. If you want to keep the test executor alive, set the value to true. For example, you could use this setting to keep the browser running between coded UI tests.
DeploymentEnabled true If you set the value to false, deployment items that you've specified in your test method aren't copied to the deployment directory.
CaptureTraceOutput true You can write to the debug trace from your test method using Trace.WriteLine.
DeleteDeploymentDirectoryAfterTestRunIsComplete true To retain the deployment directory after a test run, set this value to false.
MapInconclusiveToFailed false If a test completes with an inconclusive status, it is mapped to the skipped status in Test Explorer. If you want inconclusive tests to be shown as failed, set the value to true.
InProcMode false If you want your tests to be run in the same process as the MSTest adapter, set this value to true. This setting provides a minor performance gain. But if a test exits with an exception, the remaining tests don't run.
AssemblyResolution false You can specify paths to additional assemblies when finding and running unit tests. For example, use these paths for dependency assemblies that aren't in the same directory as the test assembly. To specify a path, use a Directory Path element. Paths can include environment variables.

<AssemblyResolution> <Directory Path="D:\myfolder\bin\" includeSubDirectories="false"/> </AssemblyResolution>

See also