Unit Test Basics

This topic describes the basics of writing and running unit test in Visual Studio Test Explorer. It contains the following sections:

Unit testing overview

• Quick starts

The MyBank Solution example

Creating the unit test projects

Writing your tests

Running tests in Test Explorer

• Running and viewing tests from the Test Explorer toolbar

• Running tests after every build

• Filtering and grouping the test list

Debugging unit tests

Additional tools for unit tests

• Generating application code from tests

• Generating multiple tests by using data driven test methods

• Analyzing unit test code coverage

• Isolating unit test methods with Microsoft Fakes

Unit testing overview

Visual Studio Test Explorer is designed to support developers and teams who incorporate unit testing in their software development practices. Unit testing helps you ensure the correctness of your program by verifying that the application code does what you expect it to do. In unit testing, you analyze the functionality of your program to discover discrete testable behaviors that you can test as individual units. You use a unit testing framework to create tests of those behaviors and to report the results of those tests.

Unit testing has the greatest effect when it’s an integral part of your software development workflow. As soon as you write a function or other block of application code, you create unit tests that verify the behavior of the code in response to standard, boundary, and incorrect cases of input data, and that verify any explicit or implicit assumptions made by the code. In a software development practice known as test driven development, you create the unit tests before you write the code, so you use the unit tests as both design documentation and functional specifications of the functionality.

Test Explorer provides a flexible and efficient way to run your unit tests and view their results in Visual Studio. Visual Studio installs the Microsoft unit testing frameworks for managed and native code. Test Explorer can also run third-party and open source unit test frameworks that have implemented Test Explorer add-on interfaces. You can add many of these frameworks through the Visual Studio Extension Manager and the Visual Studio gallery. See How to: Install Third-Party Unit Test Frameworks

Test Explorer views can display all of your tests, or only the tests that have passed, failed, haven’t been run yet, or have been skipped. You can filter the tests in any view by matching text in the search box at the global level or by selecting one of the pre-defined filters. You can run any selection of the tests at any time. When you use Visual Studio Ultimate, you can run tests automatically after every build. The results of a test run are immediately apparent in the pass/fail bar at the top of the explorer window. Details of a test method result are displayed when you select the test.

Quick starts

For an introduction to unit testing that takes you directly into coding, see one of these topics:

• Walkthrough: Creating and running unit tests for managed code

• Quick start: Test driven development with Test Explorer

• Unit testing native code with Test Explorer

The MyBank Solution example

In this topic, we use the development of a fictional application called MyBank as an example. You don’t need the actual code to follow the explanations in this topic. Test methods are written in C# and presented by using the Microsoft Unit Testing Framework for Managed Code, However, the concepts are easily transferred to other languages and frameworks.

Our first attempt at a design for the MyBank application includes an accounts component that represents an individual account and its transactions with the bank, and a database component that represents the functionality to aggregate and manage the individual accounts.

We create a MyBank solution that contains two projects:

• Accounts

• BankDb

Our first attempt at designing the Accounts project contain a class to hold basic information about an account, an interface that specifies the common functionality of any type of account, like depositing and withdrawing assets from the account, and a class derived from the interface that represents a checking account. We begin the Accounts projects by creating the following source files:

• AccountInfo.cs defines the basic information for an account.

• IAccount.cs defines a standard IAccountinterface for an account, including methods to deposit and withdraw assets from an account and to retrieve the account balance.

• CheckingAccount.cs contains the CheckingAccount class that implements the IAccounts interface for a checking account.

We know from experience that one thing a withdrawal from a checking account must do is to make sure that the amount withdrawn is less than the account balance. So we override the IAccount.Withdaw method in CheckingAccount with a method that checks for this condition. The method might look like this:

public void Withdraw(double amount)
{
    if(m_balance >= amount)
    {
        m_balance -= amount;
    }
    else
    {
        throw new ArgumentException(amount, "Withdrawal exceeds balance!")
    }
}

Now that we have some code, it’s time for testing.

Creating the unit test projects

A unit test project usually mirrors the structure of a single code project. In the MyBank example, you add two unit test projects named AccountsTests and BankDbTests to the MyBanks solution. The test project names are arbitrary, but adopting a standard naming convention is a good idea.

To add a unit test project to a solution:

1. On the File menu, choose New and then choose Project (Keyboard Ctrl + Shift + N).

2. On the New Project dialog box, expand the Installed node, choose the language that you want to use for your test project, and then choose Test.

3. To use one of the Microsoft unit test frameworks, choose Unit Test Project from the list of project templates. Otherwise, choose the project template of the unit test framework that you want to use. To test the Accounts project of our example, you would name the project AccountsTests.

   Warning

   Not all third-party and open source unit test frameworks provide a Visual Studio project template. Consult the framework document for information about creating a project.

4. In your unit test project, add a reference to the code project under test, in our example to the Accounts project.

   To create the reference to the code project:

   1. Select the project in Solution Explorer.

   2. On the Project menu, choose Add Reference.

   3. On the Reference Manager dialog box, open the Solution node and choose Projects. Select the code project name and close the dialog box.

Each unit test project contains classes that mirror the names of the classes in the code project. In our example, the AccountsTests project would contain the following classes:

• AccountInfoTests class contains the unit test methods for the AccountInfo class in the BankAccount project

• CheckingAccountTests class contains the unit test methods for CheckingAccount class.

Writing your tests

The unit test framework that you use and Visual Studio IntelliSense will guide you through creating unit tests for a code project. To run in Test Explorer, most frameworks require that you add specific attributes to identify unit test methods. The frameworks also provide a way—usually through assert statements or method attributes—to indicate whether the test method has passed or failed. Other attributes identify optional setup methods that are at class initialization and before each test method and teardown methods that are run after each test method and before the class is destroyed.

The AAA (Arrange, Act, Assert) pattern is a common way of writing unit tests for a method under test.

• The Arrange section of a unit test method initializes objects and sets the value of the data that is passed to the method under test.

• The Act section invokes the method under test with the arranged parameters.

• The Assert section verifies that the action of the method under test behaves as expected.

To test the CheckingAccount.Withdraw method of our example, we can write two tests: one that verifies the standard behavior of the method, and one that verifies that a withdrawal of more than the balance will fail. In the CheckingAccountTests class, we add the following methods:

[TestMethod]
public void Withdraw_ValidAmount_ChangesBalance()
{
    // arrange
    double currentBalance = 10.0;
    double withdrawal = 1.0;
    double expected = 9.0;
    var account = new CheckingAccount("JohnDoe", currentBalance);
    // act
    account.Withdraw(withdrawal);
    double actual = account.Balance;
    // assert
    Assert.AreEqual(expected, actual);
}

[TestMethod]
[ExpectedException(typeof(ArgumentException))]
public void Withdraw_AmountMoreThanBalance_Throws()
{
    // arrange
    var account = new CheckingAccount("John Doe", 10.0);
    // act
    account.Withdraw(1.0);
    // assert is handled by the ExcpectedException
}

Note that Withdraw_ValidAmount_ChangesBalance uses an explicit Assert statement to determine whether the test method passes or fails, while Withdraw_AmountMoreThanBalance_Throws uses the ExpectedException attribute to determine the success of the test method. Under the covers, a unit test framework wraps test methods in try/catch statements. In most cases, if an exception is caught, the test method fails and the exception is ignored. The ExpectedException attribute causes the test method to pass if the specified exception is thrown.

For more information about the Microsoft Unit Testing Frameworks, see one of the following topics:

• Writing Unit Tests for the .NET Framework with the Microsoft Unit Test Framework for Managed Code

• Writing Unit tests for C/C++ with the Microsoft Unit Testing Framework for C++

Running tests in Test Explorer

When you build the test project, the tests appear in Test Explorer. If Test Explorer is not visible, choose Test on the Visual Studio menu, choose Windows, and then choose Test Explorer.

As you run, write, and rerun your tests, the default view of Test Explorer displays the results in groups of Failed Tests, Passed Tests, Skipped Tests and Not Run Tests. You can choose a group heading to open the view that displays all them tests in that group.

Running and viewing tests from the Test Explorer toolbar

The Test Explorer toolbar helps you discover, organize, and run the tests that you are interested in.

You can choose Run All to run all your tests, or choose Run to choose a subset of tests to run. After you run a set of tests, a summary of the test run appears at the bottom of the Test Explorer window. Select a test to view the details of that test in the bottom pane. Choose Open Test from the context menu (Keyboard: F12) to display the source code for the selected test.

Running tests after every build

Warning

Running unit tests after every build is supported only in Visual Studio Ultimate.

To run your unit tests after each local build, choose Test on the standard menu, choose Run Tests After Build on the Test Explorer toolbar.

Filtering and grouping the test list

When you have a large number of tests, you can Type in Test Explorer search box to filter the list by the specified string. You can restrict your filter event more by choosing from the filter list.

To group your tests by category, choose the Group By button.

For more information, see Running Unit Tests with Test Explorer

Debugging unit tests

You can use Test Explorer to start a debugging session for your tests. Stepping through your code with the Visual Studio debugger seamlessly takes you back and forth between the unit tests and the project under test. To start debugging:

1. In the Visual Studio editor, set a breakpoint in one or more test methods that you want to debug.

   Note

   Because test methods can run in any order, set breakpoints in all the test methods that you want to debug.

2. In Test Explorer, select the test methods and then choose Debug Selected Tests from the shortcut menu.

For more information about the debugger, see Debugging in Visual Studio.

Additional tools for unit tests

Generating application code from tests

If you are writing your tests before you write your project code, you can use IntelliSense to generate classes and methods in your project code. Write a statement in a test method that calls the class or method that you want to generate, then open the IntelliSense menu under the call. If the call is to a constructor of the new class, choose Generate new type from the menu and follow the wizard to insert the class in your code project. If the call is to a method, choose Generate new method from the IntelliSense menu.

Generating multiple tests by using data driven test methods

Note

These procedures apply only to test methods that you write by using the Microsoft unit test framework for managed code. If you’re using a different framework, consult the framework documentation for equivalent functionality.

Data-driven test methods let you verify a range of values in a single unit test method. To create a data-driven unit test method, decorate the method with a DataSource attribute that specifies the data source and table that contains the variable values that you want to test. In the method body, you assign the row values to variables using the TestContext.DataRow[ColumnName] indexer.

For example, assume we add an unnecessary method to the CheckingAccount class that is named AddIntegerHelper. AddIntegerHelper adds two integers.

To create a data-driven test for the AddIntegerHelper method, we first create an Access database named AccountsTest.accdb and a table named AddIntegerHelperData. The AddIntegerHelperData table defines columns to specify the first and second operands of the addition and a column to specify the expected result. We fill a number of rows with appropriate values.

    [DataSource(
        @"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\Projects\MyBank\TestData\AccountsTest.accdb", 
        "AddIntegerHelperData"
    )]
    [TestMethod()]
    public void AddIntegerHelper_DataDrivenValues_AllShouldPass()
    {
        var target = new CheckingAccount();
        int x = Convert.ToInt32(TestContext.DataRow["FirstNumber"]);
        int y = Convert.ToInt32(TestContext.DataRow["SecondNumber"]); 
        int expected = Convert.ToInt32(TestContext.DataRow["Sum"]);
        int actual = target.AddIntegerHelper(x, y);
        Assert.AreEqual(expected, actual);
    }

The attributed method runs once for each row in the table. Test Explorer reports a test failure for the method if any of the iterations fail. The test results detail pane for the method shows you the pass/fail status method for each row of data.

For more information, see How to: Create a data-driven unit test

Analyzing unit test code coverage

Note

Unit test code coverage is available for native and managed languages and all unit test frameworks that can be run by the Unit Test Framework.

You can determine the amount of your product code that is actually being tested by your unit tests by using the Visual Studio code coverage tool. You can run code coverage on selected tests or on all tests in a solution. The Code Coverage Results window displays the percentage of the blocks of product code that were exercised by line, function, class, namespace and module.

To run code coverage for test methods in a solution,

1. choose Tests on the Visual Studio menu and then choose Analyze code coverage.

2. Choose one of these commands:

   1. Selected tests runs the test methods that you have selected in Test Explorer.

   2. All tests runs all the test methods in the solution.

Coverage results appear in the Code Coverage Results window.

For more information, see Using Code Coverage to Determine How Much Code is being Tested.

Isolating unit test methods with Microsoft Fakes

Note

Microsoft Fakes is available only in Visual Studio Ultimate. Microsoft Fakes can be used only with test methods that you write by using unit test frameworks for managed code.

The problem

Unit test methods that focus on verifying the internal code in a function can be difficult to write when the method under test calls functions that introduce external dependencies. For example, the methods of the CheckingAccount example class should probably make calls to the BankDb component to update the main database. We can refactor the CheckingAccount class to look like the following:

class CheckingAccount : IAccount
{
    public CheckingAccount(customerName, double startingBalance, IBankDb bankDb)
    {
        m_bankDb = bankDb;
        // set up account
    }

    public void Withdraw(double amount)
    {
        if(m_balance >= amount)
        {
            m_balance = m_MyBankDb.Withdraw(m_accountInfo.ID, amount);
        }
        else
        {
            throw new ArgumentException(amount, "Withdrawal exceeds balance!")
        }
    }

    private IBankDb m_bankDb = null;
    // ...

The unit tests of this CheckingAccount.Withdraw method can now fail because of issues caused by the call to m_bankDb.Withdraw. The database or network connection could be lost or the permissions on the database might be wrong. A failure in the m_bankDB.Withdraw call would cause the test to fail for reasons that aren’t related to its internal code.

The Microsoft Fakes solution

Microsoft Fakes creates an assembly that contains classes and methods that you can substitute for classes in unit test methods that cause dependencies. A substitute class in the generated Fakes module declares a method and a delegate for each public method in the target component. In a test method, you implement the delegate to create the exact behavior of the dependency call in the method that you want to test.

In our example, we can create a Fakes assembly for the BankDb project, and then use the StubIBankDb class that is generated by Fakes and that derives from the IBankDb interface to remove the uncertainty caused by interactions with the database. A modifed version of the Withdraw_ValidAmount_ChangesBalance test method would then look like this:

[TestMethod]
public void Withdraw_ValidAmount_ChangesBalance()
{
    // arrange
    double currentBalance = 10.0;
    double withdrawal = 1.0;
    double expected = 9.0;

    // set up the Fakes object and delegate
    var stubBankDb = new MyBank.Stubs.StubIBankDb();
    stubBankDb.WithdrawDoubleDouble = (id, amount) => { return 9.0; }
    var account = new CheckingAccount("JohnDoe", currentBalance, stubBankDb);

    // act
    account.Withdraw(withdrawal);
    double actual = account.Balance;

    // assert
    Assert.AreEqual(expected, actual);
}

This line in the test method:

stubBankDb.WithdrawDoubleDouble = (id, amount) => { return 9.0; }

implements the Fakes delegate for the Withdraw method by using a lamba expression. The stubBankDb.Withdraw method calls the delegate and so always returns the specified amount, which enables the test method to reliably verify the behavior of the Accounts method.

More about Microsoft Fakes

Microsoft Fakes uses two approaches to creating the substitute classes:

1. Stubs generate substitute classes derived from the parent interface of the target dependency class. Stub methods can be substituted for public virtual methods of the target class.

2. Shims use runtime instrumentation to divert calls to a target method to a substitute shim method for non-virtual methods.

In both approaches, you use the generated delegates of calls to the dependency method to specify the behavior that you want in the test method.

For more information, see Isolating Code under Test with Microsoft Fakes.