Setting Up Unit Testing for Python Code

Unit tests are pieces of code that test other code units in an application, typically isolated functions, classes, and so on. When an application passes all its unit tests, you can at least trust that it's low-level functionality is correct.

Python uses unit tests extensively to validate scenarios while designing a program. Python support in Visual Studio includes discovering, executing, and debugging unit tests within the context of your development process, rather than needing to run them separately.

This topic provides a brief outline of unit testing capabilities in Visual Studio with Python. For more on unit testing in general, see Unit Test Your Code.

Discovering and viewing tests

By convention, Visual Studio identifies tests are as methods whose names start with "test". To see this, do the following:

  1. Open a Python project loaded in Visual Studio, right-click your project, select Add > New Item..., then select Python Unit Test followed by Add.

  2. This creates a test1.py file with code that imports the standard unittest module, derives a test class from unittest.TestCase, and invokes unittest.main() if you run the script directly:

    import unittest
    
    class Test_test1(unittest.TestCase):
       def test_A(self):
           self.fail("Not implemented")
    
    if __name__ == '__main__':
       unittest.main()
    
  3. Save the file if necessary, then open Test Explorer with the Test > Windows > Test Explorer menu command.

  4. Test Explorer will search your project for tests and display them as shown below. Double-clicking a test opens its source file.

    Test Explorer showing default test_A

  5. As you add more tests to your project, you can organize the view in Test Explorer using the group by menu on the toolbar:

    Tests Explorer Group By toolbar menu

  6. You can also enter text in the search field to filter tests by name.

For more details on the unittest module and writing tests, see the Python 2.7 documentation or the Python 3.4 documentation (python.org).

Running tests

In Test Explorer you can run tests in a variety of ways:

  • Run All clearly runs all shown tests (subject to filters).
  • The Run... menu gives you commands to run failed, passed, or not run tests as a group.
  • You can select one or more tests, right-click, and select Run Selected Tests.

Tests run in the background and Test Explorer updates each test's status as it completes:

  • Passing tests show a green rick and the time taken to run the test:

    test_A passed status

  • Failed tests show a red cross with an Output link that shows console output and unittest output from the test run:

    test_A failed status

    test_A failed with reason

Debugging tests

Because unit tests are pieces of code, they are subject to bugs just like any other code and occasionally need to be run in a debugger, where you can set breakpoints, examine variables, and step through code. Visual Studio also provides diagnostic tools

To start debugging, set an initial breakpoint in your code, then right-click the test (or a selection) in Test Explorer and select Debug Selected Tests. Visual Studio will start the Python debugger as it would for application code.

Debugging a test

You can also use the Analyze Code Coverage for Selected Tests and Profile Test commands, depending on your version of Visual Studio (see the Features matrix).

Known Issues

  • When starting debugging, Visual Studio will appear to start and stop debugging, and then start again. This is expected.
  • When debugging multiple tests, each one is run independently, which will interrupt the debugging session.
  • Visual Studio will intermittently fail to start a test when debugging. Normally, attempting to debug the test again will succeed.
  • When debugging, it is possible to step out of a test into the unittest implementation. Normally, the next step will run to the end of the program and stop debugging.