Create Test Settings for Manual Tests
Test settings use diagnostic data adapters, which specify various types of data to collect when you run manual tests, automated tests, or both. Diagnostic data adapters can also specify how to affect the test machine. For example, a diagnostic data adapter might create an action recording, an action log, or a video recording, or collect system information. Additionally, diagnostic data adapters can be used to simulate potential bottlenecks on the test machine or reduce the available system memory. For example, you can emulate a slow network to impose a bottleneck on the system.
Test settings define the following:
The type of tests that you will run (manual or automated).
The set of roles that is required for your application under test.
The role to use to run your tests.
The diagnostic data adapters to use for each role.
For more information about all the available diagnostic data adapters for manual tests, see the table in the following procedure.
Collect Data on a Local Machine by Using Default Test Settings
If you just want to run manual tests on a local machine and do not want to collect data on another tier of your application, you can use the default test settings. The default test settings use the following diagnostic data adapters to collect data on your local machine only:
Actions
ASP.NET Client Proxy for IntelliTrace and Test Impact
IntelliTrace
System information
Test impact
You do not have to select a set of roles in your test settings or select an environment in your test plan for this setup. This is the simplest approach when you are getting started.
Note
If you select this approach, you do not have to create test settings for your test plans. The default settings are added when you create a test plan.
Collect Data on Local Machines by Using Specific Data Diagnostic Adapters
If you want to run manual tests and collect data on a local machine by using different diagnostic data adapters, you must create test settings and add them to your plan. However, you do not have to add a set of roles to your test settings. You can just use the default set of roles called Local for your set of roles. This will select a default matching environment of <Local Machine Only>.
Collect Data Remotely
If you want to run your manual tests on a local machine and collect data remotely, you must create an environment for the set of roles that represents the remote tiers of your application under test. You must then select this set of roles in your test settings and specify what data should be collected or how to affect the test machine for each role. You can also collect data for the local role where you will run your tests. This machine is displayed in the Set of roles area as Local for the roles view. This local machine is also displayed when you select the data and diagnostics to collect.
The first two columns in the following table list your choices for collecting data or affecting the test machine when you run manual tests on a local machine. The last three columns list the actions that you must perform in each case.
Choice 1: Collect data or affect the test machine |
Choice 2: Diagnostic Data Adapters |
Action 1: Create Test Settings |
Action 2: Set of Roles To Be Defined |
Action 3: Matching Environment Required |
|---|---|---|---|---|
Local machine only |
|
No: Use default test settings |
No |
<Local> is already added as the test environment in your test plan |
Local machine only |
Select specific adapters |
Yes |
Use default:<Local> |
Use default: <Local Machine Only> |
Local and remote machines |
Select specific adapters |
Yes |
Yes |
Yes |
Note
If you have an environment that includes a desktop client, you can run your manual tests on this machine if you install Microsoft Test Manager. Effectively this machine then becomes the local machine for your test settings because you will run your manual tests on this machine.
For more information, see Setting Up Machines and Collecting Diagnostic Information Using Test Settings.
Creating test settings for manual tests
Some of the properties that you can set for a test plan, including the test settings, are shown in the following illustration.
.png "Plan Properties")
To create test settings for manual tests
Open Microsoft Test Manager.
Note
To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.
Click the down-arrow on the center group switcher and then click Testing Center.
On the center group menu bar, click Plan and then click Properties.
The properties for the currently selected test plan are displayed.
Click the drop-down arrow on Test settings under Manual runs and then click New.
The New Test Settings page is displayed with the General page selected.
Note
You can also create test settings in Lab Center by clicking Test Settings on the center group menu bar, and then clicking New.
Under Name, type the name for the test settings.
(Optional) Under Description, type a description for the test setting so that other team members know its intended use.
Under What type of tests do you want to run, select Manual and then click Next.
The New Test Settings page is displayed with the Roles page selected.
Important
The Local role is automatically selected as the role to run tests for manual tests.
For information about how to use Automated tests, see Create Test Settings for Automated Tests as Part of a Test Plan.
.png "Test settings Roles page")
To use a set of roles to collect data remotely or to affect a test machine when you run your tests, select from the list that shows Sets of roles and Matching environments. Click Next.
The New Test Settings page is displayed with the Data and Diagnostics page is selected.
Note
If you are running tests and collecting data and diagnostics only on a local machine, do not change the default value of Local for Sets of roles selected, and then click Next.
.png "Test settings Data and Diagnostics page")
To select the data and diagnostics that you want to collect for each role, select the role. For each role, select the diagnostic data adapters as required. To configure each diagnostic data adapter that you have selected for each role, click Configure.
For details about each diagnostic data adapter and how to configure it, you can view the associated topic in the following table.
Note
The table shows only the adapters that can be used for the role that is selected to run your manual tests. For more information about diagnostic data adapters, see Setting Up Machines and Collecting Diagnostic Information Using Test Settings.
Diagnostic Data Adapters for the Role that Runs Your Manual Tests
Diagnostic data adapter
Associated topic
ASP.NET Client Proxy for IntelliTrace and Test Impact:. This proxy allows you to collect information about the HTTP calls from a client to a Web server for the IntelliTrace and Test Impact diagnostic data adapters.
No configuration is required to collect this information.
Event log: You can configure a test setting to include event log collecting, which will be included in the test results.
Actions: You can create a test setting that collects a text description of each action that is performed during a test. When you configure this adapter, the selections are also used if you create an action recording when you run a manual test. The action logs and action recordings are saved together with the test results. You can play back the action recording later to fast-forward through your test, or you can view the action log to see what actions you performed.
How to: Record User Interface Actions for Manual Tests Using Test Settings
IntelliTrace: You can configure the diagnostic data adapter for IntelliTrace to collect specific diagnostic trace information to help isolate bugs that are difficult to reproduce. This creates an IntelliTrace file that has an extension of .iTrace that contains this information. When a test fails, you can create a bug. The IntelliTrace file that is saved together with the test results is automatically linked to this bug. The data that is collected in the IntelliTrace file increases debugging productivity by reducing the time that is required to reproduce and diagnose an error in the code. From this IntelliTrace file the local session can be simulated on another computer, which reduces the risk of a bug being non-reproducible.
For more information, see Debugging with IntelliTrace.
How to: Collect IntelliTrace Data to Help Debug Difficult Issues
Network emulation: You can specify that you want to put an artificial network load on your test by using a test setting. Network emulation affects the communication to and from the machine by emulating a particular network connection speed, such as dial-up.
.gif "Note")
Note
Network emulation cannot be used to increase the network connection speed.System information: A test setting can be specified to include the system information about the machine on which the test is run. The system information is specified in the test results by using a test setting.
No configuration is required to collect this information.
Test impact: You can collect information about which methods of your applications code were used when a test case was running. This can be used, together with changes to the application code that are made by developers, to determine which tests were affected by those development changes.
How to: Collect Data to Check Which Tests Should be Run After Code Changes
Video Recorder: You can create a video recording of your desktop session when you run a test. The video can help other team members to isolate application issues that are difficult to reproduce.
How to: Record a Video of Your Desktop as You Run Tests Using Test Settings
Tip
Test Attachment Cleaner
The data that some of the diagnostic data adapters capture can use a lot of database space over time. The administrator of the database used for Visual Studio 2010 cannot control what data gets attached as part of test runs. For example, there are no policy settings that can limit the size of the data captured and there is not a retention policy to determine how long to hold this data before initiating a cleanup. You can use the test attachment cleaner tool to:
-
Determine how much database space each set of diagnostic data captures is using.
-
Reclaim the space for runs that are no longer relevant from a business perspective.
For more information and to download the test attachment cleaner tool, see Test Attachment Cleaner for Visual Studio Ultimate 2010 & Test Professional 2010.
-
To save the test settings, click Save and Close.
A matching test environment is automatically selected in Test environment. If multiple test environments match the set of roles in your test settings, you can select a different matching environment.
You can apply your changes for the test plan by clicking Save on the toolbar.
Note
If you have to change your test settings, click Open next to the Test settings drop-down arrow. You can also locate the Lab Center, click Test Settings on the center group menu bar, and then click Open. For more information, see How to: Edit an Existing Test Setting for a Test Plan.
See Also
Tasks
Create Test Settings for Automated Tests as Part of a Test Plan
Create Test Settings to Run Automated Tests from Visual Studio
Concepts
Setting Up Machines and Collecting Diagnostic Information Using Test Settings