Creating Load Tests Using the New Load Test Wizard

  • Article

A load test is created by using the New Load Test Wizard. When you use the New Load Test Wizard, you specify the following settings for the load test:

Note

The initial settings that you configure for a load test using the New Load Test Wizard can be edited later using the Load Test Editor. For more information, see Editing Load Test Scenarios Using the Load Test Editor.

  • **The initial scenario for the load test   **Load tests contain scenarios, which contain Web performance tests and unit tests. A scenario is a container within a load test where you specify load pattern, test mix model, test mix, network mix and Web browser mix. Scenarios are important because they give you flexibility in configuring test characteristics that allow for simulation of complex, realistic workloads. How to create and configure the load test scenario is covered in steps 2.0 through 2.5 in the next section.

    You can create and configure additional scenarios in your load test after the New Load Test Wizard completes. For more information, see Creating Additional Scenarios for an Existing Load Test.

    For a list of the load test scenario properties you can modify using the Load Test Editor, see Load Test Scenario Properties.

  • **Computers and counter sets in the load test   **Counter sets are a set of system performance counters that are useful to monitor during a load test. Counter sets are organized by technology, for example, ASP.NET or SQL counter sets. When you create the load test, you specify which computers and their counter sets to include in the load test.

    Note

    If your load tests are distributed across remote machines, controller and agent counters are mapped to the controller and agent counter sets. For more information about how to use remote machines in your load test, see Distributing Load Test Runs Across Multiple Test Machines Using Test Controllers and Test Agents.

    Specifying the computers and counter sets for the load test is covered in step 3 in the next section.

  • **The initial run setting for the load test   **Run settings are a set of properties that influence the way a load test runs.

    You can have more than one run setting in a load test. Only one of the run settings may be active for a load test run. The other run settings provide a quick way to select an alternative setting to use for subsequent test runs. The initial run setting is created using the New Load Test Wizard which is covered in step 4 in the next section. Additional run settings can be created and modified later using the Load Test Editor. For more information, see Configuring Load Test Run Settings.

    For a list of the run setting properties you can modify using the Load Test Editor, see Load Test Run Setting Properties.

Requirements

  • Visual Studio Ultimate

New Load Test Wizard Steps

Step 1 - Launching the New Load Test Wizard

Tip

You might have already created Web performance tests or unit tests. You can use the New Load Test Wizard to add them to the load test.

Creating a load test with the wizard

To launch the New Load Test Wizard

  1. Open a Web performance and load test project.

  2. In Solution Explorer, right-click the Web performance and load test project node, choose Add, and then choose Load Test.

    Note

    Your load test will be created with a default name of LoadTestn, where n is the next available number.

  3. The New Load Test Wizard starts with the Welcome page displayed.

  4. Choose Next.

    The Edit page is displayed, proceed to Step 2.0 - Creating and Changing a Scenario.

    Tip

    Any values you enter in the New Load Test Wizard can be modified in the Load Test Editor after the wizard is finished. For more information, see Editing Load Test Scenarios Using the Load Test Editor.

Step 2.0 - Creating and Changing a Scenario in the New Load Test Wizard

Load tests contain one or more scenarios, which are used to model how a group of users interacts with a server application. An individual scenario is made up of a load pattern, a test mix, a browser mix, and a network mix. Each of these settings corresponds to a page in the New Load Test Wizard.

Note

In the Load Test Editor, you can add more scenarios or change any of the scenario settings. For more information, see Creating Additional Scenarios for an Existing Load Test.

To specify a scenario in the New Load Test Wizard

  1. On the Scenario page of the New Load Test Wizard, type a name for your initial scenario.

    Note

    You can change the name of the scenario later by using the Load Test Editor.

  2. Select your preferred think time profile. For more information, see Editing Think Times to Simulate Web Site Human Interaction Delays in Load Tests Scenarios.

  3. Select your preferred think time between test iterations. For more information, see Editing Load Patterns to Model Virtual User Activities.

  4. After you choose the Scenario page settings, choose Next to continue to the Load Pattern page of the New Load Test Wizard. Proceed to Step 2.1 - Choosing a Load Pattern in the New Load Test Wizard.

After you create your load test, you can add more scenarios by using the Load Test Editor. For more information, see Creating Additional Scenarios for an Existing Load Test.

Additional Properties

Scenarios contain properties that you set initially in the New Load Test Wizard and can later change by using the Load Test Editor.

For example, Think Profile is a property that identifies whether think times are used or ignored in load tests. The Think Profile applies to an entire scenario in a load test. For more information, see Editing Think Times to Simulate Web Site Human Interaction Delays in Load Tests Scenarios.

Another example is the Think Time Between Test Iterations is a property that allows some time to elapse between the end of one test and the beginning of another. For more information, see Editing Load Patterns to Model Virtual User Activities.

There are additional properties that are not set by using the New Load Test Wizard, such as the IP Switching and Percent New Users properties. These properties are configured later by using the Load Test Editor. For a list of all the scenario properties that you can change, see Load Test Scenario Properties.

Note

IP switching is available only with the test agent. For more information, see Distributing Load Test Runs Across Multiple Test Machines Using Test Controllers and Test Agents.

Step 2.1 - Choosing a Load Pattern in the New Load Test Wizard

Load pattern specifies the number of virtual users who are active during a load test and the rate at which new users are added. The ability to have multiple load patterns in a load test allows you to achieve your testing objectives. You must specify a load pattern for each scenario in your load test. There are two types of simulated load available for you to choose when you create a load test by using the New Load Test Wizard:

Constant The constant load pattern is used to specify a user load that does not change during the load test. For example, when you run a smoke test on a Web application, you might want to set a light, constant load of 10 users.

Step   The step load pattern is used to specify a user load that increases with time up to a defined maximum user load. For stepping loads, you specify the Initial User Count, Maximum User Count, Step Duration (seconds), and Step User Count.

For example, a Step load with an Initial User count of one, Maximum User Count of 100, Step Duration (seconds) of 10, and a Step User Count of 1 creates a user load pattern that starts at 1, and increases by 1 every 10 seconds until it reaches 100 Users.

Additionally, you can configure your scenario to be goal-based after you create a load test by using the Load Test Editor.

Goal-based load pattern resembles the step pattern but adjusts the user load based on performance counter thresholds versus periodic user load adjustments. Goal based loads are useful for a variety of different purposes:

  • Maximizing output from the agents: measure the key limiting metric on the agent to maximize the output of the agents. Typically, it is CPU. However, it could also be memory.

  • Reaching some target resource level, typically CPU, on the target server, then measuring throughput at that level. This enables you to do run-to-run comparisons of throughput given a consistent level of resource usage on the server.

  • Reaching a target throughput level on the server.

For more information about how to configure a load test to use goal-based simulated load, see Editing Load Patterns to Model Virtual User Activities.

To specify load pattern in the New Load Test Wizard

  1. On the Load Pattern page of the New Load Test Wizard, select either Constant Load or Step Load.

    Note

    The Goal Based simulation option is only available from the Load Test Editor after the load test has been created.

  2. Depending on the pattern type that you selected, adjust the available settings. For example, set the start and maximum user counts for the step load pattern.

  3. After you choose the load pattern, choose Next to continue to the Test Mix Model page of the New Load Test Wizard. Proceed to Step 2.2 - Choosing a Test Mix Model in the New Load Test Wizard.

You can change the load pattern for a scenario later by using the Load Test Editor. For more information, see Editing Load Patterns to Model Virtual User Activities.

Step 2.2 - Choosing a Test Mix Model in the New Load Test Wizard

The test mix model specifies the probability of a virtual user running a given test in a load test scenario. This lets you simulate load more realistically. Instead of having just one workflow through your applications, you can have several workflows, which is a closer approximation of how end-users interact with your applications. For more information, see Emulating Expected Real-World Usage of a Web Site or Application in a Load Test Using a Test Mix Models and Editing Text Mix Models to Specify the Probability of a Virtual User Running a Test.

To specify test mix model in the New Load Test Wizard

  1. On the Test Mix Model page of the New Load Test Wizard, select the test mix model to use in your load test scenario. You can pick one of the following models:

    • **Based on the total number of tests   **Determines which Web performance or unit test is run when a virtual user starts a test iteration. At the end of the load test, the number of times that a particular test was run matches the assigned test distribution. Use this test mix model when you are basing the test mix on transaction percentages in an IIS log or in production data.

    • **Based on the number of virtual users  **Determines the percentage of virtual users who will run a particular Web performance or unit test. At any point in the load test, the number of users who are running a particular test matches the assigned distribution. Use this test mix model when you are basing the test mix on the percentage of users who are running a particular test.

    • **Based on user pace   **Over the course of the load test, each Web performance test or unit test is run a specified number of times per users, per hour. Use this test mix model when you want virtual users to run test at a certain pace throughout the load test.

    • **Based on sequential order   **Each virtual user runs the Web performance or unit tests in the order that the tests are defined in the scenario. The virtual user continues cycling through the tests in this order until the load test is complete.

  2. After you choose the test mix model, choose Next to continue to the Test Mix page of the New Load Test Wizard. Proceed to Step 2.3 - Specifying Test Mix in the New Load Test Wizard.

You can change the test mix model for a scenario later by using the Load Test Editor. For more information, see Editing Text Mix Models to Specify the Probability of a Virtual User Running a Test.

Step 2.3 - Specifying Test Mix in the New Load Test Wizard

A load test contains one or more scenarios. Each scenario contains one or more Web performance or unit tests in a test mix. The test mix of a scenario is a combination of various factors, such as the selection of tests that are contained within the scenario and the distribution of those tests within the scenario. The test mix should reflect the objectives of your test for a particular scenario. For more information, see Editing the Test Mix to Specify Which Web Performance, Unit and Coded UI Tests to Include in a Load Test Scenario.

Note

Distribution is a measure of the probability that a particular test will be selected by a virtual user during a load test run. Distribution is expressed as a percentage. Therefore, the sum of the distribution numbers for all the tests that are contained in a scenario is 100. For example, if a scenario contains only one test, the distribution for that test is 100%.

The test mix is just one part of specifying a load model. You can use various test mix modeling options to predict the expected real-world usage of the Web site or application that you are load testing. For more information, refer back to the procedure Step 2.2 - Choosing a Test Mix Model in the New Load Test Wizard.

To specify test mix in the New Load Test Wizard

  1. On the Test Mix page of the New Load Test Wizard, choose Add to add tests to the scenario. For more information, see Editing the Test Mix to Specify Which Web Performance, Unit and Coded UI Tests to Include in a Load Test Scenario.

  2. Select your preferred test mix by adjusting the sliders in the Distribution column, or by typing the percentage values directly into the % column. For more information, see About the Mix Control.

  3. After you choose the test mix, choose Next to continue to the Network Mix page of the New Load Test Wizard. Proceed to Step 2.4 - Specifying Network Mix in the New Load Test Wizard.

    Note

    The Browser Mix page will appear in the New Load Test Wizard if a Web Performance test was included in the test mix page.

You can change the test mix for a scenario later by adding and removing test from the scenario by using the Load Test Editor. For more information, see Editing the Test Mix to Specify Which Web Performance, Unit and Coded UI Tests to Include in a Load Test Scenario.

Step 2.4 - Specifying Network Mix in the New Load Test Wizard

A load test contains one or more scenarios. Each scenario contains one or more networks. The network mix of a scenario is a combination of two factors: the selection of networks that are contained within the scenario, and the distribution of those networks within the scenario.

Note

If your Load test does not contain Web performance tests, for example, if it contains only unit tests on an application that does not involve network access, you do not have to specify Network Mix.

The network mix should reflect the objectives of each particular scenario. For example, you could specify a mix of 75% LAN, and 25% Dial-up 56K.

Note

Distribution is a measure of the probability that virtual user will use a particular network type during a load test run. Distribution is expressed as a percentage. Therefore, the sum of the distribution numbers for all the networks that are contained in a scenario is 100.

For more information, see Specifying Virtual Network Types in a Load Test Scenario.

To specify network mix in the New Load Test Wizard

  1. On the Network mix page of the New Load Test Wizard, choose Add to add networks to the scenario. For detailed steps, see Specifying Virtual Network Types in a Load Test Scenario.

  2. Select your preferred network mix by adjusting the sliders in the Distribution column, or by typing the percentage values directly into the % column. For more information, see About the Mix Control.

  3. After you choose the network mix, choose Next to continue to either the Browser Mix page or the Counter Sets page of the New Load Test Wizard. Proceed to Step 2.5 - Specifying Browser Mix in the New Load Test Wizard or Step 3 - Specifying Counter Sets in the New Load Test Wizard.

    Note

    The Browser Mix page will be next if a Web performance test was added to the test mix. For more information, refer to the previous procedure Step 2.3 - Specifying Test Mix in the New Load Test Wizard.

You can edit the network mix for the scenario later by adding, removing, and adjusting the distribution by using the Load Test Editor. For more information, see Specifying Virtual Network Types in a Load Test Scenario.

Step 2.5 - Specifying Browser Mix in the New Load Test Wizard

A load test contains one or more scenarios, each of which contains one or more Web performance or unit tests. Each scenario contains one or more Web browsers, which are used to perform any Web performance tests in the scenario.

Note

If your Load test does not contain Web performance tests, for example if it contains only unit tests, the Browser Mix page of the New Load Test Wizard will not appear. For more information, refer to the previous procedure Step 2.3 - Specifying Test Mix in the New Load Test Wizard.

The browser mix of a scenario is a combination of two factors: the selection of browsers that are contained within the scenario, and the distribution of those browsers within the scenario. The browser mix should reflect the objectives of each particular scenario. For example, you could specify a browser mix of 95% Internet Explorer 8, and 5% Internet Explorer Mobile 6.

Note

Distribution is a measure of the probability that a particular browser will be selected by a virtual user during a load test run. Distribution is expressed as a percentage. Therefore, the sum of the distribution numbers for all the browsers that are contained in a scenario is 100.

For more information, see Editing the Test Mix to Specify Which Web Browsers Types in a Load Test Scenario.

To specify browser mix in the New Load Test Wizard

  1. On the Browser Mix page of the New Load Test Wizard, choose Add to add browsers to the scenario.

  2. Select your preferred browser mix by adjusting the sliders in the Distribution column, or by typing the percentage values directly into the % column. For more information, see About the Mix Control.

  3. After you choose the browser mix, choose Next to continue to the Counter Sets page of the New Load Test Wizard. Proceed to Step 3 - Specifying Counter Sets in the New Load Test Wizard.

You can edit the browser mix for the load test scenario later by adding, removing, and adjusting the distribution using the Load Test Editor. For more information, see Editing the Test Mix to Specify Which Web Browsers Types in a Load Test Scenario

Step 3 - Specifying Counter Sets in the New Load Test Wizard

Counter sets are a set of system performance counters that are useful to monitor during a load test. Counter sets are organized by technology, for example, ASP.NET or SQL counter sets.

Counter sets, like run settings, apply to all the scenarios in a load test instead of individual scenarios. In other words, counter sets apply to an entire load test and should reflect the goals of your entire load test.

Counter sets are gathered on computers that you specify. The association between a counter set and a computer that is used during a load test is a counter set map. For example, the Web server that you are testing might have ASP.NET, IIS, and .NET application counter set mappings. For more information, see Specifying the Counter Sets and Threshold Rules for Computers in a Load Test.

Note

If your load tests are distributed across remote machines, controller and agent counters are mapped to the controller and agent counter sets. For more information about how to use remote machines in your load test, see Distributing Load Test Runs Across Multiple Test Machines Using Test Controllers and Test Agents.

To specify counter sets in the New Load Test Wizard

  1. On the Counter Sets page of the New Load Test Wizard, you will see the existing counter sets in the Preview selections pane. There will already be default entries visible, including the LoadTest counter set; the LoadTest counter set is collected by the controller by default. If you have agents installed, you will also see the Agent counter set under Agent Computers.

  2. (Optional) Choose Add Computer to add a new computer to monitor. You will be prompted for a name. Type the name of a computer, and you will see nodes underneath the new entry that you can select. For example ASP.NET, IIS, SQL, and others. Select the check boxes in front of the nodes you want to select. The new counters appear in the Preview selections pane.

  3. For Computer Tags, type a tag to associate with the compute, for example, "TestMachine12 in lab3". Computer tags let you identify a computer by using an easy-to-recognize name. The tags are displayed in the Counter Set Mappings node in the tree in the Load Test Editor. More important, the tags are displayed in Excel reports, which help stakeholders identify what role the computer has in the load test. For example, "Web Server1 in lab2" or "SQL Server2 in Phoenix office". For more information, see Reporting Load Tests Results for Test Comparisons or Trend Analysis.

    After you choose the counter sets, choose Next to continue to the Run Settings page of the New Load Test Wizard. Proceed to Step 4 - Specifying Run Settings in the New Load Test Wizard.

Later, you can edit the computers and counter sets by using the Load Test Editor. For more information, see Specifying the Counter Sets and Threshold Rules for Computers in a Load Test.

Step 4 - Specifying Run Settings in the New Load Test Wizard

Run settings are a set of properties that affect an entire load test. The run settings determine such properties as the length of the test, warm-up duration, maximum number of error details reported, sampling rate, connection model (Web performance tests only), results storage type, validation level and SQL tracing. The run settings should reflect the goals of your load test. For more information, see Configuring Load Test Run Settings.

Note

Run settings, just like counter sets, apply to all the scenarios in a load test instead of individual scenarios.

To specify run settings in the New Load Test Wizard

  1. On the Run Settings page of the New Load Test Wizard, first select the length of the load test by choosing one of the following options:

    Use Specified duration

    1. Select the Load test duration option.

    2. Specify the Warm-up duration (hh mm ss). Use the hour, minute and second spin controls.

    3. Specify the Run duration (hh mm ss). Use the hour, minute and second spin controls.

    4. -or-

    Use Specified iterations

    1. Select the Test iterations option.

    2. Specify the number of times to run the test. Use the Test iterations spin control.

  2. Under Details, configure the following options:

    1. Use the Sampling rate spin control to specify the number of seconds between gathering sampling data.

    2. In the Description text box, type a description of the load test.

    3. Use the drop-down list box for Save Log on Test Failure to specify True to save the log file or False if you do not want to save the log file.

      Note

      By default, the Save Log on Test Failure is set to true. You should only set it to false if you are concerned about the potential performance impact that can occur from the additional overhead caused by the load agent.

  3. Use the drop-down list box for Validation level to select one of the following options:

    • High - invoke all validation rules

    • Medium - invoke validation rules marked medium or low

    • Low - invoke validation rules marked low

    For more information, see Using Validation and Extraction Rules in Web Performance Tests.

  4. After you choose your run settings, you have finished the New Load Test Wizard. Choose Finish if you are done, or use the orientation panel on the left side to return to any previous part of the wizard.

You can change the run settings you configured in the New Load Test Wizard and several additional run setting properties later, by using the Load Test Editor. For more information, see Load Test Run Setting Properties and Configuring Load Test Run Settings.

External resources

Guidance

Testing for Continuous Delivery with Visual Studio 2012 – Chapter 6: A Testing Toolbox

See Also

Tasks

Walkthrough: Creating and Running a Load Test Containing Web Performance Tests

Walkthrough: Creating and Running a Load Test Containing Unit Tests

How to: Add Additional Run Settings to a Load Test

Concepts

Load Test Scenario Properties

Load Test Run Setting Properties

Other Resources

Creating and Editing Load Tests

Editing Load Test Scenarios Using the Load Test Editor

Editing Think Times to Simulate Web Site Human Interaction Delays in Load Tests Scenarios

Creating Additional Scenarios for an Existing Load Test

Editing Text Mix Models to Specify the Probability of a Virtual User Running a Test

Configuring Load Test Run Settings