Web Test Authoring and Debugging Techniques


Josh Christie
Software Design Engineer
Microsoft Visual Studio 2005—Web and Load Testing

December 2005

Applies to:
   Visual Studio 2005 Team Suite
   Visual Studio 2005 Team Edition for Software Testers

Summary: Learn more about Visual Studio 2005 Web testing, including how the Web test engine and recorder work, and how to create effective Web tests. (17 printed pages)


Recording a Web Test
Running and Verifying a Web Test
Debugging Common Web Test Problems
Going Further with Coded Web Tests


This article is for testers and developers who want to become more experienced with the Web testing features in Microsoft Visual Studio 2005 Team Edition for Software Testers.


Microsoft Visual Studio 2005 Team Edition for Software Testers introduces a brand new set of powerful tools for Web and load testing. A load test of a Web application might span multiple machines and simulate tens of thousands of users, but at its heart it is a collection of Web tests. This article targets testers and developers who want to learn some techniques for creating effective Web tests and for debugging them to ensure they run as intended.

Recording a Web Test

Understanding the Web Test Recorder

The Web Test Recorder hooks into the Internet Explorer object model and listens for various navigation events. The primary benefit of this type of recording is that secure sockets layer (SSL) and authenticated Web sites can be recorded without special configuration requirements.

Another aspect of this recording approach is that dependent requests like images, cascading style sheet (CSS) files, and JavaScript files are not recorded into the Web test. Instead, these dependent requests are parsed out of HTML pages during Web test execution and are requested automatically. This feature helps make Web tests more resilient to cosmetic Web site changes and keeps the tests more focused on the actual use of the Web application. This feature can be disabled, if needed, by setting a request's ParseDependentRequests property to false.

A downside to this recording method is that it can fail to record requests made by JavaScript (for example, on AJAX sites), ActiveX controls, and some types of pop-up windows, since Internet Explorer does not always raise the necessary events. In most cases, these problems can be solved by manually adding the missed requests back into the Web test, as described later in this document.

Inserting Comments

Inserting comments during recording can be a helpful aid for creating an effective Web test, especially when the Web test contains many requests. You should use comments to make notes about what logical action is about to take place at different points in the Web test such as "Logging in," "Adding item X to the shopping cart," and so on. These comments can be very helpful when you later modify the Web test in the Web test editor.

You can also use comments to make notes about validation rules you need to add to ensure the Web test is successful. It is much easier to decide what needs to be validated on each request while recording and looking at the pages than when looking at a list of HTTP requests in the Web test editor.

Recording with Think Time in Mind

The ThinkTime property on a Web test request refers to the amount of time a user spends "thinking" on the current page before issuing the next request. Think time delays are used to approximate real user behavior during a load test. Because think time can dramatically affect the amount of load a Web test can generate, it can be globally disabled in a load test to apply greater load to a target server. Disabling think time allows you to issue requests to the server as fast as possible without delay between requests.

The Web test recorder automatically records think time at the same time that requests to the Web application are recorded. During recording, try to approximate the amount of time a user would normally spend on each page. Once the recording is complete, it is very important to check the recorded think time for each request. Inadvertently long think times can dramatically affect the rate at which a Web test generates requests. Think times are turned off by default in the Web test viewer. As a result, long think times might not be immediately apparent. When think times are turned on in the Web test viewer, you will see "Thinking…[n]" displayed in the HTTP Status column until the next request begins. Think times are turned on by default in load tests. The think time counter is paused when recording is paused and while entering a comment.

Running and Verifying a Web Test

Understanding the Web Test Engine

A key concept to understand about the Web test engine is that Web tests work at the HTTP layer. Web tests contain a list of HTTP requests; each of these requests is primarily made up of querystring parameters, form parameters, and a URL that targets a Web server. The Web test engine executes these HTTP requests, retrieves the responses from the server(s), and collects timing data.

Because the Web test engine works at the HTTP layer, it does not directly simulate client-side scripting like JavaScript or ActiveX controls. Web tests are concerned with generating load on a server. As a result, client-side scripting that only affects the appearance of a Web page is not significant to the Web test. Client-side scripting that sets parameter values or results in additional HTTP requests (such as AJAX) does affect the load on the server and might require you to manually modify the Web test to simulate the scripting. These types of modifications are described later in this paper.

A common misconception is that because recording occurs in Internet Explorer, Web tests must somehow execute using Internet Explorer. This is not the case. All requests are executed directly using the Web test engine; no interaction with Internet Explorer or any other browser occurs. The Web test engine communicates directly with the target Web server using standard HTTP request/response messages.

Similarly, the embedded Internet Explorer control on the Web Browser tab in the Web Test Viewer only displays response pages received by the Web test engine. The Web test engine writes the responses to a temporary location on disk and then loads the temporary files in the Internet Explorer control in the Web Test Viewer. Pages that are designed for other browsers can be verified using the Response tab if they do not appear correctly on the Web Browser tab.

Another source of confusion is that different browser templates can be selected when running Web load tests. These browser templates only affect the default set of HTTP headers sent with each request. The key header that Web servers use to determine the browser type is the UserAgent header. The Web test engine issues requests directly using standard HTTP protocol regardless of which browser template is selected.

Using the Web Test Viewer to Verify a Web Test

Before adding a Web test to a load test and running it for a long period of time, it is important to be sure that the test works exactly as intended. This is where the Web test viewer comes in. The Web test viewer allows you to watch a Web test as it runs, and to view all aspects of a previous test run.


Figure 1

Verifying a newly-created Web test goes beyond simply looking at the outcome of the test run and seeing whether it passed. For example, for a Web test without validation rules, passed only means that no exceptions were thrown, no rules failed, and no HTTP errors occurred. Verification includes making sure the Web test exhibits the correct behavior on the target Web application in addition to executing without any errors. It is important to review the response for each request to make sure that it is correct.

The following table lists items to check for when verifying a Web test, and some additional information about each type of problem.

Table 1

Problems to check for Additional Information/What to do
HTTP request error HTTP errors are signified by a response status code within the 400-599 range. Generally an HTTP error contains a response body that indicates what caused the problem.

For example, a 401 Unauthorized error might indicate that a user name and password need to be configured on the root node of the Web test. A 404 Not Found error might indicate that the Web application has changed since recording took place. A 500 Internal Server Error generally indicates a bug in the Web application.
Dependent requests that are not found A top-level request can appear to have failed because one of its dependent requests could not be found. This might indicate a problem with the Web application's HTML. These errors can be suppressed by disabling the ParseDependentRequests property on the request. This prevents dependent requests such as images, CSS, and javascript from being automatically parsed out of the HTML and requested.
Extraction and validation rule failures Extraction and validation rule failures are shown on the Details tab. These failures usually occur because the Web server returned a page that contained unexpected content, such as by redirecting to a login or error page.

Extraction rule failures frequently cause errors on subsequent requests such as, "X was not found in the Web test context."
Test-level exception A test level exception is shown in the Web test viewer as a node after the last successful request. Test level exceptions include exceptions in WebTestPlugins, PreWebTest and PostWebTest event handlers, and exceptions that are not specific to a particular request in a coded Web test.
Request-level exception Request level exceptions cause an individual request to fail, but allow the Web test to continue. These exceptions include those thrown from WebTestRequestPlugins, PreRequest, and PostRequest event handlers, and extraction and validation rules. The exception message, as well as a stack trace (if available), are shown at the bottom of the Details tab in the Web test viewer.
Incorrect page content returned by server Verifying that the Web server returns the correct content is often done by visually inspecting each page in the Web Browser and Response tabs of the Web test viewer.

Validation rules can be used to automate this process once the Web test is verified to work correctly.

Debugging Common Web Test Problems

Web Server Responds Differently During Execution than Recording

In a perfect world, you would record a set of requests to a Web application, run the Web test, and receive the same responses from the server that you saw during recording. Unfortunately, Web applications sometimes behave differently during Web test execution than they do during recording.

This type of problem can occur for a variety of reasons and often results in an error something like the following:

     Request failed: $HIDDEN1.__VIEWSTATE not found in test context.

This error occurs when the Web test attempts to use a hidden field in the Web test context that it was unable to locate and extract from a previous response page it received.

The following screenshot demonstrates this problem when it is caused by a server error. In the second-to-last request, the server error caused hidden fields the next request depends on to not exist on the page.


Figure 2

There are many reasons why a server might respond differently during execution than it did during recording. Some of the more common reasons are summarized in the following sections. In all cases, validation rules can be added to requests to automatically verify that the server responds with the correct content.

One-Time-Use Data

One common cause of this problem is one-time-use data, such as when a Web application creates a unique user name. Playing back this kind of Web test without adding data binding or a random value can result in the Web application displaying an error when the test attempts to create a duplicate username.

JavaScript Redirects

A Web application that uses JavaScript redirects (setting window.location) might respond differently during execution than during recording because the Web test engine does not run script code. This type of problem can be easily corrected by inserting the URL the script redirects to and moving necessary extraction rules to the new request from the page that performs the redirect. Because this problem exists in the Web test immediately after recording, the only extraction rule likely to be present is ExtractHiddenFields.

Redirects to an Error Page

When there is a server error, a Web application might redirect to an error page, but not return an HTTP 400 or 500 level response code. This indicates that there is either a problem in the Web application itself or a problem in the requests being issued by the Web test. This particular problem is discussed in more detail in this blog post.

Handling View State and Other Dynamic Parameters

Even before ASP.NET 1.0 introduced the __VIEWSTATE hidden form field, Web applications used dynamically-generated form and querystring parameters to pass information between pages. These dynamic parameters require special consideration in a Web test because they can change every time the Web test runs. A Web test with hard-coded parameter values might not work for very long after recording, or even at all.

Web tests enable testing with dynamic parameters by using extraction rules and context binding. Extraction rules are placed on requests for pages that will contain a dynamic value. When the extraction rule runs, it extracts the dynamic value into the Web test context using a configurable name such as "myparam". A subsequent request then contains a querystring or form parameter with a value of {{myparam}}. When the Web test runs, the value in the Web test context is substituted for {{myparam}}.

The sequence of events for an extraction rule is as follows:

  1. The Web test engine begins executing Request1.
  2. Request1 is sent to the target server.
  3. A response is received from the target server.
  4. The extraction rule on Request1 runs on the response page.
  5. The extraction rule places an entry in the Web test context.
  6. The Web test engine begins executing Request2.
  7. Querystring parameters, form parameters, and any other context-bound values on Request2 are substituted from the Web test context.
  8. Request2 is sent to the target server.

Automatic Hidden Field Tracking

Web tests contain special support for handling dynamic hidden fields, such as __VIEWSTATE. When a Web test is recorded, hidden fields are automatically matched with form and querystring parameters. What a match is found, the ExtractHiddenFields rule is applied to the request generating the source of the hidden field. At this time, context bindings are applied to parameters on the request, making use of the hidden fields.

ExtractHiddenFields is a special extraction rule because, unlike rules that extract one value into the context, it extracts every hidden field value on the page into the Web test context. Normal extraction rules use the ContextParameter property to determine the name to use for the context parameter, but ExtractHiddenFields uses that property only to differentiate from multiple groups of hidden fields that might be in the context simultaneously. For example, an ExtractHiddenFields rule with ContextParameter set to 1 will extract __VIEWSTATE as "$Hidden1.__VIEWSTATE".

Fixing __EVENTTARGET and other hidden form fields modified by JavaScript

When a hidden field is modified by Javascript in an OnClick event handler, it is possible that automatic hidden field binding will be incorrectly applied. This is a known bug in the release version of Visual Studio 2005.

<input name="btnNext" type="button" value="Next" 
onclick="__doPostBack('btnNext', '');" />

With ASP.NET sites, this problem most commonly occurs when a Web control calls the __doPostBack() JavaScript method to set the __EVENTTARGET hidden field as shown above. Automatic hidden field binding results in the form parameter having a value such as {{$HIDDEN1.__EVENTTARGET}}, instead of the actual value — btnNext. To correct this problem, the parameter value must be set to the value being set in Javascript (for example, btnNext).

Requests Missed During Recording

As discussed in the Understanding the Web Test Recorder section, some requests might not be recorded by the Web Test Recorder (for example, AJAX requests and some pop-up windows). Fortunately there is a great tool written by Eric Lawrence called Fiddler. that can help with this. Fiddler works by acting as a proxy server and can intercept all HTTP traffic (no SSL support yet). Two options are described below for using Fiddler to correct a Web test that cannot be recorded with the standard Web Test Recorder.


Figure 3

Saving a Web Test from Fiddler

When the Web Test Recorder misses some AJAX, ActiveX, or pop-up window requests, one option is to record the entire test using Fiddler. Fiddler can save a series of captured requests as a .webtest file that can be added to a Visual Studio 2005 test project.

This option is best used when you are unable to record a large number of requests using the Web test recorder. The primary limitations with this option are that Web tests created by Fiddler do not make use of automatic hidden field tracking (such as for __VIEWSTATE) and do not filter out dependent requests like images, CSS, and JavaScript.

Manually Adding Requests from Fiddler

Another option for missed requests is to use Fiddler to determine which requests you need to add to the Web test manually. This method is best when a small number of requests are missed by the Web Test Recorder since you can still benefit from features such as automatic hidden field tracking and filtering of dependent requests.

In this case, it is best to record the Web test using Fiddler and the Web test recorder simultaneously. This allows you to compare the two recordings to discover missing requests. It is also helpful to insert a comment during recording if it is apparent that a request is being missed, for example if you know an AJAX request occurred. This comment acts as a placeholder for the manually created request.

The following screenshots demonstrate manually creating a request from a Fiddler capture. Remember to add any necessary extraction rules, context binding for parameter values, and ThinkTime to the manually created request.


Figure 4


Figure 5

Going Further with Coded Web Tests

Normal Web tests are designed to handle a wide variety of Web test scenarios. Data binding, extraction rules, plug-ins, and context parameters provide a lot of control over Web test execution, but sometimes even more control is needed. Coded Web tests provide the most control and extensibility by offering a .NET API you can use from Visual C# or Visual Basic. With this API, you can use the entire functionality of the Web test engine including features described in the next section.

When to Generate Code, and Why

A coded Web test should only be generated once the limits of normal Web tests are reached. The most obvious limits in normal Web tests are looping (you cannot run a subset of the requests multiple times) and branching (you cannot conditionally execute a set of requests). Other reasons to generate code include fine-grained event handling and programmatically setting a parameter value.

It is easier and less error-prone to edit a Web test using the graphical Web test editor than it is to edit code directly. As a result, you should get the Web test as far along as possible before generating code. This includes adding extraction and validation rules, setting up data binding, creating transactions around groups of requests, and adjusting think times. The code generated for all these aspects of a Web test provides a solid foundation for further customization of the coded Web test.

The following sections demonstrate some features and examples that require, or would be significantly simplified by, coded Web tests.


Use coded Web tests when the Web test must conditionally issue a different set of requests.

This is demonstrated in the following example where the Web test logs into a Web site using databound user credentials and must create a new user account if the user does not exist in the system.

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.VisualStudio.TestTools.WebTesting;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

[DataSource("DataSource1", "Provider=Microsoft.Jet.OLEDB.4.0;Data 
Source=\"test.mdb\"", DataBindingAccessMethod.Sequential, "Credentials")]
[DataBinding("DataSource1", "Credentials", "UserName", 
[DataBinding("DataSource1", "Credentials", "Password", 
public class BranchingCoded : WebTest
  public BranchingCoded()

  public override IEnumerator<WebTestRequest> GetRequestEnumerator()
    // Go to the Web application's home page

    WebTestRequest request1 = new WebTestRequest("http://testserver/website");
    request1.ThinkTime = 4;
    yield return request1;

    // Go to the login page

    WebTestRequest request2 = new WebTestRequest("http://testserver/website/Login.aspx");
    request2.ThinkTime = 16;
    ExtractHiddenFields rule1 = new ExtractHiddenFields();
    rule1.ContextParameterName = "1";
    request2.ExtractValues += new EventHandler<ExtractionEventArgs>(rule1.Extract);
    yield return request2;

    // Attempt to login 

    WebTestRequest request3 = new WebTestRequest("http://testserver/website/Login.aspx");
    request3.ThinkTime = 6;
    request3.Method = "POST";
    FormPostHttpBody request3Body = new FormPostHttpBody();
    request3Body.FormPostParameters.Add("__VIEWSTATE", this.Context["$HIDDEN1.__VIEWSTATE"].ToString());
    request3.Body = request3Body;
    yield return request3;

    // If the login failed, create a new user account

    if (LastResponse.StatusCode != System.Net.HttpStatusCode.OK)
      WebTestRequest request4 = new WebTestRequest("http://testserver/website/register.aspx");
      request4.ThinkTime = 9;
      ExtractHiddenFields rule2 = new ExtractHiddenFields();
      rule2.ContextParameterName = "1";
      request4.ExtractValues += new EventHandler<ExtractionEventArgs>(rule2.Extract);
      yield return request4;

      WebTestRequest request5 = new WebTestRequest("http://testserver/website/register.aspx");
      request5.ThinkTime = 5;
      request5.Method = "POST";
      FormPostHttpBody request5Body = new FormPostHttpBody();
      request5.Body = request5Body;
      yield return request5;

    // Go view the user's profile now that the user is logged in

    WebTestRequest request6 = new WebTestRequest("http://testserver/website/userprofile.aspx");
    yield return request6;


Use coded Web tests when the Web test must loop to issue a number of requests dynamically determined during the test.

This is demonstrated in the following example where the Web test performs a search and then follows every link in the search results.

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.VisualStudio.TestTools.WebTesting;

public class LoopingCoded : WebTest
  public LoopingCoded()

  public override IEnumerator<WebTestRequest> GetRequestEnumerator()
    // Issue a search for the term "Microsoft"

    WebTestRequest request7 = new WebTestRequest("http://testserver/website/Search.aspx");
    request7.ThinkTime = 20;
    request7.Method = "POST";
    FormPostHttpBody request7Body = new FormPostHttpBody();
    request7Body.FormPostParameters.Add("txtSearch", "Microsoft");
    request7.Body = request7Body;
    yield return request7;

    // Loop through each anchor tag in the search result and issue a request to each tag's target url (href)

    foreach (HtmlTag tag in this.LastResponse.HtmlDocument.GetFilteredHtmlTags("a"))
      WebTestRequest loopRequest = new WebTestRequest(tag.GetAttributeValueAsString("href"));
      yield return loopRequest;

Fine-Grained Event Handling

Use coded Web tests when you want to apply PreRequest and PostRequest event handlers only to specific requests. Normal Web tests apply WebTestRequestPlugins (which handles both events) to all requests in the Web test.

This is demonstrated in the following example where the Web test logs the response body of a two requests to disk for debugging or baselining purposes.

using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using Microsoft.VisualStudio.TestTools.WebTesting;

public class EventHandlingCoded : WebTest
  public EventHandlingCoded()

  public override IEnumerator<WebTestRequest> GetRequestEnumerator()
    WebTestRequest request1 = new WebTestRequest("http://testserver/website");
    request1.ThinkTime = 8;
    yield return request1;

    // Log this response out to a file

    WebTestRequest request2 = new WebTestRequest("http://testserver/website/products.aspx");
    request2.ThinkTime = 2;
    request2.QueryStringParameters.Add("CategoryID", "14", false, false);
    request2.PostRequest += new EventHandler<PostRequestEventArgs>(request2_PostRequest);
    yield return request2;

    WebTestRequest request3 = new WebTestRequest("http://testserver/website/products.aspx");
    request3.ThinkTime = 2;
    request3.QueryStringParameters.Add("CategoryID", "15", false, false);
    yield return request3;

    // Log this response out to a file, too

    WebTestRequest request4 = new WebTestRequest("http://testserver/website/products.aspx");
    request4.ThinkTime = 1;
    request4.QueryStringParameters.Add("CategoryID", "20", false, false);
    request4.PostRequest += new EventHandler<PostRequestEventArgs>(request4_PostRequest);
    yield return request4;

  void request2_PostRequest(object sender, PostRequestEventArgs e)
    File.WriteAllBytes("c:\\request2.html", e.Response.BodyBytes);

  void request4_PostRequest(object sender, PostRequestEventArgs e)
    File.WriteAllBytes("c:\\request4.html", e.Response.BodyBytes);

Simulating JavaScript with Coded Web Tests

As described earlier, the Web test engine works at the HTTP layer and does not run JavaScript. Web sites that depend on JavaScript in a way that affects the HTTP layer can used coded Web tests to simulate the logic normally performed by JavaScript.

The following is a simple Web page that collects a telephone number and uses JavaScript to strip out some non-numeric characters when the form is submitted.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 
<html xmlns="http://www.w3.org/1999/xhtml" >
  <title>Untitled Page</title>
  <script type="text/javascript">
    //get rid of some common non-digit characters
    // in the phone number string
    function FixPhoneNumber() {
      var phoneNumberElement = document.getElementById('phoneNumber');
      var number = phoneNumberElement.value;
      number = number.replace('-', '');
      number = number.replace('(', '');
      number = number.replace(')', '');
      number = number.replace(' ', '');
      phoneNumberElement.value = number;
      return true;
  <form action="http://testserver/testwebsite/showparameters.aspx" method="post">
    <input name="phoneNumber" id="phoneNumber" type="text"/>
    <input name="submit" type="submit" value="Submit" onclick="FixPhoneNumber()" />

Problems can arise testing this sort of page when using data binding with the phoneNumber field. If the data source does not contain phone numbers in the correct format, unexpected data can be submitted to the server, causing the Web test to fail. Ideally, the server validates and fixes the data in case JavaScript is disabled, but Web tests might still be needed to simulate the client-side script running.

In most cases, a coded Web test can easily simulate the JavaScript that affects an HTTP request. The following example demonstrates simulating the JavaScript shown above.

    public override IEnumerator<WebTestRequest> GetRequestEnumerator()
      WebTestRequest request1 = new WebTestRequest("http://testserver/testwebsite/default.aspx");
      yield return request1;

      WebTestRequest request2 = new WebTestRequest("http://testserver/testwebsite/showparameters.aspx");
      request2.Method = "POST";
      FormPostHttpBody request2Body = new FormPostHttpBody();
      //get the databound phone number from the context and
      // strip out certain characters to simulate JavaScript
      string phoneNumber = this.Context["PhoneNumber"].ToString();
      phoneNumber = phoneNumber.Replace("-", "");
      phoneNumber = phoneNumber.Replace("(", "");
      phoneNumber = phoneNumber.Replace(")", "");
      phoneNumber = phoneNumber.Replace(" ", "");

      request2Body.FormPostParameters.Add("phoneNumber", phoneNumber);
      request2Body.FormPostParameters.Add("submit", "Submit");
      request2.Body = request2Body;
      yield return request2;

Running and Debugging Coded Web Tests

To run a coded Web tests select the test in the Test View or Test Manager windows and click the Run toolbar button. Once a coded Web test starts running, you can double-click in the Test Results window to display the Web test viewer. The same features are available for verifying that coded Web tests are correct as for normal Web tests.

Because Web testing is integrated with Visual Studio, you can debug a coded Web test by selecting Debug instead of Run as shown in the following screenshot. Breakpoints can be set anywhere in a coded Web test and all standard debugging mechanisms are available.


Figure 6


Hopefully you now have a better understanding of creating, verifying, and debugging Web tests in Visual Studio 2005 Team Edition for Testers. Additional information and support resources can be found in the Web testing online documentation, my blog, and our team's MSDN support forum.


About the author

Josh Christie is a software designer engineer on the Visual Studio Team System Web and Load Testing team. He works primarily on the Web test engine, coded Web tests, and the browser recorder. For more information about Web tests, check out his blog at http://blogs.msdn.com/joshch/.

© Microsoft Corporation. All rights reserved.