Getting Started with Visual Studio 2010 Ultimate - Load and Performance Testing
If you are new to Visual Studio 2010 Ultimate Load Testing - this article is for you. This gives you a quick overview and shows you how you can get started.
Getting Visual Studio 2010
There are a couple of ways you can get access to Visual Studio 2010 to try it out.
Option 1: Download a trial
Download a trial version for Visual Studio 2010 Ultimate Trial from https://www.microsoft.com/visualstudio/en-us/download . The other versions will not work for you, so select the Web Installer or the ISO download for Visual Studio 2010 Ultimate Trial. This download gives you the ability to run 250 users of virtual user load for the trial duration.
If you need to scale up load, you simply buy the Visual Studio Virtual User Pack 2010 through your traditional channel or buy it from the store at https://store.microsoft.com/microsoft/Visual-Studio-2010-Load-Test-Virtual-User-Pack/product/7AFF7BC9 (visit click for approximate pricing).
Once you’ve downloaded your trial, simply install Visual Studio 2010 Ultimate by following the "Typical" install option on a suitable machine and you are ready to try it ou
Option2: Pre-Configured VHD/VPC
You can download a preconfigured VHD that will allow you to focus on trying the product features instead of installing and configuring. If you want to install and configure experience as well you should consider downloading the trial.
To download this VHD/VPC go to the link below and follow the download instructions on the page.
This download is almost 9GB, so you should plan to kick off the download at night/off time as it’s a sizeable download and use the best practice of using the a download manager software to minimize download interruptions. Once you download it, run the appropriate .exe file (for your platform as in instructions on the above link). I downloaded the VisualStudio2010RTM_Win7VPC and had previously configured my Windows7 machine to have the VPC feature enabled.
For the size of the download, you will not only be able to give load testing a try (needs a couple more extra steps covered below), but also all of the manual, functional automation and other testing and development capabilities available in the Visual Studio 2010 release.
For the rest of the article I will focus on how to go about setting up a working Windows 7 VPC that I have downloaded and configured.
First Time VPC Configuration
Once VPC is configured and running, you should login to the VPC as “Christine Koch (Tester)” and use the password “P2ssw0rd” (without the quotes).
Once you have logged in (only for the first time), you must configure your browser defaults by just launching the browser and following the first time configuration wizard and accepting defaults.
Figure1: VisualStudio2010RTM_Win7VPC running on my machine
Figure 2: First time configuration for Visual Studio 2010 Ultimate
Additional Setup for VPC
Before you can use the downloaded and configured VPC specifically for load testing, you need to
1. Configure the SQL server that is already installed on the machine.
2. Configure Visual Studio 2010 to use the above database.
SQL is pre-installed on the image. You must Launch SQM Management Studio from the start menu, connect to the Database engine and open and run the SQL script located at C:\Program Files \Microsoft Visual Studio 10.0\Common7\IDE7\loadtestresultrepository.sql
Figure 3: Connect to SQM Management Studio
To configure Visual Studio to use the above database, from Visual Studio go to the “Test” menu and select “Manage Test Controllers…” and click on button to configure load test results store.
Figure 4: Configure Visual Studio to use SQL database (configure load test results not shown)
Ensure you use windows authentication and select “LoadTest2010” database from the list.
Figure 5: Configure Load Test Results Store
The image contains a pre-installed and configured sample application for you to use as your application under test. It is called TailSpinToys. It's an ecommerce application that allows end users to browse and buy toys. The application is running at https://Win-GS9GMUJITS8:8000/ or https://localhost:8000.
TIP: Launch the application and spend a few minutes to browse through the application and familiarize yourselves with the target application. You may want to book mark the application home page.
Web Performance Test Basics
Web performance tests enable you to generate http requests and responses, test for correctness of the responses, and measure response times and throughput. The primary scenario for web tests is to use them in a load test to generate load against a web application and measure web application performance. In order to efficiently generate load, Web performance tests work at the http layer, they do not drive the browser. For functional UI automation, you must use the Coded UI test type.
To create your first web test, from Visual Studio choose File \ New \ Project. In the dialog that follows, on the left hand side you see the installed template. Select the appropriate language of your choice on the left (Visual Basic or C#). Expand the node and choose “Test” to create a test project.
At the bottom of the dialog, choose a name for your test project. By default VS will create a new solution for you. In this walk through I have used C# for the language template and called my solution TailSpinToysLoadTestDemo
Figure 6: Create a test project with Visual Studio 2010
Note that you have an option to check in your tests into source control if you have setup Team Foundation Server. Load testing works even if you do have team foundation server installed and configured (although in the image there is a TFS instance running). Also, it is not necessary for you to check in web/load tests.
Visual studio creates a project and opens a Unit Test by default, ignore this.
TIP: If you haven’t used Visual Studio before you need to know that most of time for testing you will spend in the Solution Explorer pane. If this is not open by default, you must go to “View” menu and choose “Solution Explorer”
You must now “Add” a “Web Performance Test” by right clicking on the TailSpinToysLoadTestDemo
Figure 7: Add a Web Performance Test from Visual Studio
This opens the WebTestRecorder. Visual Studio 2010 using a browser helper object and WinINET based recorder to record your web tests. To record your web test, simply navigate to the target application URL https://Win-GS9GMUJITS8:8000/ or https://localhost:8000 and record your scenario. At each step of the way you can add a comment make your web test more readable by others.
Figure 8: Web Test Recorder opens in browser window navigate to target application to record.
In my case, I have done a complete recording of browsing for a model plane and going through a full purchase process. Once you’ve completed your recording, stop the recording and this will close the web test recorder and bring you back to the Visual Studio window with the recorded web test tab with all the recorded URLs. It will automatically re-run the web test to automatically identify and promote dynamic parameters so subsequent requests are parameterized and handled correctly.
For this specific application, and workload, the dynamic playback run fails (and this could happen depending on your application behavior) because the response URL validation test fails. This is a validation rule that is added out of the box for a web test, and the reason this fails is that in tail spin toys the application puts the receipt number as part of the response URL and this will vary each time you run the test; so you cannot compare this with earlier recorded response URLs that contains the receipt URL. For that reason, we can delete this validation for this scenario.
Figure 9: Delete Response URL validation rules
Once you have done this, you can re-run the web test to verify that it is passing.
Figure 10: Replay a web test to debug and fix any errors
TIP: In a real world scenario, there are will scenario’s where you will need to debug your web test to identify and resolve issues. Visit the content index for web and performance testing at my blog https://blogs.msdn.com/nkamkolkar/ and/or follow it to visit the post for debugging techniques https://blogs.msdn.com/b/edglas/archive/2010/03/24/web-test-authoring-and-debugging-techniques-for-visual-studio-2010.aspx
At this point you should have a web test against tail spin toys application that runs consistently and doesn’t fail. Playback allows you to view response time, error code status, and response size for a single user. For a given request/response, you see a preview of the html that was returned, and you can view the actual request and response data. The context parameters shows what values were in the virtual users context, and you can also see the results of any validation or extraction rules.
Your next step is to create a load scenario that can scale up the load, add this web test to it and run the load scenario.
Load Testing with Visual Studio 2010 Ultimate
Load tests enable to you simulate many users hitting an application at the same time. The primary scenario for load testing is performance testing. How will the server respond to so many users? Will response times be unacceptably slow? Will error rates be acceptable? Another is capacity planning, where you may want to understand how large a server you will need to support the expected user load. You start out creating a load test by walking through the load test wizard. The wizard will guide you through creating a scenario, which you can think of as a particular group of users. The load test scenario primarily contains:
1. The load pattern, which defines how users will running at any given point in the test
2. The test mix, which defines which test scripts they will be using. A web test should roughly be viewed as a single users walk through the app.
To keep this article a reasonable length, I will not go into the full details of the elements of the load test wizard since the articles primary goal is to get you started. Perhaps this is a subject of a subsequent article or blog.
When you add a load test, you will walk through a wizard. For getting started if you go through the basic wizard accepting the defaults you should be fine. Below, I will share some specific screens that are relevant to have a manageable load test for the VPC/VHD environment. For more information on load test wizard you can visit: https://msdn.microsoft.com/en-us/library/ms182572(v=VS.90).aspx
Figure 11: Load test wizard allows you to set up load scenario that scales up the load
You can select and/or define your load pattern for the scenario; given you are running this on a VPC/VHD change the default load from 25 users to say 5 virtual users as in figure 10.
Figure 12: Selecting a load pattern
Continue to choose defaults on the Test Mix Model screen. Following that, select “Add” and choose "TailSpinWebTest" or the equivalent name for the web test that you added earlier. Note that the "TestMethod1" is the default test that got created when you opened a test project, you can ignore this.
Figure 13: You can add web tests or unit tests to load scenarios using the load test wizard.
Continue to accept the defaults in the wizard until you to come to the Run Settings screen. Here you determine how your load test should run. Change the default Run duration to 1 minute again; once you deploy this to your test labs you can run 10+ hour runs easily.
Figure 14: Load test run settings
At this point you have a full load test scenario defined which you can run to generate load and then performance analysis. Note that a load test scenario in Visual Studio 2010 appears as “.loadtest” node and a web test appears as a “.webtest” node in the solution explorer under your test project. To edit either one at any time, you can simply double click the appropriate test type.
To run your load test simply hit the green “>” tool box button just below the tab title for the load test. This will start your load test and bring up the load test analyzer which shows your running load tests and performance metrics.
Figure 15: Running a load test
As you deploy the load test in your test lab and run real load tests, you will want to familiarize yourselves with the load test integration with ASP.NET profiler that allows you to not only get down to the exact method level detail for slow performing methods, but also show you the exact SQL queries and associated performance metrics. Visual Studio 2010 Ultimate also integrates with partner technologies for JAVA diagnostics. Below is an example screen shot of what ASP.NET profiler provides you insight into.
Figure 16: ASP.NET profiler integration with Tier Interaction Profiling enabled showing slow SQL queries
Working with Load Test Results Analyzer
Visual Studio 2010 Ultimate has integrated Load Test Analyzer that provides monitoring of the application and system under test in addition to providing visibility into key performance indicators, in addition to average, min, max, standard deviation, and 90th percentile data. You can view error rates, drag and drop metrics and also view innovative new visualizations such as VUSER ACTIVITY .
Figure 17: Load test analyzer monitors during the load test and provides for post run analysis
Load test analyzer also allows you to make analysis notes, generate run to run comparison as excel reports and also provides the ability to export your load test results as a load test archive which you can email to another Visual Studio 2010 Ultimate user for offline analysis.
Summary
Visual Studio 2010 Ultimate includes comprehensive load testing capabilities for your web applications. In this article I hope I have given you a way to quickly experience the basic flow and usage of the product and put you on a path to try out more advanced scenarios. There are many resources available in form of online MSDN documentation, blogs, videos, white papers, and forums to help you get to the next stage of using the product to solve real complex scenarios. You can find more information and resources at my recently started blog https://blogs.msdn.com/b/nkamkolkar