Changes to load test functionality in Visual Studio and cloud load testing in Azure DevOps
Azure Test Plans
Web performance and load test functionality in Visual Studio is deprecated. Visual Studio 2019 will be the last version of Visual Studio containing web performance and load test features (more details).
Cloud load testing (CLT) functionality in Azure DevOps is deprecated. The changes to cloud load testing functionality in Azure DevOps affect the following products for the load testing service:
Web-based experience for load testing in Azure DevOps (URL-based, HAR file, Apache JMeter and Visual Studio web test).
Running a load test in the cloud using Visual Studio (both auto-provisioned agents and self-provisioned agents).
On-premises load testing with Visual Studio and Test Controller/Test Agent
Visual Studio 2019 will be the last version of Visual Studio with the web performance and load test capability. Visual Studio 2019 is also the last release for Test Controller and Test Agent (installed through the 'Agents for Visual Studio SKU') for setting up a load test rig on-premises.
While no new features will be added, load test in VS 2019 will continue to be supported for any issues that may arise during the support lifecycle of the product. See Product Lifecycle and Servicing.
Cloud-based load testing service (CLT) availability timeframe for:
The scheduled removal of CLT on March 31, 2020 has been delayed due to COVID-19. While CLT is currently still available, we recommend that new customers do not onboard at this time, and we recommend that existing customers migrate to an alternative offering. Some options are listed in the following alternatives section.
Existing customers of CLT:
For existing Azure DevOps customers using cloud-load testing functionality in the Azure DevOps portal, this feature will no longer be available as of March 31, 2020.
For Visual Studio users leveraging cloud-based load testing, this feature will no longer be supported as of March 31, 2020. Visual Studio Enterprise customers can, however, continue to run load tests on-premises using Test Controller/Test Agent and will be supported for any issues that may arise during the support lifecycle of the Visual Studio version.
New customers of CLT:
New Azure DevOps organizations and new Azure subscriptions created after March 31st, 2019 will not have access to the cloud-based load testing functionality.
For customers who prefer to continue using a self-service model for load test and cloud-load testing, there are a number of free and commercial tools available.
For example, Apache JMeter is a free, popular open-source tool with a strong community backing. It supports many different protocols and has rich extensibility that can be leveraged to customize the tool to your needs. In addition, if you have been using Apache JMeter to run load tests using the cloud-based load testing service, you will be able to use these load tests with alternatives that support them.
The following table outlines how the concepts in Visual Studio load test map to those in Apache JMeter.
|Visual Studio Load test||Apache JMeter load test|
|Load test (.loadtest file)||JMX (.jmx file)|
|Webtest recorder||HTTP(S) test script recorder|
|Unit test||JUnit sampler|
|Selenium test||WebDriver sampler|
|Request and test plugins for extensibility||Extensibility using JMeter plugins|
|Distribution using Test Controller/Test Agent||Distribution in Apache JMeter|
Many commercial services such as Blazemeter support running Apache JMeter tests.
Download load tests and results
If you are using URL, HAR, or Visual Studio webtest-based load tests, you can download the tests by using the Export test to Visual Studio option and use the test through Visual Studio during the timeframe outlined above.
If you are using the cloud-based load testing service and want to access to results of any existing runs after the service goes offline, you can download the reports:
If you use Visual Studio load tests, URL-based load tests, or HAR-based load tests, download the reports in Visual Studio Enterprise for offline viewing.
If you use Apache JMeter, download the results file (.csv) for offline viewing using the reporters in JMeter.
Visual Studio web performance test (.webtest file)
Visual Studio web performance test (.webtest file) is tied to the load test functionality and is deprecated. Some customers have used .webtest for other purposes such as running API tests, even though it was not designed for that purpose. Many API testing alternatives are available in the market. SOAP UI is a free, open-source alternative to consider, and is also available as a commercial option with additional capabilities.