Capacity planning test methodology
Applies To: Windows Azure Pack
During the Windows Azure Pack capacity planning tests, the following resources, methods, and configurations were used.
Test resources
Resource |
Quantity |
Notes |
Total tenants |
2000 |
Passive user accounts. These accounts did not actively access the portal. |
Tenants used during test |
50 |
|
Average sessions per tenant used |
10 |
Ten sessions per each of the 50 tenants used is equivalent to testing with 500 tenants because each session creates its own connection to the server. |
Resources used per tenant |
5 to 10 |
These resources include virtual machines, networks, websites, SQL databases, MySQL databases, and Service Bus items. |
Subscriptions |
6000 |
Three subscriptions are allocated to each tenant. |
SQL Server tenant databases |
2000 |
|
MySQL Server tenant databases |
2000 |
|
Maximum VM roles |
100 |
|
Maximum virtual machines |
500 |
|
Maximum size for each tenant database |
50 MB |
|
Tenant websites |
2000 |
|
Management portal for tenants' virtual machines |
2 |
|
Tenant API virtual machines* |
1 |
|
SQL Server resource provider virtual machines |
1 |
|
MySQL resource provider virtual machines * |
1 |
|
SPF resource provider virtual machines * |
1 |
*One instance of the resource providers and tenant API can support the load from two instances of the management portal for tenants. The throughput of the management portal for tenants does not change if the load balancer settings are changed to enable both the instances of tenant API and resource providers. This behavior is expected because these components do not have bottlenecks.
Test methods
All load testing was performed by using Visual Studio Load Test, which was configured with five Visual Studio Test Agents to ensure that sufficient load could be generated.
Think time |
10 seconds |
The time between requests from a tenant |
Test duration |
2 hours |
The time for the total test |
Test scenario call ratios
The Windows Azure Pack capacity planning test scenario used the following combination of call ratios.
Calls |
Percentage |
GET |
95% |
Create/Update/Delete operations |
5% |
GET call ratios
GET calls made up 95 percent of total operations. These GET calls were distributed as shown in the following table.
GET calls |
Percentage |
Notes |
ListWebsiteAndRegions |
15% |
|
ListVmsAndVmRoles |
20% |
|
ListNetworks |
20% |
|
ListSqlDatabases |
15% |
|
ListMySqlDatabases |
15% |
|
ListServiceBusEntities |
5% |
|
WebsiteDrillDownCalls |
5% |
GET calls made while accessing the dashboard, scale, configuration pages of a website. |
VMDrilldownCalls |
5% |
GET calls made while accessing the dashboard, instances, scale configure tabs of a virtual machine. |
NetworkDrilldownCalls |
5% |
GET calls made while accessing the dashboard, rules, and site-to-site virtual private network (VPN), configure pages of a network. |
Create/update/delete call ratios
These calls made up 5 percent of total operations and were distributed as shown in the following table.
Create/update/delete call |
Percentage |
Create VM role |
8% |
Create VM |
10% |
Delete VM |
4% |
Delete VM role |
4% |
Attach disk |
4% |
Shut down VM |
4% |
Restart VM |
4% |
Create website |
10% |
Delete website |
5% |
Update scale |
2% |
Update website configuration |
2% |
Create SQL database |
10% |
Delete SQL database |
4% |
Change SQL database password |
4% |
Create MySQL database |
8% |
Delete MySQL database |
4% |
Create Service Bus namespace |
4% |
Create Service Bus queue |
4% |
Create Service Bus topic |
4% |
Delete Service Bus entities |
2% |