Site Server - Using Transaction Cost Analysis for Site Capacity Planning

August 1999 

Methodology

The purpose of capacity planning for Internet services is to support transaction throughput targets with acceptable response times, while minimizing the total dollar cost of ownership of the host platform. Conventional solutions often attempt to evaluate the cost of Internet services by extrapolating from generic benchmark measurements. However, a more effective methodology is based on Transaction Cost Analysis (TCA).

You can use TCA methodology to simulate client transactions on the host server by a load-generation tool that supports standard network protocols. By varying client load, transaction rate can be related to resource utilization. A usage profile designed to capture anticipated user behavior can then be defined. The usage profile determines the throughput target and other important transaction parameters from which resource utilization and capacity requirements can then be calculated.

TCA methodology can be used to measure the cost of individual shopper operations, such as browsing, adding an item to the shopping cart, checking out, searching, registering, and so on. The capacity of a site is determined by dividing the costs of shopper operations into the total CPU capacity available for the server.

Once you understand the cost and relative frequency of each shopper operation, you can view the performance numbers for a site's pages to see where the bottlenecks are, and to see where you can optimize performance to improve the site's capacity.

For example, TCA can help you determine which operations are the most costly, which pages are accessed the most frequently, and so forth. TCA can also help you do "what if" scenarios with the performance data to see how performance improvements (or degradation) affect your site. Once you know the cost of each operation, you can target those pages with high cost or high frequency of use for performance optimization. If you need additional capacity, identifying and lowering the cost of expensive pages will help. In addition, you'll be able to clearly see the impact of adding features to a page and can plan to install additional hardware to service the load generated by additional shoppers or new code releases.

A Transaction Cost Analysis includes the following components:

  • Set Up a User Profile 

  • Measure the Cost of Each Operation 

  • Estimate Capacity 

  • Calculate Capacity 

  • Verify Capacity 

Set Up a User Profile

The first step to running a Transaction Cost Analysis is to compile a user profile by analyzing customer Internet Information Server (IIS) logs. It's best to use as long a period of time as possible (at least a week's worth of IIS logs, if they are available), in order to obtain good averages. You can use the Site Server 3.0 Commerce Edition Usage Analysis tool (Report Writer Catalog) to gather the following types of numbers:

  • Number of visitors 

  • Number of visits to each page 

  • Number of checkout transactions 

  • Rate at which transactions are taking place 

The Detail User Report and the Detail Visit Report provide this information. The number of visits to each page is used to profile typical shopper operations for the site. This information comprises the user profile for the analysis. The following example shows a user profile for a sample site:

Shopper Operations

Page Hits

Ratio Of Hits

Default (Homepage)

58285

22.82%

Search (Good)

36641

14.35%

Search (Bad)

2599

1.02%

Add Item

4482

1.76%

Add Item + Delete

2726

1.07%

Add Item + Checkout

2800

1.10%

Browse

93507

36.61%

Login

4418

1.73%

Register

2700

1.06%

Zip Code

40771

15.96%

View Cart

6452

2.53%

Totals

255381

100.00%

Although the numbers in the example table account for only 10% of the pages (top page requests), they account for 90% of the traffic to this site. Other traffic numbers (including image and miscellaneous requests that generate a much lighter load) are not included in shopper operations in order to simplify the example.

Another important aspect of setting up a user profile is to determine the rate at which transactions are taking place. You use these numbers to create scripts for loading the sample site to verify the calculated capacity. For example, transaction rates at our sample site were:

Average session length

10 minutes

Checkout transaction rate/minute

Checkout transaction rate/minute is equal to the number of transactions/number of visitors/time. At our sample site, there was 1 transaction/31 visitors/10 minutes, or 0.003 transactions/visitor/minute.
If you include non-cookie visitors, the transaction rate was approximately 1 transaction/40 visitors/10 minutes, or 0.0025 transactions/visitor/minute.

Browse-to-buy ratio

Approximately 40:1 (for every 40 visitors browsing, 1 visitor performs a checkout/buy transaction). This coincides with a Forrester Report (The Look to Buy Imperative, 4/98).

Concurrent users

40

Measure the Cost of Each Operation

Once you have established a user profile, you can measure cost by creating a load-monitoring script to exercise each of the identified shopper operations. You then measure CPU utilization at that load level. InetMonitor, a load generation/simulation tool provided with the Microsoft BackOffice Resource Kit, is a good tool to use for this task. A set of sample scripts for measuring the cost of various shopper operations is provided in the section on Cost Measurement Scripts.

The objective of running a script exclusively for an individual operation is to load the IIS/ASP server with as many requests as possible, in order to achieve maximum ASP throughput per second. Maximum ASP throughput occurs when you measure a drop in ASP throughput with a higher shopper load or when you measure a sudden increase in operation latency or ASP requests queued. You can calculate operation latency using the following formula:

Average ASP latency (ASP execution time + ASP wait time) * number of ASP pages

Since it is difficult to determine averages before running the scripts, use an average ASP latency of less than 2 seconds, to account for additional download latency of modem users.

CPU utilization is recorded using the Microsoft Windows NT Performance Monitor when you reach maximum ASP requests/second (Active Server Pages Object—ASP requests/second, and the System Object—% Total Processor Time).

Increasing the number of users running the script increases the load. Typically, ASP requests/second and CPU utilization grow with the number of users. However, if adding more users results in lower ASP requests/second throughput, the previous step achieved the maximum throughput. If the number of ASP requests/second continues to grow until CPU utilization reaches 100%, the number of ASP requests/second at that point is then the maximum.

The result of TCA measurement is the CPU cost for each shopper operation, calculated on the basis of Megacycles (Mcycles) or 1Mhz. For example, a dual processor 400Mhz Xeon Pentium II has a total capacity of 800 Mcycles. Using the maximum number of ASP requests/second, you can calculate the cost per ASP request, as follows:

CPU utilization * number of CPUs * speed of CPU (in MHz) / ASP requests per second

For example, if you are using a server with two CPUs, and if the browse operation results in 11.50 ASP requests/second with CPU utilization of 84.10%, the cost per ASP page is then 84.10% * 2 * 400 / 11.50 = 58.50 Mcycles.

However, before you can calculate the cost of an operation, you also need to know the number of ASP pages involved in a shopper operation. For example, checkout operations typically involve several ASP pages (shopper information page, credit card page, shipping page, confirmation page, and so on). You calculate the cost of a shopper operation by multiplying the number of ASP pages involved by the cost per ASP page. Some ASP pages, such as action pages, are never seen, since they are usually posted to and redirected to a continuing page. You must also account for hidden ASP pages.

In the browse operation example, there is only one ASP page. Thus, the cost of the browse operation is 58.50 Mcycles. If there were two ASP pages, the cost would have been 58.50 * 2 = 117 Mcycles, and so on.

TCA measurement results in a set of CPU costs for each shopper operation. In the following table, the number of CPUs is two Xeon Pentium II processors. The clock speed of each CPU is 400MHz.

Shopper Operations

CPU Utilization @ Maximum ASP Requests per Second

Maximum ASP Requests per Second

Cost (Mcycles)

Cost per ASP page

Number of ASP pages

Cost per Operation

Default (Homepage)

57.80%

8.60

462.4

53.77

1.00

53.77

Search (Good)

63.80%

3.40

510.4

150.12

1.00

150.12

Search (Bad)

61.80%

23.50

494.4

21.04

2.00

42.08

Add Item

85.00%

5.20

680.0

130.77

2.00

261.54

Add Item + Delete

80.90%

7.20

647.2

89.89

4.00

359.56

Add Item + Checkout

30.40%

4.10

243.2

59.32

7.00

415.24

Browse

84.10%

11.50

672.8

58.50

1.00

58.50

Login

81.80%

7.80

654.4

83.90

3.00

251.69

Register

81.50%

20.90

652.0

31.20

2.00

62.39

Zip Code

82.80%

18.90

662.4

35.05

2.00

70.10

View Cart

67.60%

6.40

540.8

84.50

1.00

84.50

Estimate Capacity

Estimating capacity requires some assumptions about the user profile. Typical shopper operations are those of an average user. The behavior of actual shoppers can cause random peaks and valleys; however, over time these peaks and valleys statistically even out to average behavior. The user profile you create reflects average user behavior. In order to estimate capacity, you need to assume an average user.

In the following example, there are 11 shopper operations and transactions that occur 0.025 times every 10 minutes. To simplify the example, we created the user profile script to perform the 11 shopper operations in 11 minutes. This generated a ratio of operations for use in capacity calculations and for a verification script.

Although the sample site has a transaction ratio of 0.0025 checkouts/ user/ minute (1 checkout/40 users/10 minutes), we used 0.01 checkouts/ user/ minute (1 checkout/10 users/10 minutes = 0.01, or 0.1 checkouts per 10 minutes), to reflect a heavier load. This number is based on a 10:1 browse-to-buy ratio, instead of the actual 30:1 (or 40:1 when non-cookie shoppers are included) browse-to-buy ratio.

Although we used a heavier transaction rate (four times more checkouts), we kept the browse-to-buy ratio at 40:1 to anticipate future traffic at a higher volume (with the same page ratio mix) and to facilitate the simple conversion of capacity as number of concurrent users for the sample site. In this case, an estimated capacity of one user would translate to four users on the production/live sample site.

We then calculated the number of shopper operations during the course of 11 minutes, and, following that, the number of shopper operations per second. We need to know the number of shopper operations per second because the total shopper operation cost number is expressed in terms of clock speed per second (400MHz CPU speed is 400 Mcycles per second).

The cost table for the user profile is as follows:

Operations

Ratio Of Hits

User Profile Operations (over 11 minutes, 660 seconds)

User Profile Operations per Second (User Profile Operations / 660 seconds)

Cost per Operation (Mcycles)

Cost per User Operation per Second(Cost per Operation * User Profile Operations per Second)

Default (Homepage)

22.82%

2.5

0.003804

53.77

0.2045

Search (Good)

14.35%

1.6

0.002391

150.12

0.3590

Search (Bad)

1.02%

0.1

0.000170

42.08

0.0071

Add Item

1.76%

0.2

0.000293

261.54

0.0765

Add Item + Delete

1.07%

0.1

0.000178

359.56

0.0640

Add Item + Checkout

1.10%

0.1

0.000183

415.22

0.0759

Browse

36.61%

4.0

0.006102

58.50

0.3570

Login

1.73%

0.2

0.000288

251.69

0.0726

Register

1.06%

0.1

0.000176

62.39

0.0110

Zip Code

15.96%

1.8

0.002661

70.10

0.1865

View Cart

2.53%

0.3

0.000421

84.50

0.0356

Total

 

11.0

 

 

1.4496

The interesting number here is the total (1.4496), which indicates that the cost of the total user profile is 1.4496 Mcycles per user. This number reflects the cost of an average user executing shopper operations described by the user profile. This number is then used to estimate the capacity of the site, based on the assumed user profile.

For example, the cost of 100 concurrent users is 100 * 1.4496 = 145 Mcycles. These calculations are shown in the section on Calculate Capacity, and can also be plotted on a graph.

Another interesting number is the cost of a checkout transaction (Add Item + Checkout): it's very high, at 415.22 Mcycles. This translates to a server capacity of only two checkout transactions per second, since our limit for this machine is 800 Mcycles. Although this seems like a very low capacity for this operation, the average frequency of this operation is also very low (0.000183 per second), so server capacity for concurrent shoppers is not overly affected. The cost per user operation per second shows a very low number at 0.0759 Mcycles (approx. 5% of the cost of the entire user profile).

On the other hand, the browse operation has very low cost, at 58.50 Mcycles. However the frequency of the browse operation is very high, at 0.006102, so the browse operation places quite a load on the site. The cost per user operation shows a high number of 0.3579 Mcycles (approx. 25% of the cost of the entire user profile).

The search operation shows a moderate cost, at 150.12 Mcycles. However, the frequency of the search operation is also high, at 0.002391, so it results in a high cost per user operation per second of 0.3590 Mcycles (approx. 25% of the cost of the entire user profile).

The numbers suggest that it would be best to start optimizing performance with the browse pages and the search pages to improve shopper capacity at the site.

Calculate Capacity

As you can see in the following table, we calculate the maximum capacity of the site to be 550 concurrent shoppers, with a cost of 797.5 Mcycles. This is the maximum limit, since the two CPUs provide the site with a budget of 800 Mcycles of capacity. A more conservative approach is to use 70% CPU utilization. This results in a CPU budget of 560 Mcycles, with a calculated maximum capacity between 350 and 400 concurrent shoppers.

Shoppers

Cost (Mcycles)

0

0.0

50

72.5

100

145.0

150

217.5

200

290.0

250

362.5

300

435.0

350

507.5

400

580.0

450

652.5

500

725.0

550

797.5

600

870.0

Verify Capacity

You can verify site capacity by running a script that reflects the user profile with an increasing number of shoppers, and then compare the results against the table in the Calculate Capacity section. The verification script is simply a collection of all individual TCA measurement scripts aggregated and run as a single script.

The script records CPU utilization, expressed as the cost of the load, charts utilization, and compares it with TCA measurements for the individual operations. For the sample customer site (dual Pentium II Xeon 400 MHz CPUs), the table looks like the following:

Shoppers

CPU Utilization

Avg. Operation Latency (ms)

ASP Req. per sec.

Cost (Mcycles)

0

0.000%

0.000

0.000

0.000

100

14.706%

421.024

2.659

117.648

200

31.960%

439.151

5.391

255.680

300

57.939%

647.937

8.105

463.512

400

90.464%

15770.377

8.299

723.712

500

94.616%

25561.366

9.583

756.928

The table shows that the verified maximum capacity appears to be 300 concurrent shoppers. ASP request throughput is higher at a load level of 400 shoppers. However, at 400 shoppers, notice that operation latency (response time) jumps to 11.8 seconds, which is no longer an acceptable quality level. If latency is not a consideration, we see that shopper capacity is 500 shoppers, as ASP requests-per-second continue to climb at this load level, perhaps up to 550 shoppers.

Keep in mind that the verification script runs at four times the transaction and request rate of the live customer site. The maximum number of shoppers observed here, then, translates to approximately 4 x 300 = 1,200 shoppers on the customer's live site.

See the section on Verification Scripts for a collection of sample scripts you can use for verifying capacity at your site.

Transaction Cost Analysis Scripts

This section contains sample TCA scripts for use in measuring cost (Cost Measurement Scripts) or verifying capacity (Verification Scripts). Note that these scripts are to be used with InetMonitor.

Cost Measurement Scripts

The following sample scripts exercise load for TCA cost measurements:

  • Default Home Page 

  • Search (Good Results) 

  • Search (No Results) 

  • Add Item 

  • Add Item + Delete 

  • Add Item + Checkout 

  • Browse 

  • Login 

  • Register 

  • Zip Code 

  • View Cart 

Default Home Page

The following sample script goes to the site home page:

REM *** Goto Home Page ***
GET url:/st/home.asp
Search (Good Results)

The following sample script measures the cost of search operations, for which the user obtains good results:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
REM *** Search ***
LOOP 10
GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
ENDLOOP
Search (No Results)

The following sample script measures the cost of search operations, for which the user obtains no results:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
REM *** Search ***
LOOP 10
GET url:/st/search/search-results.asp?RANDLIST(TCASearchNotFoundData.txt)
ENDLOOP
Add Item

The following sample script measures the cost of adding items to a shopping basket:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
REM *** Add Item ***
LOOP 10
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
ENDLOOP
Add Item + Delete

The following sample script measures the cost of adding an item to a shopping basket, and then deleting it:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
LOOP 10
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
GET url:/st/xt_orderform_delitem.asp?index=0
ENDLOOP
Add Item + Checkout

The following sample script measures the cost of adding an item to a shopping basket, and then checking out (purchasing the item):

REM *** Goto Home page ***
GET url:/st/home.asp
REM *** Login as a Member ***
GET SEQULIST(TCAUsersLoginData.txt)
REM *** clear cart
POST url:/st/xt_orderform_clearitems.asp
LOOP 10
REM *** Add Item
GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
REM *** Check out ***
POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
POST
url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asds
ad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&
bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=41111111111
11111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&
y=18
POST
url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERI
FY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to
_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_st
ate=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expy
ear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111
111111&IMAGE1.x=55&IMAGE1.y=12
ENDLOOP
Browse

The first few lines of script prior to the browse loop set up the user environment. In the following example, a zip code must be provided before the shopper can obtain pricing information. The InetMonitor loop command executes the browse requests. Looping and performing the browse request multiple times statistically reduces the noise resulting from the zip code setup:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
LOOP 10
REM *** Browse Departments ***
%14 SKIP 1
POST
url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
%15 SKIP 1
POST
url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)
REM *** Browse SKUSet ***
%23 SKIP 1
GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)
REM *** Browse Product ***
%48 SKIP 1
GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
ENDLOOP
Login

The following sample script measures the cost of logging into the site:

REM *** Goto Home page ***
GET url:/st/home.asp
REM *** Login as a Member ***
LOOP 10
GET SEQULIST(TCAUSersLoginData.txt)
ENDLOOP
Register

The following script measures the cost of registering a new user:

REM +++ New Shopper +++
POST SEQULIST(TCARegisterUsersData.txt)
Zip Code

The following script measures the cost of obtaining a user's zip code:

REM *** Goto Home Page ***
GET url:/st/home.asp
LOOP 10
POST
url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
ENDLOOP
View Cart

The following script measures the cost of adding items, and then viewing the items in a shopping basket:

REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
REM *** Add Item ***
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM *** View Cart ***
LOOP 20
GET url:/st/cart.asp
ENDLOOP

Verification Scripts

The scripts in this section are sample scripts that can be used to exercise load for TCA verification measurements:

  • Baseline Verification 

  • Optimized Site Verification 

  • HTML Verification 

  • Checkout Verification 

Baseline Verification

The following sample script provides baseline verification for site operations:

REM *** VERIFICATION SCRIPT FOR TCA ***
REM *** 11 Operations in 11 minutes
REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
REM *** Start Timer ***
STARTTIMER MAINTIMER
REM *** Goto Home page 2.5 operations ***
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
%50 SKIP 2
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
REM *** Search Good Result 1.6 operations ***
GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)
%40 SKIP 2
GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)
REM *** Search Not Found 0.1 operations ***
%90 SKIP 2
GET url:/st/search/search-results.asp?RANDLIST(TCASearchNotFoundData.txt)
SLEEP RANDNUMBER(40000,80000)
REM *** Add Item 0.2 operations ***
%80 SKIP 2
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(40000,80000)
REM *** Add Item and Delete 0.1 operations ***
%90 SKIP 3
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(20000,40000)
GET url:/st/xt_orderform_delitem.asp?index=0
SLEEP RANDNUMBER(20000,40000)
REM *** Add Items and Checkout, Clear cart first 0.1 operations ***
%90 SKIP 12
GET SEQULIST(TCAUSersLoginData.txt)
SLEEP RANDNUMBER(8000,12000)
POST url:/st/xt_orderform_clearitems.asp
SLEEP RANDNUMBER(8000,12000)
GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
SLEEP RANDNUMBER(8000,12000)
POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
SLEEP RANDNUMBER(8000,12000)
POST
url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asds
ad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&
bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=41111111111
11111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&
y=18
SLEEP RANDNUMBER(8000,12000)
POST
url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERI
FY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to
_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_st
ate=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expy
ear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111
111111&IMAGE1.x=55&IMAGE1.y=12
SLEEP RANDNUMBER(8000,12000)
REM *** Browse Products 4 operations ***
REM *** Browse Departments show all 14% ***
%86 SKIP 1
POST
url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
POST
url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show all 14% ***
%86 SKIP 1
POST
url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
POST
url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show all 14% ***
%86 SKIP 1
POST
url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
POST
url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show all 14% ***
%86 SKIP 1
POST
url:/st/department.asp?show=all&PROPERTIES:RANDLIST(TCACategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
POST
url:/st/department.asp?show=one&PROPERTIES:RANDLIST(TCADepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET url:/st/skuset.asp?RANDLIST(TCASKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET url:/st/product.asp?skusetid=RANDLIST(TCAProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** End Of Browse Products 4 operations ***
REM *** Login as a Member 0.2 operations ***
%80 SKIP 2
GET SEQULIST(TCAUSersLoginData.txt)
SLEEP RANDNUMBER(40000,80000)
REM *** Register New Shopper 0.1 operations ***
%90 SKIP 2
POST SEQULIST(TCARegisterUsersData.txt)
SLEEP RANDNUMBER(40000,80000)
REM *** Zip Code 1.8 operations ***
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)
%20 SKIP 2
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)
REM *** View Cart 0.3 operations ***
%70 SKIP 4
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(20000,40000)
GET url:/st/cart.asp
SLEEP RANDNUMBER(20000,40000)
REM *** End Timer ***
ENDTIMER MAINTIMER
REM *** Kill ASP session ***
GET url:/st/quitsession.asp
Optimized Site Verification

The following sample script verifies site optimization:

REM *** VERIFICATION SCRIPT FOR TCA ***
REM *** 11 Operations in 11 minutes
REM *** Goto Home Page ***
GET url:/st/home.asp
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
REM *** Start Timer ***
STARTTIMER MAINTIMER
REM *** Goto Home page 2.5 operations ***
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
%50 SKIP 2
GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
REM *** Search Good Result 1.6 operations ***
GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)
%40 SKIP 2
GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)
REM *** Search Not Found 0.1 operations ***
%90 SKIP 2
GET url:/st/search/search-results.asp?RANDLIST(TCASearchNotFoundData.txt)
SLEEP RANDNUMBER(40000,80000)
REM *** Add Item 0.2 operations ***
%80 SKIP 2
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(40000,80000)
REM *** Add Item and Delete 0.1 operations ***
%90 SKIP 3
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(20000,40000)
GET url:/st/xt_orderform_delitem.asp?index=0
SLEEP RANDNUMBER(20000,40000)
REM *** Add Items and Checkout, Clear cart first 0.1 operations ***
%90 SKIP 12
GET SEQULIST(TCAUSersLoginData.txt)
SLEEP RANDNUMBER(8000,12000)
POST url:/st/xt_orderform_clearitems.asp
SLEEP RANDNUMBER(8000,12000)
GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
SLEEP RANDNUMBER(8000,12000)
POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
SLEEP RANDNUMBER(8000,12000)
POST
url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asds
ad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&
bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=41111111111
11111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&
y=18
SLEEP RANDNUMBER(8000,12000)
POST
url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERI
FY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to
_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_st
ate=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expy
ear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111
111111&IMAGE1.x=55&IMAGE1.y=12
SLEEP RANDNUMBER(8000,12000)
REM *** Browse Products 4 operations ***
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** End Of Browse Products 4 operations ***
REM *** Login as a Member 0.2 operations ***
%80 SKIP 2
GET SEQULIST(TCAUSersLoginData.txt)
SLEEP RANDNUMBER(40000,80000)
REM *** Register New Shopper 0.1 operations ***
%90 SKIP 2
POST SEQULIST(TCARegisterUsersData.txt)
SLEEP RANDNUMBER(40000,80000)
REM *** Zip Code 1.8 operations ***
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)
%20 SKIP 2
POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)
REM *** View Cart 0.3 operations ***
%70 SKIP 4
GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
SLEEP RANDNUMBER(20000,40000)
GET url:/st/cart.asp
SLEEP RANDNUMBER(20000,40000)
REM *** End Timer ***
ENDTIMER MAINTIMER
REM *** Kill ASP session ***
GET url:/st/quitsession.asp
HTML Verification

The following sample script verifies HTML code for the site:

REM *** VERIFICATION SCRIPT FOR TCA ***
REM *** 11 Operations in 11 minutes
REM *** Goto Home Page ***
REM GET url:/st/home.asp
REM POST url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
REM *** Start Timer ***
STARTTIMER MAINTIMER
REM *** Goto Home page 2.5 operations ***
REM GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
REM GET url:/st/home.asp
SLEEP RANDNUMBER(40000,80000)
REM %50 SKIP 2
REM GET url:/st/home.asp
%50 SKIP 1
SLEEP RANDNUMBER(40000,80000)
REM *** Search Good Result 1.6 operations ***
REM GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
SLEEP RANDNUMBER(40000,80000)
REM %40 SKIP 2
REM GET url:/st/search/search-results.asp?RANDLIST(TCASearchGoodData.txt)
%40 SKIP 1
SLEEP RANDNUMBER(40000,80000)
REM *** Search Not Found 0.1 operations ***
REM %90 SKIP 2
REM GET url:/st/search/search-results.asp?RANDLIST(TCASearchNotFoundData.txt)
%90 SKIP 1
SLEEP RANDNUMBER(40000,80000)
REM *** Add Item 0.2 operations ***
REM %80 SKIP 2
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
%80 SKIP 1
SLEEP RANDNUMBER(40000,80000)
REM *** Add Item and Delete 0.1 operations ***
REM %90 SKIP 3
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM SLEEP RANDNUMBER(20000,40000)
REM GET url:/st/xt_orderform_delitem.asp?index=0
REM SLEEP RANDNUMBER(20000,40000)
%90 SKIP 2
SLEEP RANDNUMBER(20000,40000)
SLEEP RANDNUMBER(20000,40000)
REM *** Add Items and Checkout, Clear cart first 0.1 operations ***
REM %90 SKIP 12
REM GET SEQULIST(TCAUSersLoginData.txt)
REM SLEEP RANDNUMBER(8000,12000)
REM POST url:/st/xt_orderform_clearitems.asp
REM SLEEP RANDNUMBER(8000,12000)
REM GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
REM SLEEP RANDNUMBER(8000,12000)
REM POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
REM SLEEP RANDNUMBER(8000,12000)
REM POST
url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asds
ad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&
bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=41111111111
11111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&
y=18
REM SLEEP RANDNUMBER(8000,12000)
REM POST
url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERI
FY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to
_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_st
ate=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expy
ear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111
111111&IMAGE1.x=55&IMAGE1.y=12
REM SLEEP RANDNUMBER(8000,12000)
%90 SKIP 1
SLEEP RANDNUMBER(40000,80000)
REM *** Browse Products 4 operations ***
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show all 14% ***
%86 SKIP 1
GET RANDLIST(TCAEarlyRenderCategoryIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Departments show one 15% ***
%85 SKIP 1
GET RANDLIST(TCAEarlyRenderDepartmentIDData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse SKUSet 23% ***
%77 SKIP 1
GET RANDLIST(TCAEarlyRenderSKUSETData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** Browse Product 48% ***
%52 SKIP 1
GET RANDLIST(TCAEarlyRenderProductBrowseData.txt)
SLEEP RANDNUMBER(13000,17000)
REM *** End Of Browse Products 4 operations ***
REM *** Login as a Member 0.2 operations ***
REM %80 SKIP 2
REM GET SEQULIST(TCAUSersLoginData.txt)
%80 SKIP 1
SLEEP RANDNUMBER(40000,80000)
REM *** Register New Shopper 0.1 operations ***
REM %90 SKIP 2
REM POST SEQULIST(TCARegisterUsersData.txt)
%90 SKIP 1
SLEEP RANDNUMBER(40000,80000)
REM *** Zip Code 1.8 operations ***
REM POST
url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
SLEEP RANDNUMBER(40000,80000)
REM %20 SKIP 2
REM POST
url:/st/zip.asp?setZip=1&url=%2Fst%2Fhome%2Easp&PROPERTIES:zip=98052
%20 SKIP 1
SLEEP RANDNUMBER(40000,80000)
REM *** View Cart 0.3 operations ***
REM %70 SKIP 4
REM GET url:/st/xt_orderform_additem.asp?sku=RANDLIST(TCAAddItemSKU.txt)&qty=1
REM SLEEP RANDNUMBER(20000,40000)
REM GET url:/st/cart.asp
REM SLEEP RANDNUMBER(20000,40000)
%70 SKIP 4
SLEEP RANDNUMBER(40000,80000)
REM *** End Timer ***
ENDTIMER MAINTIMER
REM *** Kill ASP session ***
REM GET url:/st/quitsession.asp
Checkout Verification

The following sample script verifies the checkout procedure:

REM *** Goto Home page ***
GET url:/st/home.asp
REM *** Login as a Member ***
GET SEQULIST(TCAUsersLoginData.txt)
REM *** clear cart
POST url:/st/xt_orderform_clearitems.asp
LOOP 10
REM *** Add Item
GET url:/st/xt_orderform_additem.asp?sku=380157&qty=1
SLEEP RANDNUMBER(40000,80000)
REM *** Check out ***
POST url:/st/member/orderform.asp?PROPERTIES:x=72&y=15
SLEEP RANDNUMBER(40000,80000)
POST
url:/st/member/confirm_order.asp?PROPERTIES:cc_name=asdsad+asdsad&bill_to_name=asdsad+asds
ad&bill_to_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&
bill_to_state=ma&bill_to_zip=01605&bill_to_phone=asdasd&cc_type=visa&cc_number=41111111111
11111&cc_expmonth=5&cc_expyear=1999&po_number=&coupon_one=&coupon_two=&coupon_three=&x=74&
y=18
SLEEP RANDNUMBER(40000,80000)
POST
url:/st/member/xt_orderform_purchase.asp?PROPERTIES:_VERIFY_WITH=_total_total%3D8999&_VERI
FY_WITH=bill_to_zip%3D01605&_VERIFY_WITH=_tax_total%3D0&bill_to_name=asdsad+asdsad&bill_to
_company=asdsad&bill_to_street=asdasd&bill_to_street2=asdas&bill_to_city=asdsad&bill_to_st
ate=ma&bill_to_zip=01605&bill_to_phone=asdasd&po_number=&coupon_two=&coupon_three=&cc_expy
ear=1999&coupon_one=&cc_name=asdsad+asdsad&cc_expmonth=5&cc_type=visa&cc_number=4111111111
111111&IMAGE1.x=55&IMAGE1.y=12
SLEEP RANDNUMBER(40000,80000)
ENDLOOP