Configure a BCPT suite
Configure a suite for the scenario that you want to simulate. When you configure a suite, you need to specify:
What tests to run and, if applicable, with what parameters.
How many concurrent sessions you want to simulate for each test.
The duration of the scenario.
Typically, you want to run the suite for multiple sessions at the same time. After you configure the suite, you can do that by using the Start action. You can also use the Visual Studio Code extension or PowerShell scripts. Learn more at Run a test suite using the Visual Studio Code extension.
To do a quick test, for example, during the development phase, choose the Start in Single Run mode action to run all your tests (suite lines) one time. It also skips the user delays. Single Run mode lets you monitor the number of SQL statements between runs and define baselines, and gives you quick feedback that can help identify regressions early on.
You can run up to 500 sessions at the same time for a test suite. The Total No. of Sessions field shows how many sessions are created when you run the suite.
On the suite lines, the Run in Foreground checkbox lets you run tests in the foreground rather than as a background task. You can specify this for each individual suite line. However, if your test involves UI interactions, then it must run in the foreground. From the client, it's only possible to have one foreground test running at a time. If multiple lines have the run in foreground option enabled, they run sequentially. Background tests still run in parallel.
If you must run multiple foreground tests in parallel, you must use the Visual Studio Code extension or the PowerShell scripts.
The settings to configure a suite depend on the environment that you want to simulate. The following procedure provides an example for testing multiple sessions, but the steps also apply to a single run.
Search for BCPT Suites, and choose the related link.
Choose New to open the BCPT Suite page.
In the Code, Description, and Tag fields, provide an identifier, some information about the test, and a tag that you can use to find the results of the suite on the Log Entries page. Tags are useful if you want to generate different sets of log entries from a suite. An example could be to measure performance before and after installing extensions. Use the tag to differentiate between those two runs.
To define the timing for the run, follow these steps.
The 1 Work Day Corresponds to field works with the Duration (minutes) field to update the Work Date Starts at field. Use the 1 Work Day Corresponds to field to test processes that have deadlines, such as payments. The duration can be up to 240 minutes.
The Default Min. User Delay and Default Max. User Delay fields let you simulate pauses between actions. You must specify a delay. Between each iteration, for example when creating sales orders, you can define a Delay (Delay between iterations) before the test starts on the next sales order. The delay can be Fixed or Random. Delays aren't included in the results for run times.
Configure lines for the suite. The lines contain some of the settings from the header. Updating the values on the lines also updates the header.
If your test supports parameters, in the Parameters field, enter the parameters for your test. The Parameter field must not contain spaces.
In the No. of Sessions field, enter the number of concurrent users to simulate in the workflow.
In the Min. User Delay (ms) and Max. User Delay (ms) fields, specify how long to pause between actions. When you run a suite in Single Run mode, user delays are ignored.
In the Delay between interactions (sec) field, specify how long to pause between each run of the codeunit for each session. Use this field to simulate how much work each user do. If one user creates 15 sales orders an hour, then you would need ~240 seconds (4 minutes) of delay between each test to achieve that throughput.
In the Delay Type field, specify whether to use a fixed or random delay. If you choose Fixed, the test pauses for the number of seconds specified in the Delay between iterations (sec) field.
There are three ways to run BCPT suites.
Directly from Business Central.
By using the Visual Studio Code extension or PowerShell scripts.
By using AL-Go for GitHub. For more information, see Workflow: Create new Performance Test App
When running BCPT suites, it's possible to pick which company to run your suites in. This can be an effective strategy when running performance tests against different data sizes.
There are a couple of limitations for running BCPT suites.
If you run it from the client, you can only run one session at a time in the foreground.
Depending on whether your environment is on-premises, Docker, or online, there might be a limit to the number of sessions you can run at the same time in the background. Typically, the default number of sessions is 10. Learn more at Operational Limits for Business Central Online.
If your BCPT suite runs more than one session in the foreground, or more than 10 sessions at the same time in the background, you must run the suite from the Visual Studio Code extension or PowerShell. Learn more at Run using the PowerShell scripts.
The following steps provide an example of how to run a test in Single Run mode:
On the BCPT Suites page, choose New.
In the Code field, enter a name for the test suite. In this example, we use PreTest.
In the Tag field, enter something that makes it easy to identify the suite later when you analyze the results.
The fields for duration and delays aren't used for Single Run mode, so we leave their default values.
On the BCPT Suit Lines FastTab, choose the test suite for your component.
In this example, we're calculating the total weight of the items on a sales order, so we use the Sales Order page test with a parameter that creates four lines. The parameter looks as follows: Lines=4.
Clear the Run in Foreground checkbox. This runs the test suite in a background thread.
Choose Start in Single Run mode.
To set your baseline after the run completes, in the Base Version field, enter 1.
Running suites from the Visual Studio Code extension gives you realistic simulations of multiple people working at the same time. To offer a seamless experience, we provide a command in the Visual Studio Code extension for starting:
In Visual Studio Code, run the BCPT: Run BCPT (PowerShell) command.
Select the environment type.
Select the specific environment to target.
Enter the BCPT suite code to run.
Optional: If you're targeting an online sandbox environment, you might be prompted to sign in.
An internal PowerShell process should start and an output window appears in Visual Studio Code that shows output similar to this:
Analyze the results
Results show on the BCPT Suite Lines FastTab. The following describe the fields that show results of the current run, the baseline, and the delta between the two:
No. of Iterations - How many times it ran. This is 1 because we used the Start in Single Run Mode action.
Total Duration in (ms) - How long it took to complete one iteration.
Average duration - Equal to the Total Duration in (ms) field for the first run, but after you run a test multiple times it's a result of the sum of runs since your baseline.
No. of SQL statements - The number of SQL statements generated for one run.
Avg. No. of SQL Statements - Similar to the duration fields, for the first run this is the same as the No. of SQL statements field, but becomes averaged after subsequent runs.
The following list describes the fields related to the baseline:
No. of Iterations Base - Not relevant for Single Run mode.
Total Duration Base (ms) - The total duration of the baseline run.
Avg. Duration Base (ms) - Not relevant for Single Run mode.
Avg. No. of SQL Statements Base - The number of SQL statements in your base run.
The following list describes the fields that show the delta between the run and the baseline.
Change in No. of SQL statements (%) - The percent of difference between the baseline and the latest run.
Changes in Duration (%) - The change in measured time between a baseline and the latest run.
The first iteration of any run shows a higher number of SQL Statements because nothing is cached.
Before you run a test with the change sign in, change the Tag field to Change log. When you explore the log entries afterwards, either remove the filter on the Version No. field, or change it to only include the last two runs. After exporting to Excel, select the tag as columns in the pivot table, use Operation as a filter, and set the filter to Scenario.
After running the suite, you can choose Log Entries to see what happened. Use the Show Errors and Show sessions running at the same time as this actions to apply filters to the results. For example, filtering can help troubleshoot errors by showing what a user was doing when an error occurred. By default, the page is filtered to show the latest version, but you can change or remove the filter if you want to compare runs. You can use the Open in Excel action to build dashboards that can help visualize performance results.
Log entries are listed in the order that they were created, which means that the different scenarios are mixed. Each run is identified by the value in the Version No. fields.
The Operation column shows the individual measurements, where the term Scenario is used for running the codeunit without the user wait time. The No. of SQL Statements column includes the SQL statements that were issued by the scenario and system activities, such as retrieving metadata. The counts exclude the log entries themselves. To drill down to a single session, filter on the Session No. field or choose Open in Excel to create a pivot table and pivot chart for deeper analysis.
If you enable telemetry on your test environment, then you get telemetry when the Performance Toolkit runs start/complete and when scenarios complete (including measurements). Learn more at Performance toolkit telemetry.
The telemetry data emitted from the Performance Toolkit can then be analyzed using the Power BI Performance report or directly with KQL. See sample KQL code here: Sample KQL code.