Integration of Logging with DTM Jobs

Article
01/05/2010

Test software that a DTM job's tasks run might use DTM Logger as the logging service to record information and results of running the test software. Two log files result from the creation of a DTM Logger instance:

The test software's log file. For an example file, see TraceAll.wtl.
A pass-fail rollup log file. For an example file, see TaskGUID.xml.

If the task's "Task Failure Conditions" execution option is set to "Declare task failure based on results from log," DTM will determine the pass or fail status for the task based on the contents of the pass-fail rollup log file that DTM Logger creates.

Pass-Fail Rollup Files

The pass-fail rollup file is created when the test software closes the DTM Logger object (Logger.CloseLogDevice(NULL, hDevice)). The rollup file is stored in a file named <Taskguid>.xml and is copied to the DTM controller's log file share.

DTM performs two operations on the information that is provided in the pass-fail rollup file:

Maps "Blocked" to "Not Run" and "Skipped" to "Not Applicable".
Sums the Pass, Failed, Not Run, and Not Applicable attributes of the TaskResult and AdditionalResult elements.

Note DTM does not use the "Warned" attribute.

Out-of-Context Errors

Out-of-context errors can occur during testing, depending on the way in which test software has been implemented. To help resolve problems with out-of-context errors, DTM:

Isolates test execution for a specific device handle from the log messages that are delivered for other device handles. This isolation prevents, for example, the system processes and modules that use DTM Logger, such as WTTCmd.exe or the WTTSDK API, from changing the test results of the user test application.
Indicates which messages were recorded as additional results because the computer restarted and which messages were recorded as out-of-context. Restarting, which the test software might initiate during a test, causes re-execution of a process. Re-execution is identified in the rollup file by the presence of AdditionalResult tags. Out-of-context errors within an additional result are identified by an OutOfContext attribute, which can be set to "true" or "false". An AdditionalResult tag with an attribute value of OutOfContext="true" indicates the presence of an out-of-context error.
Each additional result includes a LogFileList item that shows the names of the log files to which the result was written.

The combination of out-of-context errors and restarting leads to different entries in the pass-fail rollup file, as shown in the following examples.

Example 1: No Out-of-Context Errors, No Restart

In the case where the task runs once (without restarting) and the test software does not record out-of-context errors, the pass-fail rollup file will be similar to the following example.

The attributes of the TaskResult element contain the values that passed to the EndTest method.

Example 2: Out-of-Context Errors Occur, No Restart

When a task runs once, without restarting, and the test software records out-of-context errors, the resulting pass-fail rollup file will include an AdditionalResult element with a non-zero value for Failed, similar to the following example.

The AdditionalResult element represents each of the calls to the Trace method with an error of some kind.

Note This feature is an added feature for WTT 2.1 and later. WTT 2.0 did not provide the ability to log out-of-context errors.

Example 1: Out-of-Context Errors Occur, System Restarts

If the test software calls for a restart and the task is configured to re-run after the restart, other AdditionalResult elements will be added to the pass-fail rollup file, one for the out-of-context errors and one that calls the EndTest method. The log file will look similar to the following example.

Note This feature is an added feature for WTT 2.1 and later. WTT 2.0 reported only the results for the last iteration of a Task that performed a restart.

Share via

Integration of Logging with DTM Jobs

Pass-Fail Rollup Files

Out-of-Context Errors

See Also

Additional resources