Helpful Internals of TRX (and VSMDI files)

Article
06/13/2008

One of the questions that comes up internally, and externally is "What are these magic GUID attributes in the file? How can I generate them?". This is often from people who are creating tools to export TRX files from some sort of private test hardness, or generate VSMDI files in an automated fashion.

Note, that the XSD schemas are available with all visual studio installs in the:

 %VSINSTALLDIR%\xml\Schemas\vstst.xsd file

directory, along with many other schemas.

TRX Files

Lets take a look at TRX files first. I'm not going to go through the whole file line by line -- we put a lot of info into the file, and some of it can be easily guessed. Things in the TRX file other than the obvious "results":

Runconfig - we just dump in the whole run config that was use as part of that run
Test List info - The test list info etc that was associated with the tests as they were run. If you just run the test from test view, or use the keyboard shortcuts (Ctrl-R, T), you won't see the test list info.

Neither of these pieces of information are required per-se, we just dump them in there for reference reasons (mostly for re-running without having all the original information).

Interesting lines:

The GUID here is just a randomly generated GUID -- Guid.New().ToString() is good enough here, it's just used to allows us to identify the run compared to any other arbitrary run.

This GUID is ID of the Test Run Config that was copied from the one specified at the time the run was created. This should match, but there's no specific need -- we don't go and look up that GUID later.

This is where it gets interesting. This element, which is found under the TestDefinitions element enumerates/declares all the tests that are part of this run. Tests have a name, which is used to provide a friendly way for users to identify test cases from each other. However, the tests themselves need to be uniquely identified within the test system. Most test types (all but unit, for the ones that are shipped in-box) are identified using a random GUID, which is stored in the file the test is stored in. Example: Open up a word based manual test, and look on the properties of it -- you'll see an item which has a GUID in it, and is called "ID". The ID here on these child elements (children of TestDefinitions) is that same GUID as found the test. However, this is not the case for unit tests. Unit tests are generated using a hash fully qualified class name, and the method name [1] (i.e. Company.Product.TestClass.TestMethod).

This is another "random" GUID, however it is referred to later in the file, specifically as part of the results and tracking which tests were executed, so make sure that if you're generating here, your not just throwing the GUID out of the window after you generate it.

This is similar to the test run configuration section, where it's a dupe of the actual test list file. This contains all the test lists that are in this run, with the name (as we can see), and the GUID -- again, this really is just a random GUID. But as per <Execution />, it's used later in the file.

Test Entry provides a match between test lists, the test case, and the test result. Note that these GUIDs here refer to the IDs previously seen throughout this post. Make sure these GUIDs match up, and your results will all be peachy.

These are the results (whose elements are determined by the test type that was executed, hence the 'UnitTestResult'), and as one can see, there are some guids here, which tally back to elements & ID's that were declared earlier in the file -- specifically the execution ID, and the test ID (since this is a unit test, the computed ID).

VSMDI Files

The VSMDI file format is much simpler, although equally filled with GUIDs :). The basic format here is the runconfig (don't ask why it's in there, it's got long historical reasons, and does allow for an... interesting scenario [2]) attached to the of the test lists (which is the same GUID no matter what), and then all the other test lists in the file, with their own GUID, a guid linking to a parent test list, and the contents of that list itself. Lets look at an example.

This is the "Root" test list, the one you see at the root of test list editor window - an immovable, unchangeable test list. This GUID is constant and the same in all VSMDI files, and all machines. It's been the same GUID since we shipped the first beta of VS 2005 (amusing to think that was over 4 years ago). If you don't have this, then things are going to go a bit pete tong.

Other test lists take the same form, but with there own GUID, which is merely a randomly generated GUID:

Here, we can see that this node is a child of the parent node (Lists of Tests) by noting the parentListId GUID is the same as the ID guid of the "Lists of Tests" test list. This allows you to establish many levels of nesting of test lists to create the right grouping and categorization you need. However, for tests to appear in these test lists, you need to add child elements of this TestList element:

Here, the ID refers to the test id of the test being linked in. If you look back through this article you can see the ID of this test showing in the TRX file section -- these IDs really do identify the test across multiple stores (even into the TFS warehouse if you push your results there).

Conclusion

There you have it, a whirl wind tour of the TRX and VSMDI file formats -- which should provide some insight for understanding the rest of our file formats. Note that while some of the information here is still relevant to VS 2005, the file format in that version is radically different, and requires some what unnatural acts to be able to generate it from outside of VS -- I would recommend upgrading to 2008 if you wish to do that :)

[1] We hash the fully qualified test method with code like this:

 using System;
using System.Diagnostics;
using System.Security.Cryptography; 

namespace TeamTestSample
{
    internal static class UnitTestIdHash
    {
        private static HashAlgorithm s_provider = new SHA1CryptoServiceProvider(); 

        internal static HashAlgorithm Provider
        {
            get { return s_provider; }
        } 

        /// 
        /// Calculates a hash of the string and copies the first 128 bits of the hash
        /// to a new Guid.
        /// 
        internal static Guid GuidFromString(string data)
        {
            Debug.Assert(!String.IsNullOrEmpty(data);
            byte[] hash = Provider.ComputeHash(System.Text.Encoding.Unicode.GetBytes(data)); 

            // Guid is always 16 bytes
            Debug.Assert(Guid.Empty.ToByteArray().Length == 16, "Expected Guid to be 16 bytes"); 

            byte[] toGuid = new byte[16];
            Array.Copy(hash, toGuid, 16); 

            return new Guid(toGuid);
        }
    }
}

[2] One little know feature of the testing tools inside visual studio, is that you can re-run a test run without sources, just by picking up the test results directory, and opening the TRX file, and clicking the run button. What is even less known is that if you have the Test storages (the dlls, webtest, etc), and the VSMDI, you can open the VSMDI directly (no project, or solution), and select the tests to run. The test run configuration that will be applied will the last test run configuration that was active when the vsmdi was last edited.

Helpful Internals of TRX (and VSMDI files)

Additional resources