How To: Do image comparison in Coded UI Test

  • Article

This is guest blog by my colleague Rajeev Kumar.

The Coded UI Test does not provide the API to compare images. There are cases when user wants to verify the changes in the UI by comparing the images of two controls. This can be easily achieved using Test API with Coded UI Test. The overview of Test API is described here and Test API visual verification is described here.

Getting the Test API

  • Download the Test API from here.
  • Add reference of TestAPICore.dll in the Coded UI Test project.
  • Add this using statement in your code-  using Microsoft.Test.VisualVerification;

General Test API terms

  • Snapshot: A pixel buffer used for representing and evaluating screen image data.
  • Verifier: An object which determines whether a snapshot passes against specified inputs.
  • Actual: The snapshot being evaluated.
  • Master: The reference data (image) which is used to evaluate the actual snapshot.
  • Tolerance: The accepted bounding range based on which the actual snapshot will be accepted as valid.

The general Visual Verification workflow

  • Search the UI Test Control.
  • Capture the image of UI Test Control.
  • Generate an expected snapshot (e.g. load a master image from disk, etc.)
  • Compare the actual snapshot to the expected snapshot and generate the difference (diff) snapshot.
  • Verify the diff using a verifier.
  • Report test result.

Sample code to use Test API.

  • From the Test API copy these files to Coded UI Test:
    • Samples\MSTest\Tests\bin\Debug\Master.png
    • Samples\MSTest\Tests\bin\Debug\ToleranceMap.png
    • Samples\MSTest\Tests\bin\Debug\SampleApp.exe
  • Add these files in the Coded UI Test project and make “Copy to Output Directory = Copy Always” in the property window for each files.
  • Now add this test method the Coded UI Test.
 [TestMethod]
[DeploymentItem("Master.png")]
[DeploymentItem("ToleranceMap.png")]
[DeploymentItem("SampleApp.exe")]
public void CompareImagesTest()
{
    // Launch the application.
    using (ApplicationUnderTest app = ApplicationUnderTest.Launch(
                Path.Combine(Directory.GetCurrentDirectory(),
                "SampleApp.exe")))
    {
        // Select the check box, so that the background will change.
        WpfCheckBox checkBox = new WpfCheckBox(app);
        checkBox.Checked = true;

        // Get the actual, master and toleracne snapshots.
        Snapshot toleranceMap =
                Snapshot.FromFile("ToleranceMap.png");
        Snapshot master = Snapshot.FromFile("Master.png");
        Snapshot actual = Snapshot.FromWindow(app.WindowHandle,
                WindowSnapshotMode.ExcludeWindowBorder);

        // Get the difference of two images.
        Snapshot difference = actual.CompareTo(master);

        // Save the file to for manual viewing.
        master.ToFile(@"Master-expected.png", ImageFormat.Png);
        actual.ToFile(@"Master-actual.png", ImageFormat.Png);
        difference.ToFile(@"Master-difference.png",
                            ImageFormat.Png);

        // Use SnapshotToleranceMapVerifier to verify the comparison.
        SnapshotVerifier verifier =
                new SnapshotToleranceMapVerifier(toleranceMap);
        Assert.AreEqual(VerificationResult.Pass,
                verifier.Verify(difference));
    }
}

Description of the code

  • Launch the sample application.
  • Check the check box on the sample allocation so that the app changes its background color.
  • Create the Snapshot object for master, actual and tolerance map.
  • Compares the actual image with the stored copy of master image.
  • Saves images files for manual diagnosis (in the test fails, then these images are useful)
  • Then verifies the differences of two images are within tolerance as given by tolerance map image.


This was guest blog by my colleague Rajeev Kumar.