Testing and deploying Windows Store business apps using C#, XAML, and Prism

  • Article

[This article is for Windows 8.x and Windows Phone 8.x developers writing Windows Runtime apps. If you’re developing for Windows 10, see the latest documentation]

From: Developing a Windows Store business app using C#, XAML, and Prism for the Windows Runtime

Previous page | Next page

Learn how to perform various modes of testing in order to ensure that reliable, high quality apps result. We provide guidance for unit testing, integration testing, user interface testing, suspend and resume testing, security testing, localization testing, accessibility testing, performance testing, device testing, and validation of the app user experience against the user experience guidelines on the Windows Developer Center.

Download

After you download the code, see Getting started using Prism for the Windows Runtime for instructions on how to compile and run the reference implementation, as well as understand the Microsoft Visual Studio solution structure.

You will learn

  • How the various modes of testing contribute to the reliability and correctness of an app.
  • How to test synchronous and asynchronous functionality in automated tests.
  • How to perform different types of testing, including suspend and resume testing, localization testing, and accessibility testing.

Applies to

  • Windows Runtime for Windows 8.1
  • C#
  • Extensible Application Markup Language (XAML)

Making key decisions

Testing helps to ensure that an app is reliable, correct, and of high quality. The following list summarizes the decisions to make when testing a Windows Store app:

  • How should I test the app?
  • How should I deploy the app?
  • How can I test the app for compliance with the Windows Store certification requirements?
  • How should I manage the app after deployment?

You can test your app in many ways including unit testing, integration testing, user interface testing, suspend and resume testing, security testing, localization testing, accessibility testing, performance testing, device testing, and validation of the app user experience against the user experience guidelines on the Windows Dev Center. For more info see Testing AdventureWorks Shopper.

While you can use the Windows Store to market and distribute apps, business apps will often be distributed directly to the end-user by the IT organization within a company. For more info see Deploying and managing Windows Store apps.

Regardless of how your app will be deployed, you should validate and test it by using the Windows App Certification Kit. The kit performs a number of tests to verify that your app meets certification requirements for the Windows Store. In addition, as you plan your app, you should create a publishing-requirements checklist to use when you test your app. For more info see Testing your app with the Windows App Certification Kit and Creating a Windows Store certification checklist.

Tools such as Windows Intune and System Center Configuration Manager can be used to manage access to business apps. In addition, IT staff can control the availability and functionality of the Windows Store to client computers based on the business policies of their environment. For more info see Deploying and managing Windows Store apps.

[Top]

Testing AdventureWorks Shopper

The AdventureWorks Shopper reference implementation was designed for testability, with the following modes of testing being performed:

  • Unit testing tests individual methods in isolation. The goal of unit testing is to check that each unit of functionality performs as expected so that errors don't propagate throughout the app. Detecting a bug where it occurs is more efficient than observing the effect of a bug indirectly at a secondary point of failure. For more info see Unit and integration testing.
  • Integration testing verifies that the components of an app work together correctly. Integration tests examine app functionality in a manner that simulates the way the app is intended to be used. Normally, an integration test will drive the layer just below the user interface. In the AdventureWorks Shopper reference implementation, you can recognize this kind of test because it invokes methods of the view model. The separation of views from the view model makes integration testing possible. For more info see Unit and integration testing.
  • User interface (UI) testing involves direct interaction with the user interface. This type of testing often needs to performed manually. Automated integration tests can be substituted for some UI testing but can't eliminate it completely.
  • Suspend and resume testing ensures that your app behaves as expected when Windows suspends or resumes it, or activates it after a suspend and shutdown sequence. For more info see Suspend and resume testing.
  • Security testing focus on potential security issues. It's based on a threat model that identifies possible classes of attack. For more info see Security testing.
  • Localization testing makes sure that an app works in all language environments. For more info see Localization testing.
  • Accessibility testing makes sure than an app supports touch, pointer, and keyboard navigation. It also makes sure that different screen configurations and contrasts are supported, and that the contents of the screen can be read with Windows Narrator. For more info see Accessibility testing.
  • Performance testing identifies how an app spends its time when it's running. In many cases, performance testing can locate bottlenecks or methods that take a large percentage of an app's CPU time. For more info see Performance testing.
  • Device testing ensures than app works properly on the range of hardware that it supports. For example, it's important to test that an app works with various screen resolutions and touch-input capabilities. For more info see Device testing.

For more info on test automation, see Testing for Continuous Delivery with Microsoft Visual Studio 2012.

[Top]

Unit and integration testing

You should expect to spend about the same amount of time writing unit and integration tests as you do writing the app's code. The effort is worth the work because it results in much more stable code that has fewer bugs and requires less revision.

In the AdventureWorks Shopper reference implementation we used the Model-View-ViewModel (MVVM) pattern to separate the concerns of presentation, presentation logic, and model. The MVVM pattern makes it easier to maintain and test your Windows Store app, especially as it grows. For more info see Using the MVVM pattern.

The AdventureWorks.UILogic.Tests, AdventureWorks.WebServices.Tests, Microsoft.Practices.Prism.PubSubEvents.Tests, and Microsoft.Practices.Prism.StoreApps.Tests projects of the AdventureWorks Shopper Visual Studio solution contain all the code that supports testing the Microsoft.Practices.Prism.PubSubEvents and Microsoft.Practices.Prism.StoreApps libraries, and the AdventureWorks Shopper reference implementation. The AdventureWorks.WebServices.Tests project uses the Microsoft.VisualStudio.QualityTools.UnitTestFramework, with the remaining test projects using the MsTestFramework for Windows Store apps. Test methods can be identified by the TestMethod attribute above the method name.

You can examine the unit tests by opening the AdventureWorks Shopper Visual Studio solution. On the menu bar, choose Test > Windows > Test Explorer. The Test Explorer window lists all of the AdventureWorks Shopper unit tests, as shown in the following diagram.

Unit tests should focus on how the code under test functions in response to values returned by dependent objects. A good approach to increase software testability is to isolate dependent objects and have them passed into your business logic using an abstraction such as an interface. This approach allows the dependent object to be passed into the business logic at run time. In addition, in the interests of testability, it allows a mock version of the dependent object to be passed in at test time. By using mocks, the return values or exceptions to be thrown by mock instances of dependent objects can easily be controlled.

Testing synchronous functionality

Synchronous functionality can easily be tested by unit tests. The following code example shows the Validation_Of_Field_When_Valid_Should_Succeed test method that demonstrates testing synchronous functionality. The unit test verifies that the BindableValidator class can successfully validate the value of the Title property in the MockModelWithValidation class.

Microsoft.Practices.Prism.StoreApps.Tests\BindableValidatorFixture.cs

[TestMethod]
public void Validation_Of_Field_When_Valid_Should_Succeeed()
{
    var model = new MockModelWithValidation() { Title = "A valid Title" };
    var target = new BindableValidator(model);

    bool isValid = target.ValidateProperty("Title");

    Assert.IsTrue(isValid);
    Assert.IsTrue(target.GetAllErrors().Values.Count == 0);
}

This method creates instances of the MockModelWithValidation and the BindableValidator classes. The BindableValidator instance is used to validate the contents of the Title property in the MockModelWithValidation instance by calling the ValidateProperty method on the BindableValidator instance. The unit test passes if the ValidateProperty method returns true, and the BindableValidator instance has no errors.

For more info about validation, see Validating user input.

Testing asynchronous functionality

Asynchronous functionality can be tested by creating a mock version of the dependent service that has an asynchronous method, and specifying an asynchronous delegate in the unit test that will be executed by the asynchronous method in the mock object. The following code example shows the OnNavigatedTo_Fill_Root_Categories test method, which demonstrates testing asynchronous functionality. The unit test verifies that when the hub page is navigated to the RootCategories property of the HubPageViewModel class will contain three categories.

AdventureWorks.UILogic.Tests\ViewModels\HubPageViewModelFixture.cs

[TestMethod]
public void OnNavigatedTo_Fill_RootCategories()
{
    var repository = new MockProductCatalogRepository();
    var navigationService = new MockNavigationService();

    repository.GetRootCategoriesAsyncDelegate = (maxAmmountOfProducts) =>
    {
        var categories = new ReadOnlyCollection<Category>(new List<Category>{
            new Category(),
            new Category(),
            new Category()
        });

        return Task.FromResult(categories);
    };

    var viewModel = new HubPageViewModel(repository, navigationService, null, null);
    viewModel.OnNavigatedTo(null, NavigationMode.New, null);

    Assert.IsNotNull(viewModel.RootCategories);
    Assert.AreEqual(((ICollection<CategoryViewModel>)viewModel.RootCategories).Count, 3);
}

The method creates instances of the mock classes that are required to create an instance of the HubPageViewModel class. The GetRootCategoriesAsyncDelegate, when executed, returns a Task of type ReadOnlyCollection with three Category objects. An instance of the HubPageViewModel class is then created, with its OnNavigatedTo method being called. The OnNavigatedTo method calls the GetRootCategoriesAsync method, in this case on the MockProductCatalogRepository instance, which in turn executes the GetRootCategoriesAsyncDelegate. The result of this is that the RootCategories property of the HubPageViewModel instance is populated with the data returned by the GetRootCategoriesAsyncDelegate. The unit test passes if the RootCategories property contains three items of data.

Note  If you use the await operator in a test method, the test method must return a Task and use the async modifier in its method signature.

 

For more info about the unit testing tools in Visual Studio, see Verifying Code by Using Unit Tests.

[Top]

Suspend and resume testing

When you debug a Windows Store app, the Debug Location toolbar contains a drop-down menu that enables you to suspend, resume, or suspend and shut down (terminate) the running app. You can use this feature to ensure that your app behaves as expected when Windows suspends or resumes it, or activates it after a suspend and shutdown sequence. The following diagram shows the drop-down menu that enables you to suspend the running app.

If you want to demonstrate suspending from the debugger, run AdventureWorks Shopper in the Visual Studio debugger and set breakpoints in the MvvmAppBase.OnSuspending and MvvmAppBase.InitializeFrameAsync methods. Then select Suspend and shutdown from the Debug Location toolbar. The app will exit. Restart the app in the debugger, and the app will follow the code path for resuming from the Terminated state. In AdventureWorks Shopper, this logic is in the MvvmAppBase.InitializeFrameAsync method. For more info see Guidelines for app suspend and resume and Handling suspend, resume, and activation.

[Top]

Security testing

We used the STRIDE methodology for threat modeling as a basis for security testing in AdventureWorks Shopper. For more info see Uncover Security Design Flaws Using The STRIDE Approach and Windows security features test.

[Top]

Localization testing

We used the Multilingual App Toolkit to provide pseudo-localized versions of AdventureWorks Shopper for localization testing. For more info see How to use the Multilingual App Toolkit, Guidelines for app resources, and Guidelines for globalization.

[Top]

Accessibility testing

We used a number of testing tools to verify the accessibility of AdventureWorks Shopper, including Windows Narrator, Inspect, UI Accessibility Checker, UI Automation Verify, and Accessible Event Watcher. For more info see Testing your app for accessibility and Design for accessibility.

[Top]

Performance testing

In addition to using profiling tools to measure app performance, we also used the Windows Performance Toolkit (WPT). WPT can be used to examine app performance, both in real time and by collecting log data for later analysis. We used this tool for a general diagnosis of the app's performance. For more info see Windows Performance Toolkit Technical Reference, General best practices for performance, and Performance best practices for Windows Store apps using C++, C#, and Visual Basic.

[Top]

Device testing

Visual Studio includes a simulator that you can use to run your Windows Store app in various device environments. For example, you can use the simulator to check whether your app works correctly with a variety of screen resolutions and with a variety of input hardware. You can simulate touch gestures even if you're developing the app on a computer that doesn't support touch. The following diagram shows AdventureWorks Shopper running in the simulator.

To start the simulator, click Simulator in the drop-down menu on the Debug toolbar in Visual Studio. The other choices in this drop-down menu are Local Machine and Remote Machine.

In addition to using the simulator, we also tested AdventureWorks Shopper on a variety of hardware. You can use remote debugging to test your app on a device that doesn't have Visual Studio installed on it. For more info see Running Windows Store apps on a remote machine, Testing Windows Store apps Running on a Device Using the Exploratory Test Window, and Testing Windows Store apps Running on a Device Using Microsoft Test Runner.

[Top]

Testing your app with the Windows App Certification Kit

Regardless of how your app will be deployed, you should validate and test it by using the Windows App Certification Kit. The kit performs a number of tests to verify that your app meets certain certification requirements for the Windows Store. These tests include:

  • Examining the app manifest to verify that its contents are correct.
  • Inspecting the resources defined in the app manifest to ensure that they are present and valid.
  • Testing the app's resilience and stability.
  • Determining how quickly the app starts and how fast it suspends.
  • Inspecting the app to verify that it calls only APIs for Windows Store apps.
  • Verifying that the app uses Windows security features.

You must run the Windows App Certification Kit on a release build of your app; otherwise, validation fails. For more info, see How to: Set Debug and Release Configurations.

In addition, it's possible to validate your app whenever you build it. If you're running Team Foundation Build, you can modify settings on your build machine so that the Windows App Certification Kit runs automatically every time your app is built. For more info, see Validating a package in automated builds.

For more info, see Testing your app with the Windows App Certification Kit.

[Top]

Creating a Windows Store certification checklist

You may choose to use the Windows Store as the primary method to make your app available. For info about how to prepare and submit your app, see Overview of publishing an app to the Windows Store.

As you plan your app, we recommend that you create a publishing-requirements checklist to use later when you test your app. This checklist can vary depending on how you've configured your business operations and the kind of app you're building. For more info and standard checklists, see Publishing your app to the Store.

Before creating your app package for upload to the Windows Store, be sure to do the following:

  • Review the app-submission checklist. This checklist indicates the information that you must provide when you upload your app. For more info, see App submission checklist.
  • Ensure that you have validated a release build of your app with the Windows App Certification Kit. For more info, see Testing your app with the Windows App Certification Kit on this page.
  • Take some screen shots that show off the key features of your app.
  • Have other developers test your app. For more info, see Sharing an app package locally.

In addition, if your app collects personal data or uses software that is provided by others, you must also include a privacy statement or additional license terms.

[Top]

Deploying and managing Windows Store apps

While you can use the Windows Store to market and distribute apps, business apps will often be distributed directly to the end-user by the IT organization within a company. The process of installing apps on Microsoft Windows devices without going through the Windows Store is called side-loading. For info about some best practices to help ensure that users have a good experience installing and running side-loaded apps for the first time, see Deployment.

IT managers have several options for managing side-loaded apps and apps distributed from the Windows Store. For more info see Management of Windows Store apps.