Compatibility Testing Scenarios

  • Article

 

Microsoft Corporation

May 2006

Applies to:
   .NET Framework

Summary: This document outlines best practices for testing a managed application or component for compatibility with newer runtime versions. It covers different configurations you should test to ensure that the application continues to run properly. Full details on compatibility are available in Microsoft .NET Framework 1.1 and 2.0 Compatibility. If you have not already read that document, we recommend that you read it before proceeding. (7 printed pages)

Contents

Introduction
Application Scenarios
Test 1: Installs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side
Test 2: Runs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side
Test 3: Runs correctly on a computer with the .NET Framework 1.x and 2.0 installed Side-by-side with the application configured to run against the .NET Framework 1.x
Test 4: Runs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side with the application configured to run against the .NET Framework 2.0
Test 5: Installs correctly on a computer with only the .NET Framework 2.0 installed
Background

Introduction

Many applications using managed code require that a particular version of the .NET Framework be present on the computer. Some applications redistribute a particular release of the.NET Framework. In some cases, applications that were developed on a specific version of the .NET Framework may encounter issues when trying to execute on a newer version of the Framework. As explained in Microsoft .NET Framework 1.1 and 2.0 Compatibility, managed applications run by default on the version of the Framework they were built against. However, in certain situations where only a newer version of the Framework is available or a hosting application requires a particular version, an application may run against a newer version. To ensure the best customer experience, we recommend that you test your applications for compatibility according to the steps described below.

Application Scenarios

There are 5 scenarios that have to be covered to verify that your application works correctly against the newer .NET Framework version. In order to ensure that we can address any compatibility issues that are discovered during the testing process.

One of the important principles of .NET Framework compatibility is the idea of Side-by-side (SxS) execution. As covered in detail in Side-by-Side Execution of the .NET Framework, this means that a fully managed application will try to run against the version of the .NET Framework with which it was built. In certain configurations, including when being hosted in a native application, the application or component will need to run with the most recent version of the .NET Framework available on the computer, potentially exposing a compatibility issue.

Our testing has shown that it is important to test both installing and running the application. Installation of the application will sometimes depend on a particular version of the .NET Framework being present on the computer.

The scenarios that need to be tested for each of your applications are:

  1. Installs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side
  2. Runs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side.
  3. Runs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side with the application configured to run against the .NET Framework 1.x.
  4. Runs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side with the application configured to run against the .NET Framework 2.0.
  5. Installs correctly on a computer with only the .NET Framework 2.0 installed.

Test 1: Installs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side

This test verifies that your application will install correctly on a computer that has the .NET Framework 1.x and 2.0 installed on it.

Testing Steps

  1. Make sure you have the .NET Framework 1.x installed on your computer.
  2. If you already have an older version of the .NET Framework 2.0, uninstall it so that you can download the latest .NET Framework 2.0 bits.
  3. Before you install .NET Framework 2.0, test the behavior of your application on a clean 1.x computer. Uninstall the application once you're done.
  4. Download and install the latest .NET Framework 2.0.
  5. Install your application on this computer.
  6. Record your results

Possible Results

  • Yes—The application installed as expected, proceed to Test 2.
  • No—The application failed during installation. Please try the following:
    • Uninstall the .NET Framework 2.0 from your computer.
    • Install on your application on the computer.
    • Reinstall .NET 2.0.
    • Move on to Test 2.
  • No Installer—the application does not need an installer. Proceed to Test 2.

Test 2: Runs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side

This test verifies that your application functions correctly on a computer that has the .NET Framework 1.x and 2.0 installed on it.

Testing Steps

  1. Run your application on the computer.
  2. Explore all possible features to ensure that they are working correctly (covering as much code as possible). Pay particular attention to areas that save or recover serialized data.

Possible Results

The possible results are Yes/No.

  • Yes—The application ran as expected. Proceed to Test 4.
  • No—The application did not run as expected. Proceed to Test 3.

Test 3: Runs correctly on a computer with the .NET Framework 1.x and 2.0 installed Side-by-side with the application configured to run against the .NET Framework 1.x

Note You can skip this test if Test 2 passed.

This scenario tests to see if your application can be made to work correctly by locking it to an older runtime when v2.0 is installed on the computer. This is only applicable if the application or component is hosted by a native application.

Testing Steps

  1. Create a configuration file to force your application to load using a particular version of the .NET Framework. To do this you would have to create a new file and call it <appname>.exe.config (unless it already exists). This file should be placed in the same directory as the application being tested.

  2. Add this snippet to the file:

                <configuration>
     <startup>
     <supportedRuntime version="v1.1.4322" />       
     </startup>
  </configuration>

    You can use the same code fragment to lock-back to v1.0 except that the version should be v1.0.3705.

    Note   In production you should NOT lock back add-ins by locking back the main executable; for more details look at the Microsoft .NET Framework 1.1 and 2.0 Compatibility white paper.

  3. Run the application once more and test the feature(s) that failed in Test 2.

Possible Results

  • Yes—The application now ran as expected. Proceed to Test 4.
  • No—The application still did not run as expected.
  • Not Tested—Please proceed to Test 4.

Test 4: Runs correctly on a computer with the .NET Framework 1.x and 2.0 installed side-by-side with the application configured to run against the .NET Framework 2.0

This test configures all applications on your computer to run against the .NET Framework 2.0 and allows you to test forward-compatibility.

Testing Steps

  1. Configure all managed applications on your computer to require the latest version of the .NET Framework

  2. Create or set the following registry key:

    [HKEY_LOCAL_COMPUTER\SOFTWARE\Microsoft\.NETFramework] 
"OnlyUseLatestCLR"=dword:00000001

  3. GotDotNet User Sample: Ensuring your Applications Execute on the Whidbey CLR contains a zip file containing two files that let you toggle the value stored in this registry key.

  4. Explore all possible features of your application in order to verify that they work correctly using the .NET Framework 2.0.

  5. Restore "OnlyUseLatestCLR"=dword:00000000 after you're done with this test.

Possible Results

  • Yes—All applications functioned as expected.
  • No—You will need to fix the issues.

Test 5: Installs correctly on a computer with only the .NET Framework 2.0 installed

This test verifies that your application installs correctly on a computer that has only the .NET Framework 2.0 installed on it.

Testing Steps

  1. Create a clean computer or Virtual PC image with only the .NET Framework 2.0 installed.
  2. Install your application on that computer. Your application may require that you download .NET Framework 1.x in order to proceed with the installation.

Possible Results

  • Yes—The application installed correctly.
  • Yes after installing the .NET Framework 1.x—The application required the .NET Framework 1.x and required its installation
  • No—The application did not install as expected. Please fill out a bug report with the details.
  • No Installer—The application does not require an installer.

Background

.NET Framework SDK documents and articles describe Side-by-side in further detail and how to configure your application to run a particular release of the .NET Framework for a particular application model e.g. .EXE application, web application, managed COM component.

MSDN documentation