Xamarin UI Test- Inconsistent Behavior

Nair, Deepak 1 Reputation point
2021-02-04T10:08:06.227+00:00

We have developed a Xamarin(Xamarin.Android & Xamarin.iOS) application. We have written UI Test cases for the same using Xamarin.UITest test suite.We are using the latest version of Xamarin.UI Test nuget package(3.0.12) and using Microsoft Visual Studio Enterprise 2019 Version 16.8.4

. We have noticed inconsistent behaviour for our test runs. Please find our findings below.
• The same test case which was passed for one test run is seen failed on the next test run. - inconsisitent
Please see the below screen shots for Test Case 05(a).
• Sometimes we could see some kind of a time lag on the device which we run that the test explorer status is not matching with the device screen. For example, in the test explorer, we might see Test case no: 4 is being executed, but in the device we could only see the test case no: 3 is just being executed. Once this happens, we could find all the next Test cases fail.
• Also for some events (for example, the item click of the Navigation drawer) it takes more time to trigger the event. This too happens intermittently that same type of events consume different timings.

63994-test-run-3.png

63956-test-run-1.png

63849-test-run-2.png

Xamarin
Xamarin
A Microsoft open-source app platform for building Android and iOS apps with .NET and C#.
5,296 questions
Visual Studio Testing
Visual Studio Testing
Visual Studio: A family of Microsoft suites of integrated development tools for building applications for Windows, the web and mobile devices.Testing: The act or process of applying tests as a means of analysis or diagnosis.
329 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. John Hardman 261 Reputation points
    2021-02-05T17:51:56.013+00:00

    You haven't shown the source of your tests, so these are generalities:
    (1) Don't have any test rely on state resulting from another test. Tests should be independent, and should clean up anything that might be left over from a previous test.
    (2) If a test waits a certain length of time for something to happen, make sure that the test checks that whatever it is happened, rather than just continuing on the assumption that it happened. e.g. don't wait 3 seconds for a page to be populated and ready to accept user input. Instead, wait for the view that you want to interact with to be visible and check that it has focus. If it doesn't get focus within 60 seconds (not 3), fail that test. Tests should be designed to be resilient, not fragile, so should (except in very specific circumstances) not fail because the test runs slowly.
    (3) Read "The Art of Unit Testing" by Roy Osherove. I don't agree with everything in it, but it contains a lot of useful advice that applies to UI testing too.