We have developed a Xamarin(Xamarin.Android & Xamarin.iOS) application. We have written UI Test cases for the same using Xamarin.UITest test suite.We are using the latest version of Xamarin.UI Test nuget package(3.0.12) and using Microsoft Visual Studio Enterprise 2019 Version 16.8.4
. We have noticed inconsistent behaviour for our test runs. Please find our findings below.
• The same test case which was passed for one test run is seen failed on the next test run. - inconsisitent
Please see the below screen shots for Test Case 05(a).
• Sometimes we could see some kind of a time lag on the device which we run that the test explorer status is not matching with the device screen. For example, in the test explorer, we might see Test case no: 4 is being executed, but in the device we could only see the test case no: 3 is just being executed. Once this happens, we could find all the next Test cases fail.
• Also for some events (for example, the item click of the Navigation drawer) it takes more time to trigger the event. This too happens intermittently that same type of events consume different timings.