question

AnschultzReese-6767 avatar image
0 Votes"
AnschultzReese-6767 asked TimonYang-MSFT commented

UI Automation - Modal dialog prevents automation

I am automating an application that, when the automation uses Select, opens a modal dialog. I've seen this issue discussed several places with a couple of suggestions. I've tried all of the suggestions without success. To post this issue I put together a small application that exhibits the same issue. The application contains a listbox and a button. When a listbox item selection changes it opens a message box. The button starts an automation sequence running which selects the first item and then tries to dismiss the dialog box by pressing the OK button.

The automation sequence is running in a non-UI MTA thread (Task). Because the Select blocks, it is invoked from another non-UI MTA thread (Task). When the main automation sequence moves on after starting the Select task, it hangs when it tries to Find the dialog that pops up.

I had seen suggestions to add a WindowOpenedEvent handler and push the OK button from there. I've included that in this application but the handler never gets called. In my real application the handler gets called for all other dialogs until one is opened from within the Select call.

Are there any other ideas about how to dismiss this dialog from the automation sequence?

C# .Net framework 4.8

MainWindow.xaml.cs:

 using System;
 using System.Diagnostics;
 using System.Threading;
 using System.Threading.Tasks;
 using System.Windows;
 using System.Windows.Automation;
 using System.Windows.Controls;
 using System.Windows.Interop;
    
 namespace AutomationSelectTest
 {
     /// <summary>
     /// Interaction logic for MainWindow.xaml
     /// </summary>
     public partial class MainWindow : Window
     {
         private Task _task;
    
         public MainWindow()
         {
             InitializeComponent();
         }
    
         private void ListBox_SelectionChanged(object sender, SelectionChangedEventArgs e)
         {
             MessageBox.Show("Selection changed");
         }
    
         private void Button_Click(object sender, RoutedEventArgs e)
         {
             var windowHandle = new WindowInteropHelper(this).Handle;
             _task = Task.Factory.StartNew(() => AutomationSequence(windowHandle));
         }
    
         private static readonly System.Windows.Automation.Condition ButtonOkCondition = new AndCondition(
             new PropertyCondition(AutomationElement.ClassNameProperty, "Button"),
             new PropertyCondition(AutomationElement.NameProperty, "OK"));
    
         private static void AutomationSequence(IntPtr windowHandle)
         {
             var application = AutomationElement.FromHandle(windowHandle);
             Automation.AddAutomationEventHandler(WindowPattern.WindowOpenedEvent, application, TreeScope.Subtree, Window_Opened);
             var listBoxItem = application.FindFirst(TreeScope.Descendants, new PropertyCondition(AutomationElement.ClassNameProperty, "ListBoxItem"));
             var listBoxSelectionItemPattern = listBoxItem.GetCurrentPattern(SelectionItemPattern.Pattern) as SelectionItemPattern ?? throw new InvalidOperationException();
             // Because the call to Select will block, run it as a separate task
             var selectionTask = Task.Factory.StartNew(() => listBoxSelectionItemPattern.Select());
             Thread.Sleep (500); // Give the Select task time to do its thing
             var dialog = application.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.LocalizedControlTypeProperty, "Dialog"));
             var buttonOk = dialog.FindFirst(TreeScope.Subtree, ButtonOkCondition);
             var buttonOkInvokePattern = buttonOk.GetCurrentPattern(InvokePattern.Pattern) as InvokePattern ?? throw new InvalidOperationException();
             buttonOkInvokePattern.Invoke();
             // Now wait for the Select to return
             selectionTask.Wait();
             Automation.RemoveAutomationEventHandler(WindowPattern.WindowOpenedEvent, application, Window_Opened);
         }
    
         private static void Window_Opened(object sender, AutomationEventArgs e)
         {
             // Never gets here
             Debugger.Break();
         }
     }
 }

MainWindow.xaml:

 <Window x:Class="AutomationSelectTest.MainWindow"
         xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
         xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
         xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
         xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
         xmlns:local="clr-namespace:AutomationSelectTest"
         mc:Ignorable="d"
         Title="MainWindow" Height="450" Width="800">
     <Grid>
         <ListBox HorizontalAlignment="Left" Height="144" Margin="10,10,0,0" VerticalAlignment="Top" Width="151" SelectionChanged="ListBox_SelectionChanged">
             <ListBoxItem Content="ListBoxItem1"/>
             <ListBoxItem Content="ListBoxItem2"/>
             <ListBoxItem Content="ListBoxItem3"/>
         </ListBox>
         <Button Content="Run Automation" HorizontalAlignment="Left" Height="39" Margin="10,159,0,0" VerticalAlignment="Top" Width="151" Click="Button_Click"/>
    
         </Grid>
     </Window>















dotnet-csharpwindows-apidotnet-runtime
· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@AnschultzReese-6767
Can you get the button and dialog correctly through these codes?

    private static readonly Condition ButtonOkCondition = new AndCondition(
         new PropertyCondition(AutomationElement.ClassNameProperty, "button"),
         new PropertyCondition(AutomationElement.NameProperty, "OK"));
   var dialog = application.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.LocalizedControlTypeProperty, "dialog"));

When I tested it, UI Automation had no effect on WPF's MessageBox.

When I use Inspect to inspect UI elements, WPF cannot get the properties of these controls:

99744-4.png

But when I use WinFrom for testing, it is ok:

99745-3.png




0 Votes 0 ·
4.png (19.6 KiB)
3.png (18.4 KiB)

@TimonYang-MSFT, thanks for the quick reply.

This was some code I quickly threw together to reproduce the problem. You may not actually be able to get the dialog using that line, but, the point is that when you call the FindFirst that is supposed to find the dialog it shouldn't hang. Since the code always hung I can't tell you for certain that it finds the dialog.

Note that for this example you must not have selected the first item in the list before pushing the button to run the automation script, otherwise a dialog won't appear because the selection didn't change.

0 Votes 0 ·

@TimonYang-MSFT, to follow up, you were correct that it wouldn't find the dialog. I have updated the original question to change the localized control type name to "Dialog" from "dialog".

If you replace line 45 (starting the Select task) with Thread.Sleep(10*1000) and comment off line 51 that waits for the task to finish, and then, when you start the automation manually select an item within 10 seconds the dialog will pop-up, the Window_Opened will get hit and the dialog will automatically be closed by the automation code.

It is calling Select that causes all of the automation to hang.

0 Votes 0 ·

@AnschultzReese-6767
When I click the button, the dialog pops up immediately, and the program encounters an exception:
100553-1.png
This seems to be inconsistent with your description. Are my steps different from yours?

0 Votes 0 ·
1.png (26.2 KiB)

@TimonYang-MSFT
I might need a little more context, but...

I see that you have added a commented off Sleep. There is a possible race condition where the FindFirst for the dialog occurs before the Select occurs, so, at the point where you have a commented off Sleep you may want to remove the comment and reduce it to 1 second, just to make sure that the Select task has had time to run. If this race condition is occurring the exception you are seeing makes sense because the dialog wouldn't, yet, be on the screen at that time.

Thanks again for looking into this!

P.S. I udpated the original code to include the Sleep.

0 Votes 0 ·

@TimonYang-MSFT Were you ever able to reproduce this problem?

0 Votes 0 ·

@AnschultzReese-6767
Yes, but I haven't found a feasible solution so far.

1 Vote 1 ·

0 Answers