Making and tuning a “safe automated mouse click”

  • [アーティクル]

Over the last week I have been talking about the components needed to automate the mouse in Hyper-V.  This is because I have been working on some automation scripts for my environment.  One problem that I have encountered is this:

I will have a virtual machine that is being controlled by one of my scripts.  Something will go wrong with the script – and I will try and fix it manually.  But as I am in the middle of doing something – my script will move the mouse and click on something.

Whoops.

To solve this – I have made a “safe mouse click” function.  What this function does is:

  1. Take an x and y coordinate, and a mouse button index
  2. Check the state of the mouse buttons in the virtual machine
  3. Check to see if the mouse cursor is moving
  4. If there is no evidence of mouse activity – move the cursor and click.
  5. Otherwise exit with a warning.

Here is the code:

$VMName = "Windows 10 Enterprise" $VMCS = Get-WmiObject -Namespace root\virtualization\v2 -Class Msvm_ComputerSystem -Filter "ElementName='$($VMName)'" $mouseWMIPath = $VMCS.GetRelated("Msvm_SyntheticMouse").__PATHfunction safeMouseClick { param ( [int]$x, [int]$y, [int]$button ) $localMouse = [wmi]$mouseWMIPath $button1State1 = $localMouse.GetButtonState(1).IsDown $button2State1 = $localMouse.GetButtonState(2).IsDown $mouseX1 = $localMouse.HorizontalPosition $mouseY1 = $localMouse.VerticalPosition sleep -Milliseconds 10 $localMouse = [wmi]$mouseWMIPath $button1State2 = $localMouse.GetButtonState(1).IsDown $button2State2 = $localMouse.GetButtonState(2).IsDown $mouseX2 = $localMouse.HorizontalPosition $mouseY2 = $localMouse.VerticalPosition if (!($button1State1 -or $button2State1 -or $button1State2 -or $button2State2)) { # None of the buttons are down if (($mouseX1 -eq $mouseX2) -and ($mouseY1 -eq $mouseY2)) { # The mouse has not moved $localMouse.SetAbsolutePosition($x,$y) | out-null $localMouse.ClickButton($button) | out-null } else {write-host "Warning: The mouse is moving" -ForegroundColor Yellow} } else {write-host "Warning: A mouse button is down" -ForegroundColor Yellow} }

There is something interesting to highlight here:

At the beginning of this script I get the mouse WMI object, and store the WMI path.  I then use that path to create the mouse object again in the future.  There is an important reason for this:

When I first wrote this function I used:

$localMouse = $VMCS.GetRelated("Msvm_SyntheticMouse")

Whenever I needed to get the mouse.  But now I run:

$mouseWMIPath = $VMCS.GetRelated("Msvm_SyntheticMouse").__PATH

Once and then run:

$localMouse = [wmi]$mouseWMIPath

When I need to get the mouse.

I found that using the first approach was just too slow to click the mouse.  Investigation revealed that “$VMCS.GetRelated("Msvm_SyntheticMouse")” takes about 200 milliseconds. Calling it repeatedly in my mouse click function meant that every mouse click took about 500 milliseconds.  By getting the WMI path and using that – I reduced the amount of time for a safe mouse click to about 80 milliseconds.

Cheers,
Ben