Windows locked, sending virtual mouse and keyboard input to applications

Wont Tell 1 Reputation point
2021-06-25T10:03:00.637+00:00

I automate applications by sending virtual mouse and keyboard input to the GUI. This works fine. I use Active Accessibility API to get coordinates of a control or its label and then use Sendinput from user32.dll to interact with it.

I was wondering if its possible to run this kind of automation while the PC is locked. Most importantly to get input to firefox.

I read that a HID driver for a virtual device would be the way to go. This project looks promising https://github.com/djpnewton/vmulti. Im very open for other suggestions.

My first question is if the HID driver for a virtual device is guranteed to achieve getting virtual input to applications while the PC is locked.

Second questions is how I would interact with the HID driver for a virtual device. I assume the mouse would take coordinates of the screen and the keyboard would take, well, keys. I guess it all consits of two parts, the driver for the virtual device and the virtual device itself which I can make use of from .NET code.

Last question is, if the PC is locked and behind the lock screen is an "secure desktop". Is the GUI of the application windows even rendered with all controls so that I could make use of the Active Accessibility API and virtual inputs through the mentioned driver? Does firefox render websites then?

C#
C#
An object-oriented and type-safe programming language that has its roots in the C family of languages and includes support for component-oriented programming.
10,277 questions
C++
C++
A high-level, general-purpose programming language, created as an extension of the C programming language, that has object-oriented, generic, and functional features in addition to facilities for low-level memory manipulation.
3,537 questions
Windows Hardware Performance
Windows Hardware Performance
Windows: A family of Microsoft operating systems that run across personal computers, tablets, laptops, phones, internet of things devices, self-contained mixed reality headsets, large collaboration screens, and other devices.Hardware Performance: Delivering / providing hardware or hardware systems or adjusting / adapting hardware or hardware systems.
1,546 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Jayasankar Narayanapillai Jayakumaran 6 Reputation points
    2022-11-15T11:55:14.077+00:00

    The GUI is not present when the PC is locked. Only way known to me is to use a child windows session. create-remote-desktop-services-child-sessions. But you don't need this, you only need to use selenium api to do simulations in browser. UI is not required for it.

    0 comments