question

SK123-7670 avatar image
0 Votes"
SK123-7670 asked ·

SetCursorPos API not allowing to draw over snipping tool with pen or highlight even if correct mouse coordinates are provided,

I use SetCursorPos to set mouse coordinates over remote Desktop. and the use mouse_event API to provide appropriate flag and wheel info: ::SetCursorPos(x,y); ::mouse_event(flags, 0, 0, wheel_movement, 0); It is not allowing to draw over snipping tool with pen or highlight or Move the multiple monitor position from display settings or Move the scroll bar with left mouse click on Internet explorer.

c++winapi-sdk
· 1
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @SK123-7670, I want to confirm with you again, do you want to simulate mouse events on remote Desktop? Note that SetCursorPos cannot simulate the mouse click event. You may need to simulate mouse movement and click directly. In addition, mouse_event has been superseded. Use SendInput instead.


0 Votes 0 ·
SK123-7670 avatar image
0 Votes"
SK123-7670 answered ·

Yes I need to send mouse coordinates to remote desktop.So doing like, setting the mouse coordinates that is sent from viewer to remote desktop via "SetCursorPos" and mouse related flag and other details via "mouse_events" or "SendInput".
But by doing this some application like "Drawing on snipping tool" by left click mouse move not able to accomplish.

Cannot send mouse coordinates via "mouse_events" or "SendInput" directly because it causes problem in case of more than one monitor is present.(need to calculate for each monitor, but can have n number of monitor).



·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

DrakeWu-MSFT avatar image
0 Votes"
DrakeWu-MSFT answered ·

Hi @SK123-7670 I cannot reproduce the problem you mentioned. I can directly use SendInput on the remote desktop to draw points or lines on the snipping tool.

 #include <windows.h>
 #include <iostream>
 using namespace std;
    
 void clickFromTo(int x, int y,int w,int h) {
     INPUT click[3] = {};
    
     click[0].type = INPUT_MOUSE;
     click[0].mi.dwFlags = MOUSEEVENTF_MOVE| MOUSEEVENTF_LEFTDOWN| MOUSEEVENTF_ABSOLUTE;// Release the mouse before moving it
    
    
     DWORD fScreenWidth = ::GetSystemMetrics(SM_CXSCREEN);
     DWORD fScreenHeight = ::GetSystemMetrics(SM_CYSCREEN);
     click[1].type = INPUT_MOUSE;
     click[0].mi.dx = click[1].mi.dx = MulDiv(x, 65535, fScreenWidth);
     click[0].mi.dy = click[1].mi.dy = MulDiv(y, 65535, fScreenHeight);
     click[2].mi.dx = MulDiv(x + w, 65535, fScreenWidth);
     click[2].mi.dy = MulDiv(y + h, 65535, fScreenHeight);
     click[1].mi.dwFlags = MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE;
    
     click[2].type = INPUT_MOUSE;
     click[2].mi.dwFlags = MOUSEEVENTF_LEFTUP | MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE;
    
     SendInput(3, click, sizeof(INPUT));
 }
 void mouseClick() {
     INPUT click[2] = {};
    
     click[0].type = INPUT_MOUSE;
     click[0].mi.dwFlags = MOUSEEVENTF_LEFTDOWN;
        
     click[1].type = INPUT_MOUSE;
     click[1].mi.dwFlags = MOUSEEVENTF_LEFTUP;
     SendInput(2, click, sizeof(INPUT));
 }
 int main()
 {
     clickFromTo(2200, 500, 50, 50);
     //SetCursorPos(2200, 500);
     //mouseClick();
 }

72965-1.png

Are the coordinates you use only relative to the remote desktop coordinates or based on this virtual screen? If it is relative For remote desktop coordinates, the corresponding virtual screen coordinates should be calculated first.


If the answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.


1.png (16.6 KiB)
· 7 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

yes it is relative mouse coordinates.
Can you please mention how to calculate virtual screen coordinates?

0 Votes 0 ·

Let me explain, SetCursorPos/mouse_events/SendInput cannot send the mouse coordinates to the remote desktop. You must specify absolute mouse coordinates for these functions. You must find the window coordinates of the remote desktop, and then convert the relative coordinates to absolute coordinates(Relative coordinates (x1, y1) + coordinates of the upper left corner of the remote window (x2, y2) + (the border of the remote window, the size of the title bar)).

0 Votes 0 ·

Just needed to confirm,
instead of the below calculation:

click[0].mi.dx = click[1].mi.dx = MulDiv(x, 65535, fScreenWidth);
click[0].mi.dy = click[1].mi.dy = MulDiv(y, 65535, fScreenHeight);

we have to use the below calculation:

(Relative coordinates (x1, y1) + coordinates of the upper left corner of the remote window (x2, y2) + (the border of the remote window, the size of the title bar)). ?

0 Votes 0 ·

Also I suppose, for multi monitor, the above calculation wont work, even if considering absolute mouse coordinates?

0 Votes 0 ·

The multi monitors are in the same coordinate system, so the absolute coordinates do work. You could check the Virtual Screen above. The screenshot results also use 2 monitors ({0,0,1920,1080} {1920,0,3840,1080}), and The remote window is located in the second monitor.


0 Votes 0 ·

okay sure. But my situation is like, my remote desktop has dual monitor,
so I have to calculate coordinates for each desktop separately by considering screen width ?

0 Votes 0 ·
Show more comments