GetMessageExtraInfo signature missing for touch input

Question

When checking in c/c++ applications for touch input on Windows 10, while handling WM_INPUT window messages, the touch signature and identifier for touch (bit 7 being set) is missing. This causes apps to detect the input as being from a mouse instead, while it's actually from a touch input (verified using Remote Desktop on Android). The code used to check the source of the event to split touch and mouse events:
https://github.com/libsdl-org/SDL/blob/main/src/video/windows/SDL_windowsevents.c

Look for the function GetMouseMessageSource The code tries to filter the events, but software using the library requiring split touch and mouse events still receive invalid mouse movement on top of the processed touch movement being translated into mouse movement, causing double movement in the case of in-app movement handling.

My conversation with the library devs trying to find a solution:
https://github.com/libsdl-org/SDL/issues/3316

Answer 1

Hi, @Tim Arts
Not sure how you program touch input, but as far as I know, WM_INPUT message is used for Raw Input.
You could check Windows Touch Programming Reference, use RegisterTouchWindow to register your window as touchable, and then process WM_TOUCH message.

---

If the answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 2

Well, my app doesn't use Windows events directly.
It's a cross-platform library maintained by a lot of people.

My app uses the SDL2 library to handle all cross-platform stuff(including Windows events).
My app receives events like button down, mouse movement, button up, touch down, touch movement, touch up. All of those combined with an ID SDL2 provides to split it's own emulated mouse and touch movement based on the actual mouse and touch events it receives from the OS.

Now, what happens is that SDL2 uses getMessageExtraInfo on all of those events to determine if those events are from mouse, touch or pen. For the mouse events that works correctly, but for the WM_INPUT it thinks that it's a mouse event when it's really simulated touch events from RDP on Android (Microsoft's RDP client is what I'm using and testing with). The bottom 8 bits are weirdly set in that case(values C7/C8/F7/F8/F9 back when testing) while the signature mentioned in the documentation wasn't present in the returned value at all (which it should, according to documentation). Thus my app receives doubled mouse (movement) inputs instead of just touch inputs.

Answer 3

SDL2's current event handler is this:

    case WM_MOUSEMOVE:
            {
                SDL_Mouse *mouse = SDL_GetMouse();
                if (!mouse->relative_mode || mouse->relative_mode_warp) {
                    /* Only generate mouse events for real mouse */
                    if (GetMouseMessageSource() != SDL_MOUSE_EVENT_SOURCE_TOUCH &&
                        lParam != data->last_pointer_update) {
                        SDL_SendMouseMotion(data->window, 0, 0, GET_X_LPARAM(lParam), GET_Y_LPARAM(lParam));
                        if (isWin10FCUorNewer && mouse->relative_mode_warp) {
                            /* To work around #3931, Win10 bug introduced in Fall Creators Update, where
                               SetCursorPos() (SDL_WarpMouseInWindow()) doesn't reliably generate mouse events anymore,
                               after each windows mouse event generate a fake event for the middle of the window
                               if relative_mode_warp is used */
                            int center_x = 0, center_y = 0;
                            SDL_GetWindowSize(data->window, &center_x, &center_y);
                            center_x /= 2;
                            center_y /= 2;
                            SDL_SendMouseMotion(data->window, 0, 0, center_x, center_y);
                        }
                    }
                } else {
                    /* We still need to update focus */
                    SDL_SetMouseFocus(data->window);
                }
            }
            /* don't break here, fall through to check the wParam like the button presses */
        case WM_LBUTTONUP:
        case WM_RBUTTONUP:
        case WM_MBUTTONUP:
        case WM_XBUTTONUP:
        case WM_LBUTTONDOWN:
        case WM_LBUTTONDBLCLK:
        case WM_RBUTTONDOWN:
        case WM_RBUTTONDBLCLK:
        case WM_MBUTTONDOWN:
        case WM_MBUTTONDBLCLK:
        case WM_XBUTTONDOWN:
        case WM_XBUTTONDBLCLK:
            {
                SDL_Mouse *mouse = SDL_GetMouse();
                if (!mouse->relative_mode || mouse->relative_mode_warp) {
                    if (GetMouseMessageSource() != SDL_MOUSE_EVENT_SOURCE_TOUCH &&
                        lParam != data->last_pointer_update) {
                        WIN_CheckWParamMouseButtons(wParam, data, 0);
                    }
                }
            }
            break;

        case WM_INPUT:
            {
                SDL_Mouse *mouse = SDL_GetMouse();
                HRAWINPUT hRawInput = (HRAWINPUT)lParam;
                RAWINPUT inp;
                UINT size = sizeof(inp);
                const SDL_bool isRelative = mouse->relative_mode || mouse->relative_mode_warp;
                const SDL_bool isCapture = ((data->window->flags & SDL_WINDOW_MOUSE_CAPTURE) != 0);

                if (!isRelative || mouse->focus != data->window) {
                    if (!isCapture) {
                        break;
                    }
                }

                GetRawInputData(hRawInput, RID_INPUT, &inp, &size, sizeof(RAWINPUTHEADER));

                /* Mouse data (ignoring synthetic mouse events generated for touchscreens) */
                if (inp.header.dwType == RIM_TYPEMOUSE) {
                    if (GetMouseMessageSource() == SDL_MOUSE_EVENT_SOURCE_TOUCH ||
                        (GetMessageExtraInfo() & 0x82) == 0x82) {
                        break;
                    }
                    if (isRelative) {
                        RAWMOUSE* rawmouse = &inp.data.mouse;

                        if ((rawmouse->usFlags & 0x01) == MOUSE_MOVE_RELATIVE) {
                            SDL_SendMouseMotion(data->window, 0, 1, (int)rawmouse->lLastX, (int)rawmouse->lLastY);
                        } else if (rawmouse->lLastX || rawmouse->lLastY) {
                            /* synthesize relative moves from the abs position */
                            static SDL_Point lastMousePoint;
                            SDL_bool virtual_desktop = (rawmouse->usFlags & MOUSE_VIRTUAL_DESKTOP) ? SDL_TRUE : SDL_FALSE;
                            int w = GetSystemMetrics(virtual_desktop ? SM_CXVIRTUALSCREEN : SM_CXSCREEN);
                            int h = GetSystemMetrics(virtual_desktop ? SM_CYVIRTUALSCREEN : SM_CYSCREEN);
                            int x = (int)(((float)rawmouse->lLastX / 65535.0f) * w);
                            int y = (int)(((float)rawmouse->lLastY / 65535.0f) * h);

                            if (lastMousePoint.x == 0 && lastMousePoint.y == 0) {
                                lastMousePoint.x = x;
                                lastMousePoint.y = y;
                            }

                            SDL_SendMouseMotion(data->window, 0, 1, (int)(x-lastMousePoint.x), (int)(y-lastMousePoint.y));

                            lastMousePoint.x = x;
                            lastMousePoint.y = y;
                        }
                        WIN_CheckRawMouseButtons(rawmouse->usButtonFlags, data);
                    } else if (isCapture) {
                        /* we check for where Windows thinks the system cursor lives in this case, so we don't really lose mouse accel, etc. */
                        POINT pt;
                        RECT hwndRect;
                        HWND currentHnd;

                        GetCursorPos(&pt);
                        currentHnd = WindowFromPoint(pt);
                        ScreenToClient(hwnd, &pt);
                        GetClientRect(hwnd, &hwndRect);

                        /* if in the window, WM_MOUSEMOVE, etc, will cover it. */
                        if(currentHnd != hwnd || pt.x < 0 || pt.y < 0 || pt.x > hwndRect.right || pt.y > hwndRect.right) {
                            SDL_bool swapButtons = GetSystemMetrics(SM_SWAPBUTTON) != 0;

                            SDL_SendMouseMotion(data->window, 0, 0, (int)pt.x, (int)pt.y);
                            SDL_SendMouseButton(data->window, 0, GetAsyncKeyState(VK_LBUTTON) & 0x8000 ? SDL_PRESSED : SDL_RELEASED, !swapButtons ? SDL_BUTTON_LEFT : SDL_BUTTON_RIGHT);
                            SDL_SendMouseButton(data->window, 0, GetAsyncKeyState(VK_RBUTTON) & 0x8000 ? SDL_PRESSED : SDL_RELEASED, !swapButtons ? SDL_BUTTON_RIGHT : SDL_BUTTON_LEFT);
                            SDL_SendMouseButton(data->window, 0, GetAsyncKeyState(VK_MBUTTON) & 0x8000 ? SDL_PRESSED : SDL_RELEASED, SDL_BUTTON_MIDDLE);
                            SDL_SendMouseButton(data->window, 0, GetAsyncKeyState(VK_XBUTTON1) & 0x8000 ? SDL_PRESSED : SDL_RELEASED, SDL_BUTTON_X1);
                            SDL_SendMouseButton(data->window, 0, GetAsyncKeyState(VK_XBUTTON2) & 0x8000 ? SDL_PRESSED : SDL_RELEASED, SDL_BUTTON_X2);
                        }
                    } else {
                        SDL_assert(0 && "Shouldn't happen");
                    }
                }
            }
            break;

And the detection code is this:

/* We want to generate mouse events from mouse and pen, and touch events from touchscreens */
#define MI_WP_SIGNATURE         0xFF515700
#define MI_WP_SIGNATURE_MASK    0xFFFFFF00
#define IsTouchEvent(dw) ((dw) & MI_WP_SIGNATURE_MASK) == MI_WP_SIGNATURE

typedef enum
{
    SDL_MOUSE_EVENT_SOURCE_UNKNOWN,
    SDL_MOUSE_EVENT_SOURCE_MOUSE,
    SDL_MOUSE_EVENT_SOURCE_TOUCH,
    SDL_MOUSE_EVENT_SOURCE_PEN,
} SDL_MOUSE_EVENT_SOURCE;

static SDL_MOUSE_EVENT_SOURCE GetMouseMessageSource()
{
    LPARAM extrainfo = GetMessageExtraInfo();
    /* Mouse data (ignoring synthetic mouse events generated for touchscreens) */
    /* Versions below Vista will set the low 7 bits to the Mouse ID and don't use bit 7:
       Check bits 8-32 for the signature (which will indicate a Tablet PC Pen or Touch Device).
       Only check bit 7 when Vista and up(Cleared=Pen, Set=Touch(which we need to filter out)),
       when the signature is set. The Mouse ID will be zero for an actual mouse. */
    if (IsTouchEvent(extrainfo)) {
        if (extrainfo & 0x80) {
            return SDL_MOUSE_EVENT_SOURCE_TOUCH;
        } else {
            return SDL_MOUSE_EVENT_SOURCE_PEN;
        }
    }
    return SDL_MOUSE_EVENT_SOURCE_MOUSE;
}

But since the signature is missing in this case, it leaks through when it shouldn't? The values reported for the lower 8 bits of getMessageExtraInfo's result are C7/C8/F7/F8/F9 in the GetMessageExtraInfo result when using RDP in touch input mode, tapping and dragging accross the screen.
The signature in the upper 24-bits is 0 in this case, which is obviously incorrect.