question

BrianWhitmer-0655 avatar image
0 Votes"
BrianWhitmer-0655 asked ·

Tracking gaze interactions outside the window

Looking at the new eye gaze tracking functionality:

https://docs.microsoft.com/en-us/windows/uwp/design/input/gaze-interactions

Is there a way to track gaze events for the whole screen, instead of just those that happen in the app window? I want to be able to use eye gestures as a sort of shortcut tool, but so far I can only figure out how to track gaze point data when my app window has focus.

Any pointers would be much appreciated!

uwpazure-virtual-machines-images
· 1
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Is there an update? Please 'Accept as answer' if any of the answer provided below helped you, so that it can benefit others in the community.

0 Votes 0 · ·
RoyLi-MSFT avatar image
1 Vote"
RoyLi-MSFT answered ·

Hello,

Welcome to Microsoft Q&A!

UWP apps could only get the events that happen inside its window. This is by design. UWP apps are different from traditional desktop apps. UWP apps can't handle such global system events like click events or gaze interactions outside the window. So this is not possible to implement this in UWP apps.

Thank you.

· 1 · Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks for the reply and the detailed answer. Do you know if there is a way to get global gaze interactions (outside the window) in a non-UWP environment? Any hints in this direction would be much appreciated, I'm trying to figure this out as I go but haven't been able to make much headway in the available documentation.

Have a great day! -Brian

0 Votes 0 · ·
jadailey-MSFT avatar image
1 Vote"
jadailey-MSFT answered ·

Gaze tracking (GT) is a new technology and will likely change and expand in the future. At this time the GT capabilities are available in UWP only. Because of this they function within the context of your app's window.

To capture gaze for the full screen you can expand your app to full screen. However, it sounds like your intent is to get gaze tracking for the entire system independent of the active window. Unfortunately this is not possible with the current GT implementation.

Do you have a specific scenario where you need access to gaze tracking for the entire system? If so please post it here in the comments. I can pass it along to the GT team so they can be aware of how our devs would like to use GT.

· 1 · Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Yes thank you, I would love for you to pass the word along. I am looking at developing an app that would allow me to define "hot" targets, probably like on the corners of the screen, where if a user fixed their gaze for a period of time it would trigger a user-defined shortcut action or pop up a quick-actions menu. I think there could be lots of value for productivity and even gaming that could benefit from this sort of application.

Additionally, my friend created an app called OptiKey for people to be able to communicate and control their screen in a hybrid app. Right now they have to use the vendor-provided libraries for eye tracking in order to get the screen coordinates, but it would be useful for them to be able to use a standard Windows library instead.

Thanks for passing the request along! It would be great to have the option for global gaze events!

0 Votes 0 · ·