高级 HoloLens 仿真器和混合现实模拟器输入Advanced HoloLens Emulator and Mixed Reality Simulator input

大多数模拟器用户只需使用 HoloLens 模拟器Windows Mixed Reality 模拟器的基本输入控件。Most emulator users will only need to use the basic input controls for the HoloLens Emulator or the Windows Mixed Reality simulator. 下面的详细信息适用于已发现需要模拟更复杂输入的高级用户。The details below are for advanced users who have found a need to simulate more complex types of input.

概念Concepts

若要开始控制 HoloLens 模拟器和 Windows Mixed Reality 模拟器的虚拟输入,应该首先了解一些概念。To get started controlling the virtual input to the HoloLens Emulator and Windows Mixed Reality simulator, you should first understand a few concepts.

动作指的是控制和更改场景中某个对象的位置和方向。Motion refers to controlling and changing the position and orientation of something in the scene. 对于目标可控对象,运动由旋转和平移进行控制, (移动) 三个轴。For a targeted controllable object, motion is controlled with both rotation and translation (movement) along three axes.

  • 偏航:向左或向右旋转。Yaw: Turn left or right.
  • 音调:打开或关闭。Pitch: Turn up or down.
  • :并排滚动。Roll: Roll side-to-side.
  • X:向左或向右移动。X: Move left or right.
  • Y:向上或向下移动。Y: Move up or down.
  • Z:向前或向后移动。Z: Move forward or backward.

手势和运动控制器输入紧密映射到物理设备:Gesture and motion controller input are mapped closely to physical devices:

  • 操作:模拟按食指捏的操作,或在控制器上拉取操作按钮。Action: Simulates the action of pressing the forefinger to the thumb or pulling the action button on a controller. 例如,操作输入可用于模拟攻攻敲击手势,滚动查看内容,以及按下并保持。For example, the Action input can be used to simulate the air-tap gesture, to scroll through content, and to press-and-hold.
  • 布隆/System 手势或 home: HoloLens 布隆/系统手势或控制器的 "主页" 按钮用于返回到 shell 并激发系统操作。Bloom/System gesture or Home: The HoloLens bloom/system gesture or a controller's Home button is used to return to the shell and to fire system actions.

在 HoloLens 2 中,双手具有丰富的表示形式。Hands have a rich representation in HoloLens 2. 除了被跟踪/未跟踪,并可用于驾驶手势外,现在,我们还提供了一个可供用户使用的有表述的主干模型,并向开发人员公开。In addition to being tracked/not tracked and usable for driving gestures, hands now have an articulated skeleton model fit to them and exposed to the developer. 骨架模型每个都有26个跟踪点。The skeleton model has 26 tracked points on each hand.

  • 接点:在三维空间中具有关联点的给定跟踪位置的20个跟踪位置之一。Joint: One of 20 tracked positions for a given tracked hand with an associated point in 3d space.
  • 姿势:所有接头的完整集合,共26个联接。Pose: A complete collection of all of the Joints in a tracked hand, 26 Joints in all.

当前不会通过模拟器公开各个接点位置的直接控制,但你可以通过模拟 API 来设置它们。We don't currently expose direct control of individual joint positions through the emulator, but you can set them through the simulation API. 我们有一组有用的代表,模拟器使你可以在之间切换。We have a set of useful representative poses that the emulator allows you to toggle between.

你还可以控制模拟传感器输入的状态:You can also control the state of simulated sensor input:

  • Reset:返回所有模拟传感器的默认值。Reset: Returns all simulated sensors to their default values. 从 HoloLens 2 模拟器开始,重置的范围可以是一次或两次。Starting with the HoloLens 2 Emulator, a reset can be scoped to one or both hands. 使用修改键 (s) 使用修改键 (s) 或按钮 () (左和/或右 Alt,或游戏板) 上的左和/或右缓冲器。Engage the desired hand(s) using the modifier key(s) or button(s) (Left and/or Right Alt, or the left and/or right bumper on the gamepad).
  • 跟踪:循环浏览位置跟踪模式,其中包括:Tracking: Cycles through the positional tracking modes, including:
    • 默认值: OS 根据系统发出的请求选择最佳跟踪模式。Default: The OS chooses the best tracking mode based upon the requests made of the system.
    • 方向:强制仅限方向跟踪,无论系统请求。Orientation: Forces Orientation-only tracking, no matter the system requests.
    • 位置:强制进行位置跟踪,不管系统请求。Positional: Forces Positional tracking, no matter the system requests.

输入类型Types of input

下表显示了每种类型的输入如何映射到键盘、鼠标和 Xbox 控制器。The following table shows how each type of input maps to the keyboard, mouse, and Xbox controller. 每种类型都有不同的映射,具体取决于输入控件的模式。Each type has a different mapping depending on the input control mode. 您可以在本文档的后面部分找到有关输入控制模式的详细信息。You can find more information on input control modes later in this document.

输入Input 键盘Keyboard 鼠标Mouse Xbox 控制器Xbox controller
YawYaw 左/右箭头Left / right arrows 向左/向右拖动Drag Left / Right 向右控制杆左/右Right thumbstick left / right
音调Pitch 向上/向下箭头Up / down arrows 向上/向下拖动Drag up / down 右操纵杆向上/向下Right thumbstick up / down
RollRoll Q/EQ / E DPad 左/右DPad left / right
XX A/DA / D 左操纵杆向左/向右Left thumbstick left / right
YY Page up/page downPage up / page down DPadDPad up / down
ZZ W/SW / S 向左/向下移动操纵杆Left thumbstick up / down
操作Action Enter 或 spaceEnter or space 向右按钮Right button 按钮或触发器A button or either trigger
布隆/系统Bloom/System F2 或 Windows 键F2 or Windows key B 按钮B button
控制器手柄按钮/手型抓住Controller grip button / Hand grasp GG
控制器菜单按钮Controller menu button MM
控制器触摸板触控Controller touchpad touch UU
控制器触摸板按压Controller touchpad press PP
控制器操纵杆按下Controller thumbstick press KK
左控制器跟踪状态Left controller tracking state F9F9
右控制器跟踪状态Right controller tracking state F10F10
手动 "闭合" 姿势Hand 'Close' Pose 77
"打开" (默认) Hand 'Open' Pose (default) 88
手型点姿势Hand 'Point' Pose 99
手型姿势Hand 'Pinch' Pose 00
重置Reset 转义键Escape key “开始”按钮Start button
跟踪Tracking T 或 F3T or F3 X 按钮X button

注意:控制器按钮可以针对一个手型/控制器或使用手形目标修饰符。Note: The controller buttons can be targeted to one hand/controller or the other using the hand targeting modifiers.

目标设定Targeting

以上一些输入概念彼此独立。Some of the above input concepts stand on their own. "操作"、"布隆"/"系统"、"重置" 和 "跟踪" 是完整的概念、不需要和不受的任何其他修饰符的影响。Action, Bloom/System, Reset, and Tracking are complete concepts, don't need, and are not affected by, any additional modifiers for targeting. 其余概念可以应用于多个目标之一。The remaining concepts can be applied to one of multiple targets. 我们引入了一些方法,用于指定应将命令应用于哪个目标目标。We have introduced ways for you to specify which intended target your command should be applied to. 在所有情况下,都可以通过 UI 或通过键盘按指定的对象来指定。In all cases, it's possible to specify through the UI or through keyboard presses, which object to target. 在某些情况下,也可以直接与 xbox 控制器一起指定。In some cases, it's also possible to specify with the xbox controller directly.

下表描述了针对的选项,以及每个选项的激活方式。The following table describes the options for targeting, and the way to activate each of them.

对象Object 键盘修饰符Keyboard Modifier 控制器修饰符Controller Modifier 模拟器 UI 修饰符Emulator UI Modifier
正文Body (默认值)(default) (默认值)(default) (默认值)(default)
HeadHead 保持 HHold H (不可用)(Not available) (不可用)(Not available)
左侧/控制器Left Hand/Controller 按住左 Alt 按钮Hold Left Alt button 保持左侧肩按钮Hold Left Shoulder Button Left-Hand 图钉Left-Hand pushpin
右手/控制器Right Hand/Controller 按住右 Alt 按钮Hold Right Alt Button 保留适当的肩按钮Hold Right Shoulder Button Right-Hand 图钉Right-Hand pushpin
泄漏Eyes 保持 YHold Y (不可用)(Not available) 眼睛图钉Eyes Pushpin

下表显示了每个目标修饰符如何映射每个核心移动输入概念The following table shows how each target modifier maps each of the core movement input concepts

输入Input 默认 (正文) Default (Body) 手动/控制器 (按住 Alt、按住游戏板肩按钮或切换 UI 图钉) Hand/controller (Hold Alt, hold gamepad shoulder button, or toggle UI pushpin) Head (保持 H) Head (Hold H) 眼睛 (保持 Y 或切换 UI 图钉) Eyes (Hold Y or toggle UI pushpin)
YawYaw 向左/向右翻转正文Turn body left / right 向左或向右移动Move hand left / right 向左/右旋转Turn head left / right 眼睛向左或向右看Eye gaze looks left/right
音调Pitch 开启/关闭头Turn head up / down 上移/下移Move hand up / down 开启/关闭头Turn head Up / down 眼睛查看/关闭Eye gaze looks up/down
RollRoll 向左/右滚动Roll head left / right 向左/右滚动Roll head left / right 不 (操作) (No action)
XX 向左/向右滑动Slide body left / right 向左/向右移动右手/控制器Move hand/controller left / right 向左/右旋转Turn head left / right 不 (操作) (No Action)
YY 上移/下移正文Move body up / down 向上/向下移动手动/控制器Move hand/controller up / down 开启/关闭头Turn head up / down 不 (操作) (No Action)
ZZ 向后移动正文Move body forward / backward 向前/向后移动手动/控制器Move hand/controller forward / backward 开启/关闭头Turn head up / down 不 (操作) (No Action)

控制应用Controlling an app

建议为日常使用以下一组控件:The following set of controls is suggested for day-to-day use:

OperationOperation 键盘和鼠标Keyboard and mouse 控制器Controller
正文 XBody X A/DA / D 左操纵杆向左/向右Left thumbstick left / right
正文 YBody Y Page up/page downPage up / page down DPadDPad up / down
正文 ZBody Z W/SW / S 向左/向下移动操纵杆Left thumbstick up / down
正文偏航Body Yaw 向左/向右拖动鼠标Drag mouse left / right 向右控制杆左/右Right thumbstick left / right
头偏航Head Yaw H + 向左/向右拖动鼠标H + drag mouse left / right H (键盘) + 向右控制杆向左/向右H (on Keyboard) + right thumbstick left / right
打印头间距Head Pitch 向上/向下拖动鼠标Drag mouse up / down 右操纵杆向上/向下Right thumbstick up / down
打印头Head Roll Q/EQ / E DPad 左/右DPad left / right
手动/控制器 XHand/Controller X Alt + A/DAlt + A / D 肩 + 左操纵杆向左/右Shoulder + left thumbstick left / right
手动/控制器 YHand/Controller Y Alt + Page up/page downAlt + Page up / page down 肩 + DPad 向上/向下Shoulder + DPad up / down
手动/控制器 ZHand/Controller Z Alt + W/SAlt + W / S 肩 + 左操纵杆向上/向下Shoulder + Left thumbstick up / down
手动/控制器偏航Hand/Controller Yaw Alt + 向左/向右拖动鼠标Alt + drag mouse left / right 肩 + 向右操纵杆左/右Shoulder + right thumbstick left / right
手动/控制器间距Hand/Controller Pitch Alt + 向上/向下拖动鼠标Alt + drag mouse up / down 肩 + 向右操纵杆向上/向下Shoulder + right thumbstick up / down
手动/控制器滚动Hand/Controller Roll Alt + Q/EAlt + Q / E 肩 + DPad 左/右Shoulder + DPad left / right
操作Action 鼠标右键Right mouse button 触发器Trigger
布隆/System/HomeBloom / System / Home F2 或 Windows 键F2 or Windows key B 按钮B button
重置Reset EscapeEscape “开始”按钮Start button
跟踪Tracking TT X 按钮X button
滚动Scrolling Alt + 鼠标右键 + 上箭头/下箭头Alt + right mouse button + drag mouse up / down 肩 + 触发器 + 右操纵杆向上/向下Shoulder + trigger + right thumbstick up / down
更快移动/旋转Move/rotate faster 向左或向右 Shift 键Left or right Shift key 按住右操纵杆Press and hold the right thumbstick
移动/旋转慢Move/rotate slow 左或右 Ctrl 键Left or right Ctrl key 按住左操纵杆Press and hold the left thumbstick

将 Windows Mixed Reality 沉浸式头戴显示设备和运动控制器与 HoloLens 2 仿真器配合使用Using a Windows Mixed Reality immersive headset and motion controllers with the HoloLens 2 Emulator

使用带有 HoloLens 2 模拟器的 Windows Mixed Reality 沉浸式耳机时,移动和旋转会自动映射到耳机移动和旋转。When using a Windows Mixed Reality immersive headset with the HoloLens 2 Emulator, movement and rotation are automatically mapped to headset movement and rotation. 运动控制器的位置和方向会自动映射到模拟器中的位置和方向。Motion controller position and orientation are automatically mapped to hand position and orientation in the emulator. 下表列出了使用运动控制器时可用的其他操作。The table below lists additional actions available when using a motion controller.

备注

使用耳机时,会自动忽略标准键盘、鼠标和游戏板控件。When using a headset, standard keyboard, mouse, and gamepad controls are automatically ignored.

OperationOperation 操作Action 备注Notes
正文 XBody X 向左/向右操纵杆Thumbstick Left / Right
正文 ZBody Z 操纵杆前进/后退Thumbstick Forward / Back
正文 YBody Y 键盘 Page Up/DownKeyboard Page Up /Down 确保 Windows Mixed Reality 有焦点。Ensure that Windows Mixed Reality has focus. 如果焦点位于 Windows 桌面上以将焦点返回到 Windows Mixed Reality,请按 Win + Y。Press Win+Y if focus is on the Windows Desktop to return focus to Windows Mixed Reality.
向左/向右查找眼睛Eyes Look Left / Right DPad 左/右DPad Left / Right
眼睛查找/关闭Eyes Look Up / Down DPadDPad Up / Down
点击Tap 触发器Trigger
挤压/抓住Pinch/Grasp 手柄按钮Grip button
系统手势System Gesture “菜单”按钮Menu button
重置位置Reset Position 操纵杆单击Thumbstick click

感知模拟控制面板键盘快捷方式Perception Simulation Control Panel keyboard shortcuts

可以使用以下键盘快捷方式访问 "感知模拟" 控制面板,并启用或禁用 PC 输入设备。You can access the Perception Simulation Control panel and enable or disable PC input devices with the following keyboard shortcuts.

OperationOperation 快捷键Shortcut 说明/注释Description/Notes
切换 "使用键盘进行模拟"Toggle 'Use keyboard for simulation' F4F4 关闭时,键盘输入将进入 HoloLens 或 Windows Mixed Reality 应用程序。When turned off, keyboard input goes to the HoloLens or Windows Mixed Reality application.
切换 "使用鼠标进行模拟"Toggle 'Use mouse for simulation' F5F5 关闭后,鼠标输入会转到混合现实环境 (Windows Mixed Reality 仅) When turned off, mouse input goes to the Mixed Reality environment (Windows Mixed Reality only)
切换 "将游戏板用于模拟"Toggle 'Use gamepad for simulation' F6F6 如果关闭,则模拟将忽略游戏板输入When turned off, gamepad input is ignored by simulation
显示或隐藏控制面板Show or hide the control panel F7F7
将键盘焦点设置到控制面板Set keyboard focus to the control panel F8F8 如果面板当前不可见,将首先显示该面板。If the panel is not currently visible, it will be shown first.
在模拟器或混合现实门户窗口中停靠或取消停靠面板Dock or undock the panel to/from the emulator or Mixed Reality Portal window F9F9 如果在取消停靠时关闭窗口,则停靠并隐藏该窗口。If the window is closed when undocked, it's docked and hidden.

另请参阅See also