HoloLens as a tool for computer vision research

0900 - 1230 September 8th, 2018

ECCV 2018 Conference


  • Marc Pollefeys
  • Shivkumar Swaminathan
  • Johannes Schoenberger
  • Andrew Fitzgibbon


Microsoft HoloLens is the world’s first self-contained, holographic computer, but it’s also a potent computer vision research device. Application code can access audio and video streams and surface meshes, all in a world coordinate space maintained by HoloLens’ highly accurate head-tracking. This tutorial will dive into the new “Research Mode” capability of HoloLens. We will show you how to access the raw head-tracking and depth sensor data streams, and make use of the intrinsics and extrinsics of each stream. We will also be demonstrating recent advances in time of flight depth-sensing technologies in the Kinect for Azure project.

Sample application for viewing Research Mode sensor streams A sample HoloLens application that displays any of the six Research Mode streams in real time.


  • 0900 – Introduction
  • 0930 – HoloLens Research Mode
  • 1030 – Coffee break
  • 1100 – Applications & Demos
  • 1130 – Kinect for Azure Depth Sensor
  • 1200 – Q & A
  • 1230 – Tutorial ends

Attendees to this tutorial will leave with a good sense of how HoloLens can be used for a range of Computer Vision research tasks, and materials to quickly get them started using the device.