See, touch and explore the future of interaction

Microsoft Research is exploring the future of interaction, including direct 3D interactions with augmented reality.

Let’s take a look at the technology that is already out there:

  • Sensors. Today’s mobile devices have sensors that make Star Trek’s Tricorder look medieval. For example, a typical smart phone has a compass, accelerometers, gyros, GPS and cameras. Combined with a screen, you can already experience augmented reality. For example, the SLARToolkit for Windows Phone and this YouTube playlist.
  • Gestures. Kinect gives computers the ability to ‘see’ gestures and track movement.
  • Pico projectors. You can already buy tiny, handheld ‘pico projectors’ such as the HP Notebook Projection Companion and they’re even built into some cameras now. If they were built into phones or laptops, they could project a 3D environment into the physical world.
  • 3D graphics. Computers and even phones have the ability to display convincing 3D graphics and, thanks to KinectFusion, it’s also possible to capture 3D environments from the real world in real time. Checkout this video from Microsoft Research to see a computer build up an awareness of its entire environment and Ben Kacrya’s TED talk about long-range 3D scanning.

Each of these technologies gives developers new capabilities and lets them offer users new experiences. The challenge is to figure out what to do with them. This is where Microsoft Research is very active, combining different elements to create new kinds of interactivity, including:

  • Holodesk is an experimental system developed by Microsoft Research that brings many of these technologies together to create something very similar to Star Trek’s holodeck. Mixing multiple sensors, physical objects, 3D graphics and intuitive gesture recognition, it lets a user interact with virtual objects almost as if they were real. See the great video about Holodesk here.
  • Wearable Multitouch Interaction, which is an experimental system that lets people use their hands, arms and legs as graphical interactive surfaces – as the touchscreen of a phone, for example – and interact with them using touch gestures, as in this Microsoft Research video.

Like all research projects, these demos are pre-production versions of soon-to-be-released commercial products. What they do is point to new possibilities and capabilities. In this light, Microsoft has put together a vision of the future that extrapolates the latest touch, gesture and augmented reality technology - check out the video here.

To find out more about how Microsoft technology is supporting innovation in the Enterprise

Posted by Howard
Content and Communities Manager, Microsoft Enterprise Team