The making of the HoloLens2: How advanced AI built Microsoft’s vision for ubiquitous computing

The first time people don the new HoloLens 2 on their heads, the device automatically gets to know them: It measures everything from the precise shape of their hands to the exact distance between their eyes.

The artificial intelligence research and development that enabled those capabilities “was astonishingly complicated” but essential to making the experience of using the device “instinctual,” said Jamie Shotton, a partner scientist who leads the HoloLens science team in Cambridge, United Kingdom.

“We want you to know how to use HoloLens without having to be taught how to use it,” he said. “We know how to interact with things in the real, physical world: We pick things up, we press buttons, we point to things. We aim, as far as possible, to translate that directly into mixed reality.”

Microsoft today announced the HoloLens2 is now shipping to customers. The sensor-packed holographic computing headset uses AI to displace space and time, creating a mixed reality of people, places and things in order to facilitate one of our most basic human impulses: exchanging knowledge.