Sat in a conference room in Microsoft’s Redmond headquarters, I reached out my hand. A glowing blue hummingbird fluttered across the room to hover over my palm, its flickering wings buzzing in my ears. I’d just put on the second-generation HoloLens, and hadn’t been prepared for quite how different the experience was: images were clearer, more responsive and — above all — filled much more of my field of view.
But hovering hummingbirds, as delightful as they may be, are only a minor part of the HoloLens story. What’s much more interesting is thinking about what businesses can do with a wearable, connected, augmented reality device.
Putting HoloLens 2 on
For a device that’s not much lighter than the original HoloLens, when you put on HoloLens 2 it certainly feels lighter. That’s down to a major redesign, which has resulted in a device that’s more balanced and much easier to wear. The battery and an Arm-based computer are now at the back, with sensors and cameras at the front. The result is a device that’s no longer front heavy, with a forehead rest that makes it much more comfortable: as a glasses-wearer I’m no longer afraid that the heavy displays will push my frames into my cheeks or my nose.
SEE: Virtual and augmented reality policy (Tech Pro Research)
In fact, Microsoft has designed HoloLens 2 to work with glasses. There’s more space behind the larger holographic waveguides, and if the display is too hard to see through, you can just tilt it up and away from your eyes without taking the headset off. Those larger waveguides use a new two-layer approach to give you a clearer view of the outside world, while also making 3D images projected through them brighter, with a much greater feeling of solidity.
HoloLens 2 uses MEMS-based scanners to send light into the waveguides, with two separate MEMS arrays — one high-speed, one more akin to familiar screen refresh rates. The light that makes Microsoft’s projected ‘holograms’ (they’re not really holograms, just 3D images rendered using your own stereoscopic vision to give depth) comes from a set of quantum well lasers, and they’re bright enough to see against a sunlit window in a normally-lit office. There’s no need to darken the room you’re working in: HoloLens 2’s displays have more than enough brightness.
The tilting displays are a big part of the HoloLens 2 business story. Microsoft is working with safety gear vendors to integrate HoloLens with protective headgear. If you’re an architect or an engineer on-site, you’ll put on an integrated HoloLens helmet, and tilting the displays makes it possible to meet various safety regulations around the world by giving you unrestricted vision when you’re not using the device.
Exploring the HoloLens 2 software
Getting started with HoloLens 2 is easy. You look at your wrist, and wait for the Windows logo to appear. Tap it, and a start menu floats in the air in front of your eyes. From there you can use settings to customise the display for your eyes, and launch apps with a tap on a virtual icon.
I didn’t spend time in the bundled firstline worker apps; instead, I explored a toolkit that demonstrated various user interface models and interaction modes. One of the biggest changes in how HoloLens 2 works is a much more accurate set of sensors, with built-in eye tracking and the same depth sensor Microsoft developed for its Azure Kinect machine-learning-powered 3D camera. Instead of simple glance pointers and ‘air tap’ selection, the new HoloLens sensor suite can map your hands, with 25 points of articulation tracked.
Developers can use this to add virtual keyboards and interactive buttons to applications, pulling through windows to scroll content and grabbing handles on the edges of windows and around 3D objects to scale and rotate. With more ways to manipulate objects, working with HoloLens starts to feel more natural. You can pick things up, push things, pull them. If you can do it in the physical world, the physics engine used by HoloLens 2 means you can do much the same in the virtual spaces of mixed reality. And depending on the code behind the mixed reality objects, you can do a lot more — dropping models all the way to the center of the earth or throwing something so far it vanishes outside your mixed reality boundaries.
Building code for HoloLens 2
Making HoloLens 2 makes semse as a tool that extends your existing 3D models; Microsoft’s mixed reality world is one that overlays information and images on the physical world. The Kinect camera in HoloLens 2 gives the device a map of the world that can be used to accurately place objects, so you can mix a building under construction with its drawings to see any differences between the plan and reality. Similarly, in a common HoloLens demo its cameras can show a support engineer exactly what a field engineer sees, so they can collaborate on repairs and installations, the support engineer using pens to annotate the world.
There’s another aspect to the HoloLens 2 story that often gets missed: its integration with Azure. Apps can use Azure compute resources to pre-render images for download. HoloLens 2’s Snapdragon 850 chipset may have more horsepower than the Intel original, but to get the highest fidelity it’s a good idea to have images ready to go, downloaded over wi-fi as you need them.
Similarly its support for Azure’s Spatial Anchors means you can preload models to specific locations, along with navigation cues. Engineers working on-site can have their models loaded automatically when they reach the equipment they’re inspecting, and when done, they can be guided to their next stop. Spatial Anchors’ cross-platform options can support a new class of augmented reality application — using a phone or tablet as a window into HoloLens’ mixed reality to allow anyone to join a mixed-reality application via tools like Teams or SharePoint. A demo at BUILD 2019 showed local and remote HoloLens users exploring a 3D model of a toy, with a tablet user interacting with the same model at the same time.
Tools like Teams have shown Microsoft’s commitment to taking its applications beyond its traditional knowledge-worker customer base. HoloLens 2 is another step on that journey, providing workers in the field with a mixed reality platform that goes a long way beyond phone- or tablet-based augmented reality, while still working with those alternative technologies.
So put on your headset, raise your wrist, tap the Windows logo, and get to work.