HoloLens IRL
What it's like in Microsoft's version of augmented reality
Lance Ulanoff
SAN FRANCISCO — I really didn’t get comfortable with the most recent version of the HoloLens until I got comfortable.I’d traveled to San Francisco, California, ostensibly for the Microsoft Build Developers conference, but my real quarry was the company's now-shipping augmented-reality headset, the HoloLens Development Edition. It’s finally shipping to developers who preordered for $3,000. This wasn’t my first experience with HoloLens, but it was the first time I’d get to see the packaging, unbox it myself and use it, unfettered by Microsoft handlers, for hours.Perhaps more importantly, for the first time, I could finally document my experience in photos, videos and even Vines.Before I can explain what I felt, saw and heard, you need to understand what the HoloLens Development Edition is and what it isn’t.
As an augmented-reality headset (or as Microsoft calls it, a self-contained holographic computer), HoloLens is almost the opposite of virtual reality. VR cuts you off from the outside world and replaces it with, often, a fantastic one; HoloLens enhances your world, creating what's sometimes called “mixed reality.”Instead of goggles, the HoloLens Development Edition is a visor. Instead of headphones that cover the ears or, at least, fill the ears with sound, HoloLens’ downward-firing audio speakers bathe your ears in clear 3D sound, but still let you hear the real world around you.
[img src="http://i.blueprint.mashable.com/RRmBhCvIsx9pJ-vY3iwkh4kBYdI=/fit-in/1000x0/blueprint-api-production.s3.amazonaws.com/uploads/card/image/42425/20160329_144657_HoloLens.jpg" caption="You can take any virtual object and, using gesture like Air Tap and pinch and move it anywhere in your real environment. I placed a key on Mashable video producer Kayvan Ghavim." credit="Lance Ulanoff/Mashable" alt=""]
HoloLens also distinguishes itself from virtual reality headsets by being fully-self-contained. It doesn’t connect to another PC to run and doesn’t need a smartphone. HoloLens Development Edition is a full-blown, 32-bit(!) Windows 10 PC (complete with 64GB of flash storage and 2GB of RAM).More notably, the HoloLens Development Edition isn't going after the same piece of enhanced-reality pie as the Oculus Rifts and HTC Vives of the world. This version of the product is for developers and businesses only. The release is designed to excite and engage these communities and to show them the possibilities of mixed reality in the office, on the factory floor, in the laboratory and in the classroom.Obviously, I’m not working in any of those environments and the HoloLens Development Edition, perhaps, isn’t for me, either. That said, Microsoft hasn’t delivered a stiff, boring device. HoloLens Development Edition is full of fun and engaging ways to use it — a fact I quickly discovered.
HoloLens is a Windows 10 device, but it has as much in common with your laptop as a Tesla does with a tricycle. They both can take you places, but one is going to do a much, much better job and provide features and interactions not possible with the three-wheeler.The shipping Development Edition looks just like the HoloLens you can see at Microsoft’s Flagship store in Manhattan. It’s come a long way since I tried out an early version shortly after Microsoft’s surprise HoloLens announcement on January 21, 2015. That device had most of its guts on the outside. By late last fall, the HoloLens design was nearly final, though, in my demo at the time, it still felt a bit uncomfortable.
I noticed immediately that the final 1.3-pound (579 grams) well-balanced HoloLens Development Edition was the most comfortable HoloLens headset yet and, not for nothing, with its smooth matte-black finish, curves and sunglass-like visor, it’s a pretty cool-looking device. Microsoft told me that it’s made dozens of adjustments since the last time I tried it. It showed.Microsoft gave me a brief walk-through on how to wear and use the headset. It’s not complicated: The headset comprises two rings. The outer one houses the visor and all the key technology for your augmented-reality experience (sensors, mics, speakers, camera, lenses, CPU, etc.); it’s also not completely closed. There are four buttons on the top edge (two on each side, one set for volume, the other for screen brightness) and a power button on one side (it sits in the space between the two sides of the open ring).
The second, inner ring is actually an adjustable brace for your noggin. It rotates to a 45-degree angle offset from the visor. It has as dial on the back to tighten and loosen it. The HoloLens also ships with a strap that you can add to the inner ring. It fits over the top of your head. I didn’t need it.I placed the HoloLen Development Edition on my head and reached around the back to tighten it. I made it snug, but not uncomfortable. There’s a nose guard, but the device never really rested my nose. I could also pull the visor forward or push it closer to may face.
An on-screen tutorial walked me through how to first position the visor so I could see all of a blue bounding box suddenly floating in front of me and then how to control the HoloLens Development Edition via gaze, gestures and voice. I had to learn to look at what I wanted to control, which is pretty natural (HoloLens tracks both your head and eye movement) and speak to Microsoft’s voice assistant Cortana when I wanted to control HoloLens via voice.The most important control mode, though, is gesture. The five key gesture controls are Air Touch (explained below), a sort of flourish move (palm up and fingers quickly spreading out as if you’re splashing water into the air), pinching, pushing and pulling.
To do an Air Touch, you raise your hand up in front of your face and point one finger skyward and then tap your finger down. It’s the most common move, and I did it so often I got finger fatigue (thankfully the $3,000 package also includes a physical Bluetooth button that you can use instead of the Air Touch).Since HoloLens' specialized displays are so close to each eye, it’s critical, for an effective 3D effect, that each image is pointed directly at each eyeball. The only way to do this is to measure the pupillary distance of each user. In my previous HoloLens demos, Microsoft had to measure this space with a special device that typically lives in an optometrist's office. Not anymore.Now the system guides you through special measurement process. It asks you to close one eye and then to position one finger within an image floating in front of you (what Microsoft calls "holograms") and hold it still. You have to do this around five times for each eye, but then the HoloLens Development Edition is calibrated for your eyes. I found this to be a little tedious, but it seems to work.
If you own a virtual reality headset, you are usually looking directly at one or more screens. They fill your field of vision. With the HoloLens Development Edition, you're looking through two sets of specialized glass lenses that guide a 3D holographic projection down from inside the device onto the surface of the glass. When you look through the lens, you see what is clearly an HD image, though Microsoft cannot specify the resolution in a way that’s comparable to, say, an HDTV. Instead they count points of light (2.3 million) and “holographic density" (2,500 light points per radian).
[img src="http://i.blueprint.mashable.com/CWlweV3PnFMZ5Lz4CFN7WMNCP7o=/fit-in/1000x0/blueprint-api-production.s3.amazonaws.com/uploads/card/image/42419/20160329_140031_HoloLens.jpg" caption="The hotel room was filled with virtual reality objects, art, apps, games, video screens and web browsers." credit="Lance Ulanoff/Mashable" alt=""]
Much of what I saw through the HoloLens looked almost real enough to touch, but the display quality is only half the experience. On the front of the HoloLens are a pair of cameras that build a comprehensive 3D map of the environment. As a result, everything I saw through HoloLens — and Microsoft had prepopulated the room with at least a half dozen holograms — interacted with the physical world as if they were really there.
[img src="http://i.blueprint.mashable.com/nyuEvfptwAl0I8CW6gqJH1eDhkI=/fit-in/1000x0/blueprint-api-production.s3.amazonaws.com/uploads/card/image/42417/20160329_135950_HoloLens.jpg" caption="I was able to pick up this VR tiger and move it to a table in the center of the room." credit="Lance Ulanoff/Mashable" alt=""]
In one room, I found an Octopus in the bathroom. There was a tiger statue blocking the front door, a 75-inch display on one wall, a realistic-looking Planet Earth rotating near the couch and a toolbox on the ottoman. Some things moved and I moved in response to them, while others were static. I was generally impressed with the device’s performance, though I did notice a few visual glitches where images failed to redraw around objects, both real and virtual, that had been moved.
One of the chief criticisms of Microsoft’s HoloLens has been the rather narrow viewing frame. It’s true, the space for holograms doesn’t fill your entire field of vision. Imagine a 42-inch HD display floating in front of your face and you sort of understand your viewing window. However, that doesn’t tell the whole story.
[img src="http://i.blueprint.mashable.com/Xn5VHxKQbEMBbGsGzhKtZa9zIWw=/fit-in/1000x0/blueprint-api-production.s3.amazonaws.com/uploads/card/image/42421/20160329_152537_HoloLens.jpg" caption="This image gives you a good idea of how the HoloLens Development Edition bathes the room in a 3D mesh ." credit="Lance Ulanoff/Mashable" alt=""]
Since the viewport follows your gaze, that window shows you more of the VR world in real-time. It can be a little frustrating to be up close with a VR engagement that’s larger than the window, but a step or two back puts it all in perspective. For example, when I played the VR detective game Fragments, I encountered a character who was nearly 6 feet tall. He told me to come close and watch as fragments at his feet resolved into bits of memories. I did as I was told, but found that I could only see his head and shoulders and needed to look down at the ground to see the shards turn into a virtual rat that ran over my feet. I had to step back a bit for a better experience.In another demo, HoloTours, I used the real-looking globe to travel to Peru. First a Peruvian artifact appeared at my feat. A moment later, my entire surroundings turned into a Peruvian landscape. The ground at my feet transformed from a gray, industrial rug into beige dirt and gravel. Off to my right were tourists and above was a mountain top. I was there, and yet still firmly grounded in the hotel room. That’s the beauty of mixed reality, even with a relatively small viewing frame.
Throughout my demo, I practiced using gestures and voice to control the HoloLens. I got pretty good at grabbing things, moving them around, bringing up menus — one of them looks a lot like the Windows Start menu — and even making the system record mixed-reality videos and take photos of my experience (this task is handled by a 2-megapixel/1080p video camera that sits dead center on the visor). I also practiced with Cortana. She was pretty finicky. The device is equipped with four microphones, but even though I was in a fairly quiet room, Cortana often did not understand me.I played the game Robo Raid (as I did in my last demo experience) and had just as much fun shooting aliens as they broke through the hotel room walls, though I wouldn’t suggest you do the same with your windows open. At one point, aliens were streaming in through a virtual portal in the window, but I could barely see them because of all the light streaming in.
[img src="http://i.blueprint.mashable.com/_nGqxAe0oSLrv5v0OJZwFguI4QE=/fit-in/1000x0/blueprint-api-production.s3.amazonaws.com/uploads/card/image/42436/20160329_141914_HoloLens.jpg" caption="A left a cool piece of VR art is affixed to the wall. At right, Microsoft placed a virtual video screen on top of the real TV. I later moved that screen. " credit="Lance Ulanoff/Mashable" alt=""]
I watched TV and opened various Microsoft Edge browser windows. It’s not clear if I can install other browsers on HoloLens, though this is a PC, and I do know that I can add PC peripherals like a Bluetooth keyboard, something I’d recommend for would-be HoloLens customers since the act of air-tapping URLs one letter at a time is not fun (you can use voice, too).
Microsoft is not trying to attract consumers and the price, $3,000, will certainly limit its appeal. However, after a couple of hours using it, I wasn’t certain I was getting it. Then I sat down on the couch.I’d been wearing and using the HoloLens Development Edition for almost two hours when I stopped walking around the hotel room where Microsoft had set up my free-form demo and, perhaps out of fatigue, plopped myself down on one of the cushy wraparound couches and put my feet up on the table.
When I arrived in the demo room, Microsoft had placed the virtual video screen on a big wall. I found it hard to figure out where to stand to view it. After I sat down on the couch, I looked around (while still wearing HoloLens) and found another virtual screen and, using gestures, resized it and moved it so it floated about 8 feet in front of me.I found that I could pull it closer or push it away with a gesture. Leaning back on the couch with my feet up, I found this viewing experience pretty comfortable. I then launched another Microsoft Edge Window, opened Twitter and placed it floating to my right, also about 8 feet away. Then I added a second browser window on my left, brought up the NCAA site and moved it into position next to the video screen, but angled toward me. For fun, I even added another browser overhead.
Then I just looked around, realizing I had, in a matter of moments, constructed my own virtual world. It wasn’t perfect, but I think I finally understood the power and potential of the HoloLens.After more than two hours, we’d drained the HoloLens almost all the way down. It had warned me at 40%, then again at 4%. Suddenly, as I sat on the couch, it blinked out and a sad “Goodbye” floated in front of me. Then that disappeared, too.Back to reality.