What to expect from next-gen HoloLens

The time has come. After many years of waiting, and even a second iteration of the device scrapped away, Microsoft is finally going to announce the next generation of HoloLens. But, what to expect from it? Well, let me speculate a bit…

When and where

HoloLens vNext will be launched on February, 24th 2019 in Barcelona, Spain, at the Mobile World Congress. The keynote will start at 9 a.m. PT (12 p.m. ET / 6 p.m. CET). On stage, there will be Satya Nadella (CEO of Microsoft), Alex Kipman (the inventor of Kinect and HoloLens) and Julia White (Corporate Vice President).

How to follow the keynote of HoloLens launch?

If you can’t make it to Spain, you can follow the livestream directly on Microsoft website, at this link. Probably Microsoft or some other entity will also live stream it over Youtube, according to Kipman… but I don’t have a link for that yet.

The trailer

Some days ago, Alex Kipman shared on social media this video to rise the hype for the upcoming launch. Not that this was necessary, since we were already incredibly hyped 😀

This is the only official info that we have on the next-gen HoloLens. And as you can see from the video, it is not that much (just some evocative images and sounds). All the rest, that I am going to detail in the rest of the article, are just a bunch of thought speculations. I hope you will enjoy them.

The name

This is the second HoloLens that is going to be released on the market, while actually, it is the third version of the device that has been produced by Microsoft. A report of some years ago said that Microsoft has canceled the second version of the device that it was going to produce because the market for such a device was too little (AR glasses currently occupy only a niche) and there were anyway no credible competitors to fight with a new generation of the device.

So, should the name be HoloLens 2 or HoloLens 3? IMHO, calling it HoloLens 3 would be confusing for people that don’t know about the report mentioned above, that would start questioning where is the number 2. So, my guess is that it will be called HoloLens 2 or HoloLens ’19. At maximum, they will find some fancy name, like HoloLens Quest, to describe it.

Design and comfort

Microsoft has bet on the all-in-one design since the beginning. It avoided adding an external box that the user has to wear on this pants, but included all the required hardware into the HoloLens headset. This has pros and cons (pros: no need to mess with an external box + cable; cons: the glass is heavier) and I think they will continue with this road, so I expect the HoloLens 2 to be another all-in-one systems. I think that in industrial environments, where the HoloLens is having great success, the less the cables and the worn hardware, the better for the workers. The same hold for the military, having to find a place to wear a box in a secure way is not easy.

The comfort of the HoloLens v1 was so-and-so, and I think that in these 3 years, Microsoft has had all the time to improve it. I guess they will steal some ideas from Magic Leap and provide a more comfortable headset (more cushions) and more customization options (more accessories to fit it on your nose, support for prescription lenses, etc…). Again, they aim at enterprise usage, and this means that workers should wear the HoloLens 2 for many hours every day, something that is truly uncomfortable with HL 1. I think they worked really a lot on this and we will be pleased by it.

Regarding the design, I think it will look like bulky standard glasses. When I published the news saying that Microsoft had teased the HoloLens 2 in a video by NASA JPL (later on confirmed to be false), some friends of friends told me that the real HoloLens 2 was actually smaller than that glass shown in the video (that is already smaller than HoloLens 1). So this means that HL 2 is smaller of the glass of the picture that you see above.

Furthermore, a report of some weeks ago told that Facebook has in its lab an AR headset that looks like standard glasses. And at CES 2019, we have seen a Chinese startup called nReal has been able to produce an AR glass that looks more or less like standard glasses. A giant like Microsoft has for sure a similar technology… so I expect the HoloLens 2 to be more or less as big as the Magic Leap One (maybe slightly bigger), but without the external box (so, a bit heavier than Magic Leap One).

FOV

The Field of View of the original Hololens, that is around 30-35°, is really limited and is one of the pain points of the hardware.

In April 2017, Microsoft filed a patent on a new technology that could help them in doubling the current FOV by actually doubling the imaging technology. So, in HoloLens 2, we can expect to have a FOV up to 70°.

Honestly, I think that for some compromises that are necessary for every next-gen hardware (e.g. sacrificing the image quality to increase battery duration), Microsoft won’t be able to reach that maximum and so it will ship a device with around 60-65° FOV, that is anyway bigger than the one of Magic Leap (that has a horizontal FOV of 50°).

View post on imgur.com

Resolution

Here we go in the full speculations realm. According to what I’ve seen at CES with nReal, I think that we will have a full HD 1920×1080 per eye. 4K seems too much for a mobile headset at the moment. But I’m really throwing numbers in this case.

Multi-focal display

I admit that I am in doubt with this… and I don’t know. I think that maybe it won’t be present in this iteration because it can’t still be implemented in a reliable way (see the glitches that there are on ML sometimes).

AR/VR Mixed mode

I think that this iteration of HoloLens will only feature AR/MR and not also VR capabilities. This is because offering both would require compromising the goodness of the AR/MR mode and this is not advisable at this moment.

Processing power

According to a recent rumor, the Hololens 2 will feature Qualcomm Snapdragon 850 chipset, so it will be an ARM device. The Qualcomm 850 is a ratherly powerful chip, considering that it is already used in some tablets and convertibles like the Lenovo Yoga C630.

You can read its full set of specs on the official Qualcomm website, but I will highlight here some of the most important ones:

• CPU Clock Speed: Up to 2.96 GHz
• CPU Cores: Qualcomm® Kryo™ 385 CPU, Octa-core CPU
• CPU Architecture: 64-bit
• Up to 8GB RAM
• Support for Wi-Fi, Bluetooth 5.0, USB 3.1 connectivity
• Image Signal Processor: Dual 14-bit ISPs, Qualcomm Spectra™ 280 image signal processor
• Megapixel Support: Up to 16 MP dual camera, Up to 32 MP single camera
• Video Capture: Up to 4K Ultra HD video capture @ 30FPS
• Maximum On-Device Display Support: 4K Ultra HD, Up to 4K
• GPU Name: Qualcomm® Adreno™ 630 GPU
• API Support: OpenGL® ES 3.2, Vulkan® 1.1, OpenCL™ 2.0 full, DX12
• LTE Peak Download Speed: 1.2 Gbps
• LTE Peak Upload Speed: 150 Mbps

As you can see, we are talking about very interesting specs: it is a chipset that can support a display up to 4K, it can support capturing devices up to 4K and also manage a lot of different connectivities. Just WOW.

With HoloLens v1 there were absurd limitations regarding the maximum allowed polygons in a Unity scene (like 20K polygons), so almost every produced program run sloppy. I expect with this one to switch to at least 100K polygons in every Unity AR game. This will allow creators to do a lot of new kinds of applications.

But in HoloLens 2, there will be even more: Microsoft will continue adding an external chip dubbed HPU (Holographic Processing Units) dedicated to perform the spatial tracking and scene understanding so that to free the main processing unit from this heavy task. The HPU will perform all the tracking IN HARDWARE, so it will be really fast. It is confirmed that the HPU of the HoloLens v2 will be an improvement over the previous one and it will also include an AI coprocessor to handle deep neural networks, so the next-gen HoloLens will also be able to perform very fast artificial intelligence tasks.

Tracking technology

The tracking of the HoloLens v1 was already very stable and I was really impressed by it when I tried it. When you wear Magic Leap, you see that the “holograms” tremble a very little bit, while with HoloLens v1, this doesn’t happen, at least for small objects. I expect that the tracking will work even better, allowing also bigger virtual objects to not tremble.

According to a recent patent of March 2017, the HoloLens v2 may substitute all the 5 cameras used on the v1 (4 RGB + 1 infrared) with only one multi-spectral camera. This would allow for a dramatic reduction in costs and weight, while keeping the same tracking quality. We don’t know if this technology will be implemented in this generation of Hololens or the next one, but I like to think that we’ll see it already in this device.

Regarding the depth sensor, it will be the same of th Kinect v4 (Kinect for Azure), that will be very little, with very little power consumption, and a really improved depth sensing resolution above the previous iteration. You can see the difference in the below GIF

With this sensing difference, for sure the sensor will be able to perform the tracking in a more realiable way (maybe also with low light, thanks to the use of IR) and also understand better the environment.

Scene understanding

The reasonings of the previous paragraph still holds: due to the better hardware, the device will feature better scene understanding, with a far better reconstruction of the mesh of the environment (I expect a quality level at least equal to the one of Magic Leap One).

Apart from this, thanks to the new AI co-processor, I bet that Hololens will be able to detect some objects in the scene, and will also be able to detect the faces of the people surrounding the user (and maybe also the body, but I am not sure about that). Things like that were already possible with HoloLens v1 thanks to Microsoft Cognitive Services, but they required all the computation to happen in the Azure cloud. Now things will be able to be much faster thanks to the embedded AI coprocessor. I can’t wait to create some AR applications that are really aware of the environment around the user.

Controls

Interacting with the HoloLens using the air-tap was a PITA, so I guess that Microsoft will offer two little controllers to let the user interact with the AR apps without stressing his hands and arms.

Hand Tracking

This has practically been confirmed by an official demo by Microsoft, that in 2017 showed on stage the device performing real time hands segmentation (sorry, the video has been removed).

It is years that Microsoft works with full hands tracking and the below video of hands tracking performed with Kinect by Microsoft Research in 2016 proves that. So I guess that we’ll have full hands tracking in HoloLens v2. Or at least gestures tracking at launch, that will be converted to full hands tracking in a future update.

Eye tracking

All the most recent XR hardware feature it, so I guess that HL 2 will have it as well. Eye tracking is fundamental in the enterprise sector: think about the applications in training or the possibility of interacting with the device by workers that are doing something with the hands. HoloLens must have it on board.

Connectivity

This will be one of the most interesting innovations. Apart from the USB, Wi-fi and Bluetooth connectivities, that are surely useful, we’ll finally have LTE connection. Not a 5G one, but a 4G one. See the above specs of the Snapdragon 850 chipset: it has an integrated modem.

This means that finally the HoloLens will be free from being confined to places with Wi-fi networks. It will finally be able to be taken everywhere. Maybe it won’t be ideal to be worn in the streets yet, but this means that workers from all over the world will be able to use it in industrial settings when there won’t be Wi-fi connectivity (e.g. in outside environments). This is pure freedom. It is one of the most disruptive innovations of this HoloLens: maybe it is the first headset performing this wild move.

Audio

In HoloLens 1, audio was integrated, so I expect this new version to have an improved version of both integrated microphone and integrated spatial headphones.

Operating system

For sure HoloLens will run on Windows 10. To be more precise, it will run on Windows Core OS, a new version of Windows 10 that has a more modular approach and will support better features specific of each device it will be run on. Notice that the HoloLens 2 has been delayed a bit because of the delays on the development of Core OS.

UX

The interface of the HoloLens v1 wasn’t full of eye candies like the one of Magic Leap, but it was anyway effective. I think they won’t change that that much, and will continue using that Windows-10 look.

I think that they will instead revolutionize the user interface, in the sense of how the user interacts with the hardware: the air-tap was just obscene, so I guess we will have more use of eye-tracking, controllers and voice interactions. As I have said, the use of alternative technologies will also reduce the arm and hands strain of the user.

Apps ecosystem

I don’t think that Microsoft will follow Magic Leap in its pursuit of offering apps that will appeal to the general consumers. This will still be a device targeted at enterprises, so I think that they will announce new services and solutions that will appeal to companies (e.g. better integrations with the cloud, some collaborative environments, etc…)

Battery duration

I guess we will have something around 3-4 hours of continued use of the device.

Price

That’s the great unknown. If Microsoft wants to target prosumers and enterprise users, the price won’t be that low. But if they want to get rid of Magic Leap, they have to propose a price that is affordable. I bet the price will be in the range $1500-2000, so that to offer a product that is cheaper than Magic Leap, while offering more features, so that to kill the competition. This way, it will also be more expensive than headsets like nReal, while offering more features. It would be coherent.

Final judgement

If Microsoft will deliver the features I have depicted above, I think that HoloLens 2 will take us to Augmented Reality 2, setting a high bar for all the competitors to reach. It could be a disruptive piece of hardware.

But we don’t know yet, we have to wait 4 days to discover it…

---

And that’s it. Of course, all of these are just speculations thought with a grain of salt. I love the speculation game… and you? What are your speculations? How do you think the HoloLens 2 will be? Just let’s talk about it here in the comments or on my social media channels!

(Header image by Microsoft)