HoloLens problems show us that AR is still not ready for the mass market

HoloLens is maybe the best AR glass on the market nowadays and I had the pleasure to try it some months ago and to write an enthusiastic review about it. I was delighted by its capabilities, potentials and the incredible spatial positioning of its “holograms” (hate this term, since they’re not holograms, just virtual objects you see through a lens, but… whatever).

In the last month I had the pleasure to work in a HoloLens project, so I spent some days developing for it using Unity. It’s great, but as within a long term relationships living with your partner you discover a lot of hidden problems… working with HoloLens I started discovering lots of defects it has.

This article is about them. I still think that HoloLens is awesome, but I’m writing this to tell you that it’s awesome, but it’s the future of computing and not the present. Reasons are:

• FOV is terrible. This is a well known issue of this Microsoft device, but working with it you realise how it is really terrible. You have just a smartphone-sized window in front of your eyes through which you see the holograms and nothing more. This breaks every kind of immersion or presence of whatever you play. This is especially true for big nearby objects. You can see only a portion of them and that’s terrible. Even watching 360 videos is bad, since you see only a little part of them. If you feel bad about the FOV of VR headset, don’t try the HoloLens then!
• FOV is terrible, really. Whatever application you can think about, it’s broken by this. I mean, we were thinking about a medical application X, then we tried simulating it in our lab and we discovered that in that situation the user’s eyes are naturally pointing towards the floor, i.e. looking at a portion of the glass where there are projected no holograms. I mean, you can create only applications where the user’s eyes are naturally aiming at the portion of the screen where the AR happens. All other applications are unfeasable at the moment. So, if you’re thinking about making an AR application because you looked at some Youtube videos on HoloLens, contact a HoloLens expert and ask him if this can be realized with current technology. Consider that this magic portion of the glass is not exactly centered on the glass, but it’s a bit moved on the upper part of it. So, all application where eyes are pointing low are a big “nope” at the moment.

• FOV is terrible, but camera near plane can be even worse. Microsoft suggests you to put HoloLens camera near plane in Unity to 0.85f. If you’re not a dev, this means that the game engine has to cut out everything that is closer to your eyes than 85cm! We tried to break this rule, but putting a value below 40cm makes your eyes cross when a hologram appears too close to your eyes. So, again, you can’t do applications where you interact with objects too close to your face: either you don’t see them, or you cross your eyes to see them.
• Comfort is discrete. To be a complete computer on your head, HoloLens is really light and comfortable. But if you start using it too much, you start feeling its weight. And in my case, I feel also some discomfort near the nose, so after some time I have to remove it from my head.

• Eyes gets dry. Having this lighting stuff happening near your face causes eye strain and your eyes may get dry. Drink a lot and remove the headset when you don’t need it.
• Cortana pops out randomly. As all the vocal assistants, Cortana sometimes believes that you’ve called her… and answering her to f*ck herself doesn’t help 😀
• Spatial mapping is good, but not optimal. Remember all those videos where you make a perfect scan of your room using HoloLens? Well, they’re just a dream. HoloLens does a great job in reconstructing the mesh of the environment, but it’s absolutely not perfect: some objects are perceived so-so, some room corners are detected as holes and so on. Things get worse over time: HoloLens recognizes the room you’re in (even after days you haven’t entered the room!) and so tries to update constantly the model of the room using new data. Don’t know how this happens, but this process needs improvement: over time the room model starts growing in dimension (one room arrived at 200Mb of model for us) and the new detected data starts to have very poor weight, so that the already existing model doesn’t get updated well, even if it is wrong. The result is a flawed model that you have to delete from Control Panel to have it work again. Furthermore, the “plane recognition” feature, that should detect the tables and floor of your room, has a lot room for improvements.
• Holograms are fixed… maybe. In my first review I was super excited (and surprised) by how much the Holograms stay fixed in the place you put them. That’s true, but there are two problems:
  • if the device starts getting sloppy, the holograms are not so stable anymore;
  • If the holograms are big, you start seeing them tremble. This is a well know issue, that I noticed a lot in the “The floor is lava” game, which simulates that your floor is all covered with hot lava. The lava mesh part that was very far from me was absolutely unstable and trembled a lot.

• Low frame rate. HoloLens is not famous for being a powerful device. It’s not a i7 with GTX-1080 graphic card. So you have to be super-careful about the complexity of the scene you’re rendering. This is especially true if you develop using Unity: someone has told me that you have to use at maximum 20K polygons (whaaat??). For a comparison, on a PC you now run scenes with millions of polygons. With native DX12 development things get better, but using them is a true PITA.
• Speech recognition is very American: as an Italian sometimes I’ve difficulties in making the HoloLens to understand me. Sometimes in the lab I scream “SAVE” for hours and all people starts laughing. I seem like someone that is training a dog, a guy told me. I’ve seen that Japanese people study how to pronounce correctly in american style the HoloLens keywords. So, speak as an american or the device won’t understand you well.
• Mediocre hand detection. Using your fingers is very natural, but keeping your fingers straight all the time and using them to click using the pinch gesture is tiring in the long time. Furthermore, the system detects your hands only if they’re in your field of view, so again the “FOV of finger detection” is low and it’s frustrating: if you have to click you have to have your hand somewhere in front of you and this makes interaction a bit less natural (thanks God there is the clicker to make interaction more natural!). If you want to make a scaling gesture, this starts to become a mess. Ah, and you don’t have full hands-n-fingers tracking, just to be clear.

• Very unstable SDK. The HoloToolkit is fantastic, but it makes continuously breaking changes. This is natural for an SDK that is starting to exist, but this means that you have the patience to stay updated and to update all your projects whenever there is a breaking change.
• Holographic remoting is in its infancy. There is this cool stuff called “Holographic remoting” that means that you can launch play mode in Unity and see the app running on HoloLens, so that you can preview it, exactly as you do with Oculus when you launch the play mode (you see everything in the headset while you modify the scene inside the editor). This happens thanks to wi-fi magic: the app is executed on your PC (let’s all remember that HoloLens is just a Windows 10 device) and then the content is streamed on your headset, while headset position and rotation is streamed towards Unity, to keep everything coherent. This is really an awesome feature, but it’s still buggy: there are really lots of artifacts in the streaming if the wi-fi starts having little problems, then sometimes Unity crashes if it does not find the headset, etc. Furthermore, since the app is executed on your desktop PC, some HoloLens features are disabled (like AnchorStoreTransferBatch… if you try to call Load on it, Unity crashes and makes you say some bad words :D)
• Battery is meh. Battery doesn’t last that much if you make an intensive use of HoloLens. And with Low Battery there are things you can’t do (like updating the device) and Cortana starts yelling at you (f*ck you, seriously!). We tried to use the HoloLens while it was connected via USB to a charging station, but this is not a smart choice, since it has not been conceived to be used this way… and this can make your USB-C charging port to damage itself.
• High price: we already know it: it’s 3000$, while its competitors (like Meta, Epson Moverio, etc…) are in the 700-1000$ price range.
• No warranty: if you buy the developer version for 3000$ and something breaks on your HoloLens, you’re screwed. If you want a warranty, you have to buy the 5000$ version.
• Little documentation, little help. Everything is still poorly documented and if you try to make a HoloLens question on Stack Overflow, you see a thimbleweed rolling on the ground.
• No apps. If Oculus users are concerned about content, HoloLens users are completely cyring. There are almost no apps. The best apps are the one made by Microsoft itself or by its partners, like the amazing game RoboRaid… but they’re a few. If you received a HoloLens for your birthday, you would try all the demos in 1-2 days and then you would put it on your bookshelf taking dust. Buy it only if you’re a developer, an innovator or someone that has necessity of using an app that runs on it. Otherwise, it’s useless for you.

And that’s it… there are some other little issues that I should have said, maybe… but these are the main ones. I want to repeat that I made on purpose an article only on the defects of this device just to be informative, but there are also a lot of positive things about it and I still think that, while in its infancy, this is really a powerful device. I love it.

And that’s all: hope you’ll like and share my article and that you’ll come give me a shout at Immotionar! Have a nice day!