You probably have heard about Leap Motion’s Project North Star, that should be able to offer people affordable augmented reality. Anyway, there’s still a lot of confusion and mistery about this device. There are some typical questions that arise in the mind of the AR enthusiasts of the world: how is it? Is it good? Is it easy to assemble one? Where is it possible to buy one?
Let me try to answer some of these questions thanks to a special person that I met in Shenzhen…
What is Project North Star?
Project North Star is an opensource augmented reality headset that Leap Motion has designed and gifted to the community. The idea of Leap Motion is going against the current trend of augmented reality glasses, that is mostly based on expensive devices that have closed ecosystems (e.g. HoloLens or Magic Leap One), but to create a headset that is:
- Affordable;
- Completely open, both from a software and hardware standpoint.
For this purpose, it has created a reference design for an AR headset dubbed North Star, giving full instructions on how everyone can build one at home. People can download all the instruction files and access the GitHub repo from the official page and then build their own headset.
The device also exploits Leap Motion hands tracking and offers a completely natural interface all based on hands interactions. Videos about the experimental UX Leap Motion is developing have become pretty viral, because it is damn cool.
Reference Design Issues
The project is pretty amazing, but it has some flaws, like for instance:
- Leap Motion is not offering the device already mounted. So to have it, you have to buy the pieces and assemble it yourself. And while I love assembling hardware, personally, I also think
- There is not even an assembly kit like with Google Cardboards. You must buy yourself the various components. There’s no help in doing this. And some of the parts (as I will detail later on), are complicated to be found or expensive;
- Since it is so difficult to have a device, there are almost no developers. And being no developers, there are almost no apps. This means that the ecosystem is currently only a little niche. And it is very complicated to jumpstart.
Probably Leap Motion had this great idea for an affordable AR headset with natural interactions that could democratize augmented reality, but didn’t have the means (money, etc…) to put it on the market and make it succeed, so it gifted the project to the community hoping that the community could help it in growing.
But who could take this big bargain?
Meet Noah Zerkin
When I went to the Vive Ecosystem Conference to launch our game “HitMotion: Reloaded”, I got in contact via Twitter with a guy called Noah Zerkin, that works in Shenzhen. If you have already heard his name, it is probably because he is the author of this famous photo of Sergey Brin wearing Google Glass in the metro
Or because he’s been the first person attaching a Leap Motion to an Oculus DK1 with some duct tape, envisioning how hand tracking could be vital for virtual reality.
He invited me to his offices inside the company Noalabs, and I went there with the other innovator Enea Le Fons. Since from the first instants we met him, we discovered how he is a very kind and nice guy, that is incredibly passioned about new technologies. He made us see all the stuff he was experimenting on, going so deeply in technical details, that some times I found myself nodding my head without having a clue of what he was saying. He’s a fucking genius, he knows a lot of stuff about hardware and creates a lot of innovative stuff.
But it was after some minutes, when he told us that he was working on Project North Star and that he would have left us try one, that we went in full awe.
Noah and Project North Star
Noah loves the philosophy of North Star and really believes in an open augmented reality ecosystem. That’s why he put all his efforts in trying to create an actual North Star headset, transforming it from an abstract reference design to a real product, ready for the market.
And I can tell you that he is doing that pretty well. If you look at his twitter feed, you can see various videos of him assembling or using the device.
Here you are a video of him using the North Star (modified by him so that to be portable):
And here there is a series of two videos where he assembles a full kit from zero:
The office I was in was a complete laboratory, where he was 3D printing all the parts of the glass, weilding pieces together and experimenting with different configurations. He’s thinking about manufacturing North Star glasses both from a technical and a logistic standpoint.
Of course, he is in touch with Leap Motion and the other people of the North Star community, and they are helping each other to make this project to succeed.
How he implemented the project
While I was talking with him, he explained to me that there are various difficulties in building a North Star headset, and these are actually preventing the AR community from adopting this device. The biggest one is that some parts are difficult to be found.
The main issue is with lenses: Leap Motion advises the use of some special lenses that are hard to be found and cost around 400$. This is surely not the price that a VR enthusiast can accept lightly.
Then there are the displays: the required screens must be bought from manufacturers that usually produce only hardware in big batches and you can’t just buy two of them.
And even if you buy all the pieces, you have still to assemble them overcoming all the hardware and software technical problems. Assembling an AR glass is not an easy task.
Since Noah is in Shenzhen, he is in a favorable position to create some hardware. 90% of the worldwide electronics are produced in the big area that goes from Shenzhen to Taiwan and Hong Kong. Noah has so been able to find some local partners that could help him with lenses and displays.
For the lenses, he has been able to buy from a manufacturer a type of lenses that is not perfect like the one advised by Leap Motion, but that gives acceptable results. He explained me the differences, but it was one of those moments where I nodded my head without fully understanding…. but basically, instead of using lenses with certain diffractive properties, he uses some other cheaper lenses that are coated with some reflective material that emulates the properties of the first type. The price is one-tenth of the original one, and according to him, it could become one-hundredth if they were produced at scale.
For the display, he followed a similar approach and got an agreement with a manufacturer. It was not easy, because usually display manufacturers sell only in big quantities… but he managed to get an agreement with one for a big amount, but not an enormous amount of pieces (e.g. 500 instead of the usual 10,000), for a special type of little custom displays with the right framerate, able to be installed inside the glasses.
Then he had also to try various display drivers boards until he found a valid one that was somewhat compatible with the special displays he found. He had to modify the firmware of these pieces of hardware, because the original one was not working perfectly well.
After all of these technical difficulties, he had also to study how to 3D print the frame of the North Star and how to assemble it.
He also solved two big problems of the North Star:
- Portability. North Star is a tethered headset, and for this, it works attached to a PC. But tethered AR is almost useless, since AR is a contextual technology, so he found a little computer that you can attach to your waist and connect the North Star to it, so that you can move wherever you want. The mini PC works thanks to a battery, so you can use the device in full mobility;
- 6 DOF Tracking. North Star has no positional tracking. So he mounted an Occipital Structure sensor on top of it. Structure sensor is used for SLAM, so it can track your position in space, giving room-scale super-powers to North Star. This additional hardware, not considered in the official specifications, of course required various further adjustments, starting from the plastic mount of the headset that had to be modified so that it could also accomodate a Structure sensor and not only a Leap Motion one.
You can imagine how all this process required him a big investment of time, money and stress. And also required big technical skills: I think I wouldn’t have been able to go through all of this, and with me, various other XR enthusiasts. That’s why the work of Noah is being fundamental for the success of the North Star headset.
Anyway, his efforts has not been useless: in the end, he has been able to create a working 6 DOF semi-standalone North Star headset.
Hands-on review
I was happy like a kid in an amusement park when I discovered I could try the North Star. Let me tell you what have been my first impressions.
The headset is of course not super easy to be worn. Especially the computer that I had to attach to my waist was pretty big, and so I can’t tell it was very comfortable. But the PC attached to the pants was just an experiment by Noah and he had not made an exhaustive search about the lightest one, so I can accept that it was quite bulky… especially because it had to power two displays, one Leap Motion and one Structure sensor... so it had to be quite powerful.
To wear the headset, I needed a bit of help, also because I was afraid of possibly breaking it (it was made by 3D printed parts!). Anyway, I noticed that it was actually easy to wear. Like all the other headsets, I had just to put it on my head and then rotate a knob to make it fit around it.
The comfort in general was a bit rough: being used to comfy devices like the Rift CV1, or the Vive Focus Plus, with all their cushions, wearing a device hacked using plastic resulted a bit uncomfortable to me. I had to say it was very light, though. Even if it seems so big, actually the plastic is very little and so the device sits well on the head. Maybe some cushions would help in making it more comfortable on the skin.
North Star provides you the AR magic by showing content on two displays that are installed at the sides of your eyes, whose content is reflected onto the semi-transparent lenses that are in front of the eyes. With this easy trick, on the lenses you are able to see both the real and the virtual world.
The lenses appeared to me less dark than the ones of HoloLens, Magic Leap, or nReal, so the real world appeared quite well illuminated. There was a bit of darkening, of course, but I haven’t found it dramatic. The holograms appeared a bit transparent, like with Magic Leap One, and had not the bright colors of nReal.
The field of view was really nice and Noah told me that it was around 70°, that is incredible for an AR headset. He specified that in the first version of the reference design, it was even bigger, but then it was sacrificed to improve other characteristics of the headset.
What was incredible was the hands tracking: I could wander around Noalabs’s offices, moving my hands, moving my fingers and see some virtual skeleton hands moving together with my hands. Tracking of all the fingers was great, since it is the specialty of Leap Motion, and so I think that this device could be great to experiment with natural UX in AR.
The visuals had a problem: due to the various reflections necessary to have AR, there were some halo reflections on the lenses in front of me, and so I had some sort of a ghost double vision of my virtual hands. Furthermore, the position of the real and virtual hands were not exactly identical. To reduce these drawbacks, you have to calibrate the device and wear it so that to find its sweet spot.
Then Noah told me about the problem of focus: North Star is a fixed-focus headset and while fixed-focus is somewhat acceptable in VR, it is a true nuisance in AR. Basically, if my eyes focused on my virtual hands, I couldn’t see my real hands welll and viceversa. Fixed Focus in AR means that you have to decide if focusing on real or virtual elements and this breaks the magic a lot. This is why having a varifocal headset in AR is fundamental.
In North Star, you can modify the virtual elements’ focus point by moving the displays that render the content. But this has to be done by hands… to have a realtime focus change, you should connect some motors to an eye tracking device and this is not present in this version of the headset.
The Occipital sensor is good to experiment with AR room scale, but Noah told that since it has a 25ms latency, it can’t be efficient for a final implementation of the product.
To develop for the Leap Motion North Star, you can use Unity or Unreal Engine. Since it has to be connected to a Windows PC, you can simply develop for it like you develop every other XR Leap Motion application, and also preview your programs on the fly by just pressing the Play button. Using it in Unity was really easy, from what I have been able to see.
First impressions
I actually loved the North Star. As you can read above, it has some issues, especially for what concerns the non-perfect comfort and the visuals that have some drawbacks. But we are talking about a device completely hacked, not by a product of a big corporate.
The possibility of seeing my hands and being able to use them in a natural way to interact in augmented reality was something that made me very excited. Also because the big field of view let me really see a lot of the virtual elements around me. Being used to HoloLens 1, I found the North Star FOV really intriguing.
Of course, it is not the headset that you can use industrial facilities, but it is a device you can buy to experiment with AR without having to sell a kidney. And it is a device that wants to foster collaboration between people in AR: it is something completely opensource created by the community for the community. It is fully open and hackable, and thanks to it, Noah has been able to hack it adding portability and room-scale tracking. Imagine what it could become if lots of smart people like him are able to mod it adding new features.
Noah has made an enormous work to transform the North Star from just some specs to a real usable product. He says that there is still a lot to do, but he considers this North Star as the DK1 or augmented reality: a cheap and buggy device, that lets you understand the full potential of the technology. I think that it is a perfect comparison.
How can you buy the North Star?
If you are interested in this device, you can download the specifications, and try building one yourself. Otherwise, if you don’t want to mess with all the problems described above, you can contact directly Noah Zerkin. He is working to become a North Star reseller, so that you can buy a headset directly from him, without having to mess with electronic materials that are difficult to be found and 3D printed stuff. He thinks that this way he can also help the Project North Star to take foot and so help opensource AR in becoming widespread.
If you are interested in getting one, write a message to Noah on Twitter, or ask me for an introduction. He is starting selling the first batches, so you could preorder yours!
Let’s all support crazy people that want to make the XR ecosystem a better place 🙂