Kinect is dead. R.I.P. awesome VR body tracking sensor

I was sharing yesterday post on Hacker News, when I gave a look to the latest post on that social network. Its title was “Microsoft Kinect is dead” and I had quite a shock. I went through the 5 stages of grief, quickly negating the fact that that one could be the truth: “It is not possible, maybe it is one of those rants against the Kinect, where people say that is a useless sensor”. I opened it and I found the exclusive article on Fast Company that talks about the end of the Microsoft sensor. “Maybe it talks only about the old version, the v1“, “Maybe it talks about the end of the current version in this current form factor”, “Maybe…”, Maybe not. Kinect is dead and there is nothing we can do.

Microsoft has stopped the production of its innovative body-tracking sensor, so it will be possible to buy only the devices that the various stores have in stock. In few months, it will become very hard to find such a device. The reason for this decision? Basically, the one that we all know, the fact that the Kinect project failed a long time ago.

Kinect was born to fight the super-successful Nintendo Wii, offering the ability to have even more natural interactions within console games. But it had some problems:

• It had not the same kind of content as the Wii: games like Wii Sports have made the history of home entertainment (eh… how many hours I’ve spent playing with it at home of my friend Davide and having tons of fun!). With Kinect, apart from Dance Central, I can’t recall any popular game;
• It needed a decent free area space to play inside the room. Not many people have it, especially in Europe;
• It had some problems with infrared interferences, especially with the sun;
• It wasn’t very cheap.

In the end, even Nintendo Wii, one of the most successful consoles of the history of gaming, has almost died… so maybe that one was not the right format for modern consoles. As Fast Company points out, Kinect has actually died because PS4 won the battle against Microsoft Xbox+Kinect, investing on providing a great product at a lower price, with a lot of awesome content and with full resources dedicated to standard gaming. I’ve learned in these years of struggle to make VR succeed, that people don’t care about novelty, about the awesomeness of a tech product. They just want something that works well and is affordable. And the Kinect v2, costing $200, was from cheap. Furthermore, there was no great game dedicated to it. I think that Microsoft made too many business errors with the Kinect device.

In fact, Kinect got an enormous success in everything but console gaming:

• A lot of digital signage stations employ a Kinect to make people interact with the device or to detect people passing by;
• I’ve read about video surveillance stations made with Kinect;
• I’ve seen a VR porn application that streamed the 3d version of Ela Darling, recorded through Kinect;
• At EXPO Milan 2015, there was the “Supermarket of the future”that had been made with something like 200 Kinects;
• As Max always states, Kinect allowed for an incredible number of artistic installations;
• Various early robotics systems used Kinect to implement the vision system of the device;
• Thanks to programs like Brekel Studio and iPi Soft, it was possible to record animations for virtual characters in a cheap way using Kinect. This is a feature that has been used a lot by indie game developers;
• In R&D of companies and inside universities’ research centers, Kinect was used for every application requiring face and body tracking. For instance, prof. Bottino of Polytechnic of Turin, with whom I collaborated, used it in a lot of research projects, like this one

Reason for this success was that Kinect was an incredible sensor, offering depth sensing up to 9m and skeletal tracking up to 4.5m, for only $200 (+$50 to buy a PC adapter). Price may seem not so low (especially because above I’ve said that it was too much!), but for professional usage, it is ridiculously low. Especially because it offered a lot of features. The body tracking implemented there was developed by Microsoft training the random decision forest classifier with something like 35.000 people. Only a big corporation can make something like this. As prof.Bottino always states, “The algorithm that uses Kinect is almost known, but Microsoft has been able to hire the best computer vision researchers to design an ultra-robust algorithm for body tracking. Furthermore, the problem is that very few companies are able to perform such an impressive training of the classifier and obtain such incredible results”.

All these niche usages have guaranteed a longer life to Kinect, that was practically abandoned in 2014. I know it very well: the latest runtime version has been released in September 2014 and this meant to us all that the company was not believing in this project anymore. After that, all the team working on Kinect (with people like Ben Lower and Carmine Sirignano) has been disbanded and people started working on other Microsoft projects, like HoloLens or Cortana. Alex Kipman itself, the inventor of Kinect, has moved on and invented the best AR glass out there, that is HoloLens.

Reading Fast Company article, you’ll learn how Kinect has been the first device allowing the humans to interact with computers in a completely natural way.

There’s something very special about using Microsoft’s Kinect system, something that separates it from every other combination of software and hardware I’ve ever used. Kinect adapts and accommodates the user. I’m not learning it; it’s learning me . . . I’ve never felt that a computer understood me—a flesh-and-bone human—so well.

Its spirit will continue living in all the devices offering some kind of natural human-computer interaction: Siri, iPhone X and, of course, HoloLens. Inside HoloLens it is embedded a new little Kinect, the so-called Kinect v4… with a Kinect v5 that is coming. Its power consumption is now of 1.5W and has a field of view of 120 degrees. Think how things have improved over time: Kinect v2 was big and needed a lot of cables and power boxes to work. v4 is a tiny embedded device. But this is the problem: it is embedded, it can’t be used as a standalone device for custom experiments by research centers. Kinect has gone the same road of Leap Motion. That’s a terrible moment for creators. At least its spirit will continue carrying on inside a new device, that will not only understand humans but humans and the environment surrounding them.

I’ve spent 3 years surrounded by Kinects. Me, Gianni & Beps Engineering tried with Immotionar to offer full body inside virtual reality. Kinect was the ideal device inside virtual reality, because it let the user live the virtual experience with his/her full body, like if he/she was really there and without wearing gloves, trackers or other nerd stuff. Unluckily, we started in 2014 and Kinect died a few weeks later. We found ourselves trapped with a startup idea that employed a cumbersome device that wasn’t updated anymore, with all its tracking issues that were never solved. It was one of the reasons of our failure. We had the same fate of Kinect, in the end.

I still remember the first time when we and Gianni looked in our eyes and decided that virtual reality was awesome but needed something more than just a headset + mouse&keyboard and we had the fantastic idea of using the Kinect to take VR to next level.

I still remember my first experiments with this device inside VR: trying to transform movements from Kinect frame-of-reference to VR one was not easy… and I found a lot of time myself performing a step forward in the real world and perceiving it for instance as a step to the left in the VR world. These things made me feel motion sickness levels that standard people can not even imagine: going out from the lab after all these experiments, my head was completely crazy and I had to take some minutes to adapt again to the rules of the real world. Sometimes I felt like when Peter Griffin forgets how to sit down.

I remember the sensation of feeling really inside the virtual reality with my own body, to walk there and interact without wearing any device. It was freedom, it was immersion.

And the magical moments of the exhibition here in Turin where we showcased our product and got a huge success. I remember when at Global Game Jam 2016 I got to know Max thanks to Kinects and VR. I have lots of memories when I think about Kinects. I’ve shared a very beautiful part of my life with them. That’s why this piece of news has really made me sad. I’ll miss you, dear Kinect.

If you have a research center and you want to substitute your Kinects in the future, my advice is to use Orbbec devices. It is the best alternative to Kinect out there: it is little, wi-fi connected, offers 9m of depth detection and, thanks to Vico VR, there’s even a good full body tracking implemented… even if at the moment it can’t reach Microsoft levels (for the reasons explained above). Vico VR already uses it to perform full body VR.

Anyway, today I cry. Tomorrow I’ll fight. If you want to comment here sending me some hugs, I would be very happy of receiving them!