All you need to know on Azure Kinect

You all know that I am a big fan of the Kinect. My story in the virtual reality ecosystem is deeply interwoven to the presence of this sensor, since I entered the VR field in 2014 with a startup on full body virtual reality (called Immotionar) that was able to offer the user its full body in VR thanks to the mixing of an array of Kinects and a VR headset. Being able to dance and kick objects in VR without wearing any sensors was really fantastic. We understood that full body VR was the future… and in fact now we can see how many people use a Kinect to dance in VRChat.

The end of my startup was also connected to the fact that the Kinect v2 sensor was left to slowly die by Microsoft without updates. We all wondered why: Kinect was not great as a gaming sensor, but was fantastic as a lab sensor: all the research centers and all the Universities had some Kinect sensors to play with for animation recording, depth sensing, full body tracking, etc… It was the best off-the-shelf sensor available.

Actually, with much joy, we discovered months ago that Microsoft clearly understood the potential of the Kinect: it teased that it was going to release a new Kinect sensor, that was connected to Azure. And yesterday, in the same day of the announcement of the HoloLens, finally this new sensor has been released: it is called Azure Kinect and in this post I am going to tell you everything that we know about it.

Azure Kinect DK

The actual name of the device is Azure Kinect DK, since Microsoft sees it as a developer kit. It is a very small version of the Kinect (it is like the v4 version) and its depth sensor is the same that is contained inside the revolutionary HoloLens 2.

Form factor

Azure Kinect is incredibly small: it occupies only 103 x 39 x 126 mm and weighs only 440g. It is like half of the original Kinect, but the advantage, more than the only size of the sensor is the one of the accompanying cable: Kinect v2 came with a big mess of cables, power boxes, adapters, etc… that were bigger than the sensor itself, while all of this seems to have disappeared in the Kinect Azure.

Sensors

The Azure Kinect comes with the 3 same kind of sensors of the Kinect v2 and v1, plus one (the IMU). It so features:

• A depth camera
• An RGB camera
• An array of microphones
• An IMU sensor

The depth sensor

The depth sensor is still a time-of-flight IR depth camera, exactly as in the Kinect v2 (not as in the Kinect v1, that worked thanks to the projection of IR light patterns). The change is that it is now a 1MP camera, so it allows for finer detection of depth values. It seems to have lost some range capability, though, since while the original Kinect v2 depth sensor could detect stuff up to 9m, this one arrives at less than 6m.

If you look at the above table, you will see “NFOV” and “WFOV”. But what do these terms mean? They mean “Narrow FOV” and “Wide FOV”, because this sensor can work in two different modes: with a narrower field of view and longer range, or with wider field of view and a shorter operating range.

I’m really excited by the resolution of this depth camera, but I’m a bit less excited by the framerate of 30 FPS. To be really great to be used for full body VR, in my opinion based on my experience, it had better to work at 60 FPS.

The RGB Camera

The RGB Camera is a OV12A10 12MP CMOS sensor rolling shutter sensor. We have gone from the Full HD of the Kinect v2 to the 4K of the Azure Kinect.

It is also interesting to notice that this camera is more “standard”, meaning that you can finally use the Kinect Azure as a standard USB camera for your PC (with Kinect this was not possible in the beginning) and that finally you are able to calibrate the camera with OpenCV (I remember having lost a lot of time trying to calibrate the Kinect v2 camera with OpenCV, with no success… halleluja).

The Microphone array

Azure Kinect DK embeds a high quality 7-Microphone circular array that identifies as a standard USB audio class 2.0 device.

The specifications, for the audio experts that are reading this post, are:

• Sensitivity: -22 dBFS (94 dB SPL, 1 kHz)
• Signal to noise ratio > 65 dB

IMU Sensor

The IMU sensor (gyroscope+accelerometer) is the great news of this device, since the original Kinect v1 couldn’t detect any movement and the v2 could only detect movements on one axis. With the Azure Kinect, it will be possible to detect movements on all 3 axes and read them with a frequency of 208 Hz.

Power consumption

All this device consumes only up to 5.9W.

Connectivity

You can choose if connecting the Azure Kinect to your PC (via USB A to USB C cable) or if using it as a standalone sensor connected to the Azure cloud.

This gives the user an incredible freedom of choice: for instance, the Kinect v2 could be very interesting as a surveillance device, but each station required a big Kinect sensor + all its cables and a PC. With the new Kinect, you can just use the sensor and let it talk with the cloud.

The Azure Kinect must be connected to your PC via a USB A to USB C cable. It then can connect to the Azure cloud via the PC.

Once it is connected, it can stream raw data (the frames read by the cameras or the audio detected by the microphones), it can detect the bodies of the people that are in its range or, exploiting the power of Azure, it can also be used to understand speech or to perform complex computer vision tasks like detecting objects or reading written text.

What makes me happy, and will make happy all the other fans of full body VR, is that this new Kinect will continue offering the best-in-class body tracking on the market. I can’t wait for someone to connect it with a VR headset!

Minimum PC requirements

If you plan to connect the Azure Kinect to a PC, be sure to have at least:

• 7th Gen Intel Core i3 Processor (Dual Core 2.4 GHz with HD620 GPU or faster)
• 4GB RAM Memory
• Dedicated USB3 port
• A recent operating system:
  • Windows 10 April 2018 release (x64) or later
  • Linux Ubuntu 18.04 (x64) with OpenGLv4.4 or later GPU drive

It’s interesting that you can connect this new Kinect both to Windows and Linux PCs: Windows will have the priority for what regards the SDK development, but the fact that Linux will have support too is a great news for all the makers’ ecosystem.

Multiple Kinects

Surprise, surprise: the Azure Kinect is meant to be used with other Azure Kinects. Every device features a sync-in and a sync-out port so that you can connect various Azure Kinects together in a daisy-chain so that they will sync their signals and will grab the RGB and depth frames exactly at the same time. This is fundamental for instance for performing 3D object recostruction or for recording volumetric videos: you connect various Azure Kinects together, they grab the frames synchronously and later on you reconstruct the object you were recording.

I would have loved to have had something similar with the Kinect v2 when I used an array of Kinects to perform 360-degrees-full-body-tracking for VR…

Kinect v2 vs Azure Kinect

Do you want a comparison chart between the two devices? Well, here you are

Price and release date

If you are from US or China, you can already preorder the Kinect Azure from Microsoft website for $399. Shipping will start by June 27, 2019. The new SDKs will be available when the hardware will begin shipping.

All of us that live in the rest of the world, can just cry in a corner.

---

And that’s it with this journey inside the new Azure Kinect. Long live to Kinect!

(Header image by Microsoft)