Finally, after waiting for it for weeks, I have received the update to Hands Tracking v2 on my Quest. Do you remember that update of some months ago that improved dramatically the performance of hand tracking on Quest? Well… that one. Better late than never, I guess. Since I was very curious to try it and to compare it to the one of my friends at Ultraleap, I decided to give it a go, testing it in various applications, to see how usable it has become. And of course, I am sharing with you my impressions of it. I know I am two months late, but do you still want to hear my opinion on it?
A great improvement
The very first thing I noticed, and that made me realize that I got the update, was that I could make the two hands touch without them disappearing. In my Oculus Home, I was used to see my hands disappearing when one was touching or even slightly occluding the other one. But with the new hands tracking update, this is not happening anymore: I was able to make the two hands touch and still see both of them. This is great because seeing the hands continuously disappearing even at the slightest touch was truly immersion-breaking.
I kept playing with them, and I saw that I could easily put one hand in front of the other, I could make the two hands cross their fingers, and see the tracking still working! The stability of two-hands interaction has really changed from night to day! Now finally the hands tracking system is usable for two hands interactions, and this pose well for the future of natural interactions on Quest (and Cambria).
Quest Hands Tracking v2 review
After this positive first impression, I wanted to dig deeper and evaluate carefully the performance of hand tracking, so I made a few tests in various apps, in particular the Oculus Home menu, Hand Physics Lab, and VR Workout.
Talking about the tracking of a single hand, the performances are very good, and the virtual hand is now able to follow very well my real hand. It is especially impressive how the fingers tracking is very accurate. The virtual hand and all its fingers have most of the time the exact pose of my real one. There are some poses that challenge hard the tracking algorithms, and sometimes the pose of the pinky or ring finger is slightly inaccurate: for instance, when I make the ring finger touch the thumb, sometimes I don’t see the fingertips of the virtual hand actually touching, but still quite distant. I have verified the tracking using passthrough, so having some ground truth, and I can tell you that apart from these small issues, which anyway are not a big deal, the tracking went usually very well. I was quite impressed.
I read a claim saying that now the tracking algorithm is also very reactive, so suitable to be used also in fitness applications. I tried immediately to punch stuff, to discover that actually, the tracking fails miserably when you start giving real punches. I tried a level of VR Workout, and as long as I was providing slow punches, everything worked, but when I truly started punching stuff, the hands were disappearing. When the hand is too fast, the tracking still fails. I can see that the tracking is better than before in this situation, but in my opinion, it is still not enough for boxing games.
As for the field of view of tracking, I could see my fingers still tracked at the edge of the visual FOV of the headset, so as long as you see your hands, they are surely tracked. I even tried to click on a button that it was outside of my peripheral view and the interaction still worked, so the tracking FOV is bigger than the vision FOV.
Talking about two-hand interactions, the system is now much more stable than before, and as I said in the previous paragraph, I could even interwave the fingers of the two hands and the tracking still worked! This is truly a great result achieved by Meta engineers. Anyway, when I moved one hand in front of the other, it still happened many times that the occluded hand lost the tracking or that it started glitching. Also some poses with the two hands heavily touching still made the tracking go crazy. So, two hands interactions are much better than before, but the tracking is still not 100% reliable when the two hands are occluding each other.
Comparison with Ultraleap tracking
Let me compare the tracking of the hands on Meta Quest 2 with the tracking offered by the Ultraleap Stereo IR 170 Sensor and Gemini tracking on the previous 4 points:
- Single hand tracking accuracy: I would say that now Meta is on par. The tracking of the fingers worked very well, with just some minor glitches, exactly as it happens with Ultraleap devices;
- Robustness to fast movements: I would say that Ultraleap is slightly better on this. They are almost on par, but I could move my hands a bit faster before making the tracking on the hand get lost when using the latest Ultraleap sensor;
- Tracking FOV: I think this is another tie. Both of them have hands tracking FOV bigger than the visual FOV.
- Two-hands interactions: Ultraleap is more robust on this, and it is much more difficult to make the tracking on one hand get lost because the other one is occluding it. By the way, the gap between the two solutions is not that huge.
It’s impressive how this update puts Quest hands tracking close to the gold standard in the industry, that is Ultraleap tracking.
Final impressions
This new version of hands tracking of the Quest has impressive performances if compared to the previous one. It is much better in everything, and I am personally pleasantly surprised to see how robust it has become even with two-hands interactions. It is not the best hand tracking solution on the market, yet, because Ultraleap still has an edge, but it has got close. And if the jump from the v1 to the v2 has been this high, I’m very curious to discover how the v3 will be…