Some days ago we were all excited about the news of the upcoming new controllers for SteamVR system: the Knuckles. And we still are. These controllers appear as an Oculus Touch 2.0, since as Oculus Touch they’re controllers you wear in hands, that offer precise hands and fingers tracking, but are more ergonomic and offer a much better finger tracking than their competitors. We’re all very curious about this hardware, but news are very few and we have to wait that some selected owners share with us some info on social media to know something more. On today’s post I want to collect in one place all info I’ve read through reddit, twitter, youtube, steamvr website and other sources. Hope you’ll like it.
The timeline
First of all, let’s talk about delivery. Why are there people already having these controllers? Who are they? And when we’ll have ours? A reddit comment gives un some insights on this process:
Who answers, VR_Nima, is member of Cloudhead Games, one of the very few studios that have already received the hardware (you’ll read this name many times, since it has shared a lot of info about this controllers). The surprising news is that they haven’t received the hardware one month ago, but around ONE YEAR AGO. In this year, these selected studios have helped Valve in developing the hardware, that has in fact evolved a lot. When the controllers have been announced for the first time, Valve stressed the fact that the only true way to develop a controller is use it to develop a game. So, while experimenting with the controller in games creation, you realize if it has been made in the right way, if it is ok for developers and for gamers… if it let people do what they expect from such kind of device. For this reason, Valve has offered the controllers to some game studios to help them in making this research, to experiment stuff, to go trial-and-error and create an awesome product.
And this has happened. Reading Cloudhead Games twitterline, you can discover lots of evolutions that the controller have had in this year. Road To VR has made a zoom on an image posted by this game studio, where you can see the first version of the controller and the current one.
Old year version was really rough, seems like a modified glove, while new one is more polished and a lot more comfortable. Cloudhead underlines how this has happened thanks to a collaboration between Valve and its partners in this tweet
Road To VR underlines the fact that the new controllers have still visible photodiodes, that will have to be covered by opaque (but IR-transparent) plastic, so this Knuckles version is still not the final one. Valve is still planning to evolve them a bit.
About the previous reddit comment, VR_Nima forecasts a larger developer roll-out before fall, so just get ready to have them in your hands in some months! I guess that, as always, Valve will open up a form and will send some of the controllers for free to the developers offering the best ideas implementing this new input device. So, start brainstorming some good ideas to send them…
At the moment, apart from Cloudhead games, other confirmed studios having them are Owlchemy Labs and Zulubo Productions (developer of Vertigo). Most of the data of this article has been taken from information shared by them (especially Cloudhead games, which is very active on twitter).
The unboxing
Unluckily there’s still not an official unboxing of the new Knuckles controllers, but we can have a look at their beautiful box that features a complete teardown of the controllers
Inside the box, the controllers fit this way
And this is another photo with the box and the controllers next to it. I love this photo since it is a good close up on the controllers, so we can clearly see how they will be
How they’re born
Here a redditor (I don’t remember his name, sorry man) has depicted a rare image of the moment of the conception of a Knuckles controllers. Be careful on sharing it because it is clearly NSFW.
Ok, after this epic moment, let’s get back to the serious stuff..
The technology
Knuckles controllers are SteamVR Tracking 2.0 devices. This is quite obvious, since Valve has recently announced this update to its tracking system… and would have been quite stupid producing new devices with the old tracking technology (that is slowly going to die). This is confirmed by a post by Cloudhead games, that highlights the fact that they’re still usable with current version of Lighthouse station (as all SteamVR 2.0 devices)
Tracking of hand is already super precise and stable and even making the hand move super-fast and then stop doesn’t cause the virtual hand to wobble. This is awesome.
The ability to track all fingers is possible thanks to capacitive sensors. While the Oculus Touch have capacitive sensors only to detect if the thumb and index are touching the buttons, here the controller is full of this kind of sensors, so to detect an accurate pose of all parts of the hand.
From SteamVR docs, we can read where these sensors are placed:
Capacitive sensors are placed under the following button surfaces:
- Trigger surface
- Outer Face button surface
- Inner Face button surface
- System button surface (not accessible by applications)
There are also separate arrays of capacitive sensors in the grip which enable two new user inputs :
- A capacitive grip axis for grasp/ungrasp actions (Joystick Axis 2.x)
- Independent index, middle, ring, and pinky finger axes (Joystick Axes 3, 4)
Thanks to this, fingers from index to pinky are able to detect how much they’re bent.
Each finger axis returns a curl value between zero and one, where zero indicates that the finger is pointing straight out and one indicates that the finger is fully curled around the controller.
From what I can understand from this information, seems that thumb finger has a rougher detection than the other four fingers. This appears confirmed by the fact that in all videos out there, no one is making smooth movements with this finger, while instead moves all the other ones. Even the index, having its position tracked only thanks to the touch of the trigger, seems not tracked very precisely. This is due to the fact that the other three fingers have big capacitive zones with which they’re in touch, while thumb and index only have some capacitive “spot” points.
About precision of detection of curliness of fingers, we seem to be at a very good status.
So, we have something like 20-50 possible steps of curliness of the fingers (I guess higher resolution values are for best tracked fingers like the middle, ring and pinky), that means that we’ve the fingers tracked very smoothly. Considering that Oculus Touch have something like 2-3 status at maximum, you can see how this controllers are a big improvement for hands presence and interaction. They are somewhat like Leap Motion, but with a physical controller, that can also provide some form of haptics. Ah, about haptics… it is still not clear how it will be provided (maybe some form of vibration?). Someone has asked to Cloudhead Games, but they’ve not answered (in the worst case, we can emulate it with sounds, as Leap Motion suggests).
Hand fit
The new controllers are comfortable and light, but there is still not many data about its usage over time that was one of my possible issues, since having something that grips your hand for long periods of time may be uncomfortable. We’ll see. Surely Valve is working a lot on comfort as we’ve seen, so if there were any issues, I think they’d fix them.
About the grip, one of the things that I liked a lot was the possibility to take objects and launch them freely in VR. Oculus Touch require you to keep the controllers with at least one finger, it is not possible to open completely the hand, so you can’t really throw stuff in a natural way or the controller slips away, while Knuckles let you open completely it. But is it so safe to use that to launch an object at high speed? Well, not…
So, the grip is strong, but not super-strong. This is compatible with the idea of offering comfort to the user and not grinding his hands. Maybe with Knuckles v2 we’ll be able to launch objects with anger.
Leap Motion comparison
I’ve cited Leap Motion in the last paragraph, because this device is able to track fingers exactly like Leap device, but without the hassle of having to handle occlusions. I was curious about the difference between this and Leap, so I’ve asked directly Cloudhead…
So, Leap is more precise… it is still more accurate in finger tracking, especially because not only it has a better tracking resolution of fingers states, but it has also tracking of more bones for each finger. This means that we have not only a bend factor for the fingers, we have for each finger its three bones tracked independently (so for instance I can keep the first bone of the finger open while closing the other two… this is not possible with Knuckles). Furthermore I guess it can handle better tracking of thumb and index and can also detect “horizontal” movement of fingers, so them spreading away from each other. Leap still wins.
I’ve also asked Leap Motion directly what’s their opinion on controllers and they don’t seem too worried by them… maybe because their business model is now to be included into Qualcomm headsets…
SDK & Hands interactions
Leap Motion has produced a marvelous interaction system to let you interact with your hands in VR in a natural way. For Knuckles there is still not something that works that way, but Zulubo is trying to code it. Looks at this demo video he’s made!
This is an experience he’s made to showcase the interaction system he’s developing for the Knuckles: at the moment it is not as vast as Leap Motion one, but already includes super-interesting things like the ability to grab stuff using the thumb and the index and then throw it, use of different hand models and interaction with UI elements.
Interaction system is crucial to let developers using Knuckles to do awesome stuff using Knuckles controllers, that’s why Zulubo’s work is very interesting. It lets us developers to concentrate on the most important things of our experiences, while he’s doing all the low-level stuff.
In my opinion this interaction system should be integrated into official SteamVR SDK and maybe this is what Valve will do over time (not sure about that, I’m just guessing). Otherwise we can just continue using it downloading it from its GitHub repo. You can access the presentation of the project with a little documentation about it on the guide that Zulubo has written.
About easiness of usage of the official Knuckles SDK, we still don’t have feedback from devs. But from the docs it seems ratherly straightforward, since you can access a curliness value for each finger just with a simple function call.
Reading about how to retrieve values on SteamVR docs, I’ve learned that Valve advices developers to smooth values returned by the device (through Unity’s Lerp method) and this makes me think that this means that detection of finger status is still a bit rough and not super-smooth.
Usage inside SteamVR
When used to play games inside SteamVR, Knuckles are pretty cool. In these image gallery published by Cloudhead, you can see how they appear inside VR and also how they are shown inside SteamVR Status Window
As you can see, the scroll wheel is missing from that model. Actually it appears only when it is useful, as when you’re scrolling through menus.
Here you can see a guy from Cloudhead playing with Knuckles inside that awesome playground which is SteamVR Home Beta.
And here there is the same guy making a middle finger to us all! As you can see the tracking is marvelous…
Here instead there’s another video of people playing in a multi-player environment… as you can see Knuckles will be awesome for social VR, too
There may be a question popping into your head: how do I use Knuckles to play existing content? I mean, all games made for SteamVR are not compatible with these controllers. Well, there is a guide for it. The key is going into Knuckles configuration inside SteamVR and configure them so that to disable some of their functionalities so that they can be used as old-fashioned controllers.
This makes me think about the future of OpenVR: currently OpenVR driver maps all possible input types to Vive VR controllers input axes and buttons. But now, Vive Controllers are going to be replaced by Knuckles… will this mean that all OpenVR drivers will have to be rewritten to map themselves into Knuckles? Or will OpenVR support two kind of controllers (wands and hands)? If someone more skilled me has the answer, please let me know in the comments!
And that’s it for all the info I’ve been able to gather around. Hope this has been useful to satisfy part of your curiosity… if this is the case please use that innovative mouse controller you have in your hand to share this article!
(Header Image by Cloudhead Games)