virtual reality Vive Knuckles new information

Some new infos, images and videos about Valve’s Knuckles

by Skarredghost

Some days ago we were all excited about the news of the upcoming new controllers for SteamVR system: the Knuckles. And we still are. These controllers appear as an Oculus Touch 2.0, since as Oculus Touch they’re controllers you wear in hands, that offer precise hands and fingers tracking, but are more ergonomic and offer a much better finger tracking than their competitors. We’re all very curious about this hardware, but news are very few and we have to wait that some selected owners share with us some info on social media to know something more. On today’s post I want to collect in one place all info I’ve read through reddit, twitter, youtube, steamvr website and other sources. Hope you’ll like it.

The timeline

First of all, let’s talk about delivery. Why are there people already having these controllers? Who are they? And when we’ll have ours? A reddit comment gives un some insights on this process:

Comment
byu/VR_Nima from discussion
invirtualreality

Who answers, VR_Nima, is member of Cloudhead Games, one of the very few studios that have already received the hardware (you’ll read this name many times, since it has shared a lot of info about this controllers). The surprising news is that they haven’t received the hardware one month ago, but around ONE YEAR AGO. In this year, these selected studios have helped Valve in developing the hardware, that has in fact evolved a lot. When the controllers have been announced for the first time, Valve stressed the fact that the only true way to develop a controller is use it to develop a game. So, while experimenting with the controller in games creation, you realize if it has been made in the right way, if it is ok for developers and for gamers… if it let people do what they expect from such kind of device. For this reason, Valve has offered the controllers to some game studios to help them in making this research, to experiment stuff, to go trial-and-error and create an awesome product.

And this has happened. Reading Cloudhead Games twitterline, you can discover lots of evolutions that the controller have had in this year. Road To VR has made a zoom on an image posted by this game studio, where you can see the first version of the controller and the current one.

Steam VR Knuckles controllers before and after
Evolution of Valve’s controller from last year (on the left) to now (on the right). Notice how it is evolved from a raw prototype to a semi-finished product (Image by Cloudhead Games, cropped by Road To VR)

Old year version was really rough, seems like a modified glove, while new one is more polished and a lot more comfortable. Cloudhead underlines how this has happened thanks to a collaboration between Valve and its partners in this tweet

Road To VR underlines the fact that the new controllers have still visible photodiodes, that will have to be covered by opaque (but IR-transparent) plastic, so this Knuckles version is still not the final one. Valve is still planning to evolve them a bit.

About the previous reddit comment, VR_Nima forecasts a larger developer roll-out before fall, so just get ready to have them in your hands in some months! I guess that, as always, Valve will open up a form and will send some of the controllers for free to the developers offering the best ideas implementing this new input device. So, start brainstorming some good ideas to send them…

At the moment, apart from Cloudhead games, other confirmed studios having them are Owlchemy Labs and Zulubo Productions (developer of Vertigo). Most of the data of this article has been taken from information shared by them (especially Cloudhead games, which is very active on twitter).

virtual relaity Vive Knuckles new information
Owlchemy Labs people playing with Vive Knuckles (Image by Owlchemy Labs)
The unboxing

Unluckily there’s still not an official unboxing of the new Knuckles controllers, but we can have a look at their beautiful box that features a complete teardown of the controllers

Vive Knuckles controllers box
Beautiful box of the Vive Knuckles (Image by CloudHead Games)

Inside the box, the controllers fit this way

virtual relaity Vive Knuckles unboxing
Vive Knuckles inside the box: in this photo you can also see the sides of the box, with awesome writings (Image by Owlchemy Labs)

And this is another photo with the box and the controllers next to it. I love this photo since it is a good close up on the controllers, so we can clearly see how they will be

Vive Knuckles with box VR
Controllers with box. Click on the image to zoom it and appreciate better the devices (Image by CloudHead games)
How they’re born

Here a redditor (I don’t remember his name, sorry man) has depicted a rare image of the moment of the conception of a Knuckles controllers. Be careful on sharing it because it is clearly NSFW.

NSFW

Ok, after this epic moment, let’s get back to the serious stuff..

The technology

Knuckles controllers are SteamVR Tracking 2.0 devices. This is quite obvious, since Valve has recently announced this update to its tracking system… and would have been quite stupid producing new devices with the old tracking technology (that is slowly going to die). This is confirmed by a post by Cloudhead games, that highlights the fact that they’re still usable with current version of Lighthouse station (as all SteamVR 2.0 devices)

Tracking of hand is already super precise and stable and even making the hand move super-fast and then stop doesn’t cause the virtual hand to wobble. This is awesome.

The ability to track all fingers is possible thanks to capacitive sensors. While the Oculus Touch have capacitive sensors only to detect if the thumb and index are touching the buttons, here the controller is full of this kind of sensors, so to detect an accurate pose of all parts of the hand.

Knuckles controllers
Close-up of the thumb Trackpad of the Knuckles controllers (Image by Cloudhead Games)

From SteamVR docs, we can read where these sensors are placed:

Capacitive sensors are placed under the following button surfaces:

  1. Trigger surface
  2. Outer Face button surface
  3. Inner Face button surface
  4. System button surface (not accessible by applications)

There are also separate arrays of capacitive sensors in the grip which enable two new user inputs :

  1. A capacitive grip axis for grasp/ungrasp actions (Joystick Axis 2.x)
  2. Independent index, middle, ring, and pinky finger axes (Joystick Axes 3, 4)
Valve Knuckles capacitive sensors
Capacitive zones for the thumb (Image by Valve)

Thanks to this, fingers from index to pinky are able to detect how much they’re bent.

Each finger axis returns a curl value between zero and one, where zero indicates that the finger is pointing straight out and one indicates that the finger is fully curled around the controller.

VR Knuckles controllers capacitive sensors
SteamVR Knuckles capacitive sensors for fingers from index to pinky. You can access their data as joystick axes in game engines like Unity as a value from 0 to 1. (Image by Valve)

From what I can understand from this information, seems that thumb finger has a rougher detection than the other four fingers. This appears confirmed by the fact that in all videos out there, no one is making smooth movements with this finger, while instead moves all the other ones. Even the index, having its position tracked only thanks to the touch of the trigger, seems not tracked very precisely. This is due to the fact that the other three fingers have big capacitive zones with which they’re in touch, while thumb and index only have some capacitive “spot” points.

About precision of detection of curliness of fingers, we seem to be at a very good status.

So, we have something like 20-50 possible steps of curliness of the fingers (I guess higher resolution values are for best tracked fingers like the middle, ring and pinky), that means that we’ve the fingers tracked very smoothly. Considering that Oculus Touch have something like 2-3 status at maximum, you can see how this controllers are a big improvement for hands presence and interaction. They are somewhat like Leap Motion, but with a physical controller, that can also provide some form of haptics. Ah, about haptics… it is still not clear how it will be provided (maybe some form of vibration?). Someone has asked to Cloudhead Games, but they’ve not answered (in the worst case, we can emulate it with sounds, as Leap Motion suggests).

Hand fit

The new controllers are comfortable and light, but there is still not many data about its usage over time that was one of my possible issues, since having something that grips your hand for long periods of time may be uncomfortable. We’ll see. Surely Valve is working a lot on comfort as we’ve seen, so if there were any issues, I think they’d fix them.

About the grip, one of the things that I liked a lot was the possibility to take objects and launch them freely in VR. Oculus Touch require you to keep the controllers with at least one finger, it is not possible to open completely the hand, so you can’t really throw stuff in a natural way or the controller slips away, while Knuckles let you open completely it. But is it so safe to use that to launch an object at high speed? Well, not…

So, the grip is strong, but not super-strong. This is compatible with the idea of offering comfort to the user and not grinding his hands. Maybe with Knuckles v2 we’ll be able to launch objects with anger.

Leap Motion comparison

I’ve cited Leap Motion in the last paragraph, because this device is able to track fingers exactly like Leap device, but without the hassle of having to handle occlusions. I was curious about the difference between this and Leap, so I’ve asked directly Cloudhead…

So, Leap is more precise… it is still more accurate in finger tracking, especially because not only it has a better tracking resolution of fingers states, but it has also tracking of more bones for each finger. This means that we have not only a bend factor for the fingers, we have for each finger its three bones tracked independently (so for instance I can keep the first bone of the finger open while closing the other two… this is not possible with Knuckles).  Furthermore I guess it can handle better tracking of thumb and index and can also detect “horizontal” movement of fingers, so them spreading away from each other. Leap still wins.

I’ve also asked Leap Motion directly what’s their opinion on controllers and they don’t seem too worried by them… maybe because their business model is now to be included into Qualcomm headsets…

SDK & Hands interactions

Leap Motion has produced a marvelous interaction system to let you interact with your hands in VR in a natural way. For Knuckles there is still not something that works that way, but Zulubo is trying to code it. Looks at this demo video he’s made!

This is an experience he’s made to showcase the interaction system he’s developing for the Knuckles: at the moment it is not as vast as Leap Motion one, but already includes super-interesting things like the ability to grab stuff using the thumb and the index and then throw it, use of different hand models and interaction with UI elements.

Knuckles VR interaction system
Interaction with UI elements performed using Zulubo’s interaction system (image by Zulubo)

Interaction system is crucial to let developers using Knuckles to do awesome stuff using Knuckles controllers, that’s why Zulubo’s work is very interesting. It lets us developers to concentrate on the most important things of our experiences, while he’s doing all the low-level stuff.

In my opinion this interaction system should be integrated into official SteamVR SDK and maybe this is what Valve will do over time (not sure about that, I’m just guessing). Otherwise we can just continue using it downloading it from its GitHub repo. You can access the presentation of the project with a little documentation about it on the guide that Zulubo has written.

About easiness of usage of the official Knuckles SDK, we still don’t have feedback from devs. But from the docs it seems ratherly straightforward, since you can access a curliness value for each finger just with a simple function call.

Reading about how to retrieve values on SteamVR docs, I’ve learned that Valve advices developers to smooth values returned by the device (through Unity’s Lerp method) and this makes me think that this means that detection of finger status is still a bit rough and not super-smooth.

Usage inside SteamVR

When used to play games inside SteamVR, Knuckles are pretty cool. In these image gallery published by Cloudhead, you can see how they appear inside VR and also how they are shown inside SteamVR Status Window

As you can see, the scroll wheel is missing from that model. Actually it appears only when it is useful, as when you’re scrolling through menus.

SteamVR Knuckles controllers
Scrollwheel appearing to let the user scroll menus in SteamVR (Image by Cloudhead Games)

Here you can see a guy from Cloudhead playing with Knuckles inside that awesome playground which is SteamVR Home Beta.

And here there is the same guy making a middle finger to us all! As you can see the tracking is marvelous…

Here instead there’s another video of people playing in a multi-player environment… as you can see Knuckles will be awesome for social VR, too

There may be a question popping into your head: how do I use Knuckles to play existing content? I mean, all games made for SteamVR are not compatible with these controllers. Well, there is a guide for it. The key is going into Knuckles configuration inside SteamVR and configure them so that to disable some of their functionalities so that they can be used as old-fashioned controllers.

This makes me think about the future of OpenVR: currently OpenVR driver maps all possible input types to Vive VR controllers input axes and buttons. But now, Vive Controllers are going to be replaced by Knuckles… will this mean that all OpenVR drivers will have to be rewritten to map themselves into Knuckles? Or will OpenVR support two kind of controllers (wands and hands)? If someone more skilled me has the answer, please let me know in the comments!

Valve Knuckles configuration
Knuckles configuration inside SteamVR. It is shown only if Knuckles driver is detected (Image by Valve)

And that’s it for all the info I’ve been able to gather around. Hope this has been useful to satisfy part of your curiosity… if this is the case please use that innovative mouse controller you have in your hand to share this article!

(Header Image by Cloudhead Games)

Disclaimer: this blog contains advertisement and affiliate links to sustain itself. If you click on an affiliate link, I'll be very happy because I'll earn a small commission on your purchase. You can find my boring full disclosure here.

2 thoughts on “Some new infos, images and videos about Valve’s Knuckles”

  1. Nice post there Tony! Regarding the question about OpenVR support for the Knuckles I guess that makes sense to add them as a new type of TrackedDevice, the component used to represent any of the 15 possible tracked devices inside OpenVR. So we already have TrackedDevice_Controller, TrackedDevice_Tracker, and maybe now TrackedDevice_Knuckles. Thinking twice maybe the new controllers can be represented as a standard TrackedDevice_Controller but adding new possible events to be triggered so we have not only VREvents for each button press but also for the curl of each finger and stuff. This way the driver/runtime should just be updated to implement those new events. I can be wrong but that’s my bet after playing a bit with “raw” OpenVR

Comments are closed.

Releated

virtual reality hog awe europe

My experience at AWE Europe and a comparison with the US chapter

I’ve just come home after my trip to Vienna to attend AWE Europe. Many people asked me about my experience with the event and especially what are its differences with the US version, so I thought it could be worth writing a little post about it. AWE is one of the best XR events out […]

by Skarredghost
shiftall meganex superlight 8k review hands on

AWE EU – MeganeX Superlight 8K hands-on: impressive resolution but at a price

Yesterday was my first day at AWE Europe, and I tried some very interesting devices there. My first test was with MeganeX Superlight 8K, Shiftall’s new headset with a very high resolution in a very compact design. MeganeX Superlight 8K Just a brief history of the MeganeX Superlight 8K, before we delve into the real […]

by Skarredghost