Valve shows Knuckles prototype controllers v2

One of the biggest drawbacks of HTC Vive headsets with regard to Oculus Rift are the controllers: while Oculus Touch is ergonomic and can emulate the hands quite well, Vive has controllers that are quite big and are usable only as wands. To solve this issue, some time ago Valve announced it was producing a new kind of ergonomic controllers for SteamVR compatible headsets (that is Vive ones, since LG headset has mysteriously vanished): Valve Knuckles.

Valve Knuckles are a new generation of VR controllers, that can track all the fingers of the users and that, since they are gripped to your hand, also let you throw virtual objects by opening all the fingers. Both of these things are impossible with Oculus Touch, so we are all very excited regarding these new devices. In June 2017, Valve showcased some photos and videos of these controllers, but after that moment, we had very few info about them. Until today.

Today, a year after the last update, we have new info about Knuckles controllers. And they are amazing info.

Knuckles have been evolved to a new version called EV 2, that is the v2 prototype of these controllers. They have been dramatically improved wrt the previous version, thanks to the feedback of all the selected developers and studios that are using these new devices (e.g. Owlchemy Labs, Cloudhead Games, etc…). All the improvements are highlighted in this SteamVR Community page, that I advise you to read, and they are:

• Improved industrial design and ergonomics;
• Updated input set and layout;
• Improved strap fit and feel for more hand sizes;
• New sensors, enabling new interactions;
• Improved battery life, USB-C charging port;
• Added support for SteamVR Tracking 2.0.

Basically, the controllers have been completely revolutionized, so that they are:

• Compatible with SteamVR Tracking v2;
• More comfortable for every kind of hand (they tested them with smaller and bigger hands);
• Offer a completely new kind of interactions.

The last point is the one that made my jaw drop. First of all, the new Knuckles can track all the fingers in a great way: thanks to the sensors present on all the device, the runtime is able to detect the pose of all the fingers in an accurate way.

This is possible thanks to capacitive sensors present on all the controller. The handle of the Knuckles is able to infer the pose of all the fingers holding it, that is middle, ring and pinkie (e.g. if the sensors feel that the area relative to the middle finger is touched completely by the finger, they can infer that the middle finger is closed, if they sense that only half of the handle is touched, then the middle finger is half opened and so on). But we have even more: all buttons have capacitive sensors, meaning that they can sense your thumb and index fingers pose. This means that all fingers are tracked: this was something already present in the previous prototype, but it has been dramatically improved.

Then we have FORCE SENSORS. The handle and one of the top buttons can sense the force you’re applying on the controller: this is a game changer for games UX. The controller can sense the force with which you are closing your hand. This means for instance that you can hold something in your hand and destroy it by closing your fist with a certain force.

In the prototype game Moondust, Valve engineers make you hold rocks in your hands and then you can crush them by closing your fist. But you have to do that with a certain amount of force, otherwise, the rocks won’t break. Just WOW. A funny thing is that Valve reported that during the tests, someone broke the controller by applying too much force in order to destroy something :D.

The result of all these sensors is that you have a great emulation of the pose of the fingers in every moment. It is still not perfect (Leap Motion is still more accurate), but surely it is a step forward from the Oculus Touch.

And with the info from triggers, capacitive buttons and force sensors, you can produce a lot of new interactions: see this grenade throwing experiment from Moondust:

Look at how the grenades are launched: Valve now lets you launch them without using any kind of triggers: by just using the data from the fingers sensors, it can understand when you are launching the grenade, at what speed and in which direction without forcing you to push a binary button. Furthermore, you may notice in the video that the grenade is activated only when the fingers are pushed against it with a certain force.

As Valve underlines, these new sensors can not only create new interaction types, but can also makes the old ones more natural: until now we only used triggers to do stuff, but this is not natural (we don’t have triggers in our hands!) and can also produce confusing interfaces. With force and capacitive sensors, we can make things in a different and more usable way. Read this example written by Valve to understand why:

If I pick something up by pulling a trigger then I can either no longer use that trigger to release that object, or I can’t interact with that object using the trigger. Think about a fire extinguisher: if I pick it up with the trigger how do I spray it? While it’s possible to come up with a solution for existing controllers, all of these solutions require explanation to experienced users. With Knuckles EV2 you just tell the user to grab it and by some combination of capacitive and force sensing we recognize that grab at the location it happened on the controller. Then they can naturally pinch the extinguisher to spray it.

Yes, it’s awesome. And will make VR devices more usable for non-techie people.

And it is also awesome that some interactions algorithms are going to be released by Valve to developers inside the SteamVR SDK so that we’ll be able to use them from our day 1 of coding.

Since we are talking about SDKs, Valve has just announced a new SDK to take in count hands tracking: SteamVR Skeletal Input. SteamVR Skeletal input will abstract the actual controller used by the player (Oculus Touch, Knuckles, Leap Motion, etc…), giving the developer the best pose of the hand detectable with the actually used sensor. This way, the developer can develop his/her game based on the hand pose of the user without caring about the actual controller owned by the player. This will be possible because every controller will have a dedicated SteamVR Skeletal Input driver, providing the animations of the hands related to that controller based on the input provided by that controller. Watch the below video showed by Valve to see how the same hand in Skeletal Input SDK behaves differently based on the actual controller:

HTC Vive Wands

Oculus Touch Controllers

Knuckles EV2

With only one SDK, developing only once, the developer can target all these controllers. Of course, people owning Knuckles will have better in-game performances (hands’ pose detection is more accurate).

The schema of the new Knuckles version is this one:

As you can see, Valve has removed the touchpads and substituted them with thumbsticks, because developers have claimed that Thumbsticks are far better for games (notice that this is the same thing that Oculus has done with the Santa Cruz controllers). The touchpad has been substituted by the “track button“, that is a place where you can make your thumb rest, but that can also work as a little trackpad and as an additional button, thanks to the force sensors embedded into it. This is an innovative multi-purpose area of the controller, that otherwise would have had a top part almost identical to the one of Oculus Touch.

Then we have the main trigger, useful for instance for FPS games; the handle, with the capacitive + force sensors described above, to detect what all fingers are doing; and the grip to secure the controller to your hand. As I’ve already said, the grip has been engineered to be more comfortable for the biggest number possible of users.

This is how you wear the device

And this how the thumb is able to reach all the top controller’s thanks to the improved ergonomics

Regarding the batteries, Knuckles have embedded rechargeable batteries, like the Vive wands. This is great IMHO because I find annoying to change the batteries of my Touch controllers when they run out of charge. You just use USB-C and you charge them in just one and a half hour to have 6 hours of gaming.

What about content? Valve says that selected developers can try using Knuckles with existing games, but of course players may have troubles with some of them, since they have not been conceived with Knuckles in mind (anyway, thanks to the new button mapping offered by SteamVR Input, they can go in and adjust the button configurations for Knuckles and have better results). The only content that is fully compatible with Knuckles is Moondust, that is the sandbox that Valve has used to experiment with these new controllers and the related UX they enable (read this article to know all they’ve learned about interactions with Knuckles by playing in Moondust). Playing around with Moondust, Valve engineers have been able to experiment on how to use the Knuckles to do some operations, like throwing objects or commanding a car.

The source code of Moondust is available to developers, so they can learn how to develop with Knuckles. Have a look at the below video to learn about the content of Moondust:

Of course, we don’t know anything about price and release date of Knuckles, yet. But one of the articles released today contains this sentence

Soon, hundreds of developers will get their hands on developer kits and begin experimenting, but for the last few months that task has been left up to us.

“Soon” is the magic world. This sentence states that in little time Valve will start giving dev kits to a bigger group of developers. How to enter this group? Well, instructions from Valve are clear:

If you’re a developer interested in participating in the developer kit program, you can apply on the Steam Partner page. Log in, and look for “VR Developer Kit Request” along the right side of the page.

I think that the Steam Partner page is this one.

I’m hyped by Valve Knuckles and I’m happy to see that they’re evolving in an amazing way. Now they look like Oculus Touch v2, they’re fantastic. And since IMHO they will be released in some months, I can’t wait to try them!

I hope you are as excited as me: if this is the case, don’t you mind subscribing to my newsletter to sustain my magazine? 😉

(Header image by Valve)