Dexmo force feedback gloves show the future of hands presence in VR

One of the most interesting encounters I had in my trip to Beijing has been with Dexta Robotics, the company behind the Dexmo force-feedback gloves.

They have been so kind to set up a demo just for me in Beijing (the company is from Shenzhen) to let me try their product and have a chat about it, and I really want to thank them for their kindness and the time they spent with me.

So, here you are a very long deep dive into Dexmo gloves and my experience with them.

Dexta Robotics

If you are into VR since a while, for sure the name Dexta Robotics isn’t new to you. They announced some years ago a glove to let you feel the objects in VR and got some interest from the VR community. At a certain point, information got confused: someone received devkits, but then we had no more info about the company and the product. Honestly, I thought that they had shut down.

This until I was contacted by one of their PRs (Miss Zhang Zizi) about the launch of an enterprise edition of the gloves. Dexta CEO Aler Gu (Xiaochi Gu) explained to me that that period of silence was due to the fact that they decided to focus less on marketing and focus fully on making the product better in stealth mode. They were going from the prototype stage to the actual shipping-ready production stage. “The world had seen enough fancy demos people can’t have access to. A product cannot make a difference unless it actually ships” he said.

This required a lot of effort, ri-organization, money raising and other stuff that made the status of the company appear confused from the outside. Actually, they never had stopped working on the project. They were working on it even more than before, just giving less priority to marketing.

Now Dexta has managed to perform this jump from the devkit to the actual product and it is ready to show it to the world.

Dexmo

The Dexmo force-feedback glove is an exoskeleton that you wear to have true fingers presence in any virtual environment. It can be used both with a standard screen or an AR/VR headset, but I think that virtual reality is where this device really shines. It comes in three flavors:

1. Low tier: 11 DoF finger tracking and 7-points tactile sensations on the palm and fingertips;
2. Medium tier: 11 DoF fingers tracking and 5-points force feedback on all fingers;
3. High tier: 11 DoF fingers tracking and 7–points tactile + 5-points force feedback sensations on all fingers.

Force-feedback is the true specialty of this company and this is why Dexta has made me try the version 2 and 3 of the gloves.

All three versions of Dexmo are intended for enterprise usage for now (e.g. for advanced training) and so even the price is very enterprise-oriented. Interested companies from all over the world may directly contact Dexmo to discuss the sale.

I specified “for now” because people from Dexta told me that they have a plan to firmly march towards delivering Dexmo to general consumers, once they gain enough economy of scale. I really hope this can happen, because VR with haptic feedback is overly cool!

Force-feedback

You may wonder what “force-feedback” means. Basically, the glove is able to simulate the forces that objects apply to your hands in the real world. Here you are some examples:

• If you close your hand around a real object, your fingers can’t penetrate it, so the object exerts a force against your fingers that prevent them from entering its surface;
• If something falls on your fingers, this applies a force on your fingers, that so move as a consequence of this;
• If you push a virtual button, the virtual object applies a counter-force to your fingertips.

https://gfycat.com/spectacularwillinggelada

These are all examples of things that happen in real life and that currently don’t happen in VR. In current commercial VR setups, your hands permeate objects, and no object can apply a force on you… and that’s why all action titles feel so fake. But with “Force-feedback” gloves, all of this change.

Force-feedback gloves are able to apply a force to your fingers so that to simulate a force happening in real life. For instance, if you hold a virtual object in your hands and you try to close your hand, the force-feedback glove will prevent your fingers to enter the surface of the virtual object, exactly as it happens in reality. This will increase a lot the realism of your experience. This kind of gloves can also simulate something actively touching your fingers: e.g. if someone pushes against your fingers in VR, you would feel that sensations applied to your real fingers in real life. That’s like magic.

https://gfycat.com/idleidealinchworm

Dexmo force-feedback gloves

Aler told me that creating this force-feedback effect is very difficult. He explained to me that once that you have found the right way of implementing force-feedback, adding all the other features to the glove is an easier task. That’s why he and his team have experimented a lot with various methods until they found the one that in their opinion was the best one. And now that they have been able to set up this main part of the device, they can focus on adding features over time, until they will have the perfect glove for VR.

The technology that they are using for force-feedback is servo motors. This gives the Dexmo various advantages over other methods, like strings pulling. The most important of them is the compact factor: Dexmo exoskeletons are small and light. Some of its competitors, like HaptX (formally named AxonVR) are really bulky and heavy, while other ones require a haptic system that extends all over the arm.

The other big advantage is that with servo motors you can apply a variable force: some other methods act as a on/off mechanism, so for instance you can let the fingers move or stop them to simulate an external force, while with servo motors you can even only apply a resistance force. Your fingers will still be able to move, but the movements will be harder to perform. Think of what happens when your hands have to move in a fluid or when they have to squeeze an object made of rubber.

The third advantage is that the gloves provide pressure only to places where you expect them to be, like fingertips, due to its exoskeleton design that isolates part of the glove from the back of fingers. With other methods, this is not guaranteed: for instance, the continuous pulling of the strings may cause downward pressure to the back of hands and create confused haptic sensations.

Dexmo fingers tracking

Dexmo gloves track 11 degrees of freedom for each hand. Exactly, a single glove is able to detect:

• The bending factor for all fingers (how much the fingers are closed);
• The horizontal angle (splitting) of all fingers (how much the fingers are spread);
• 3DoF for the thumb, so to better approximate its movements.

You may recall that actually, every hand has 27 degrees of freedom, so it is clear that this glove is not able to recreate the exact pose of all the fingers, but only a smart approximation. Anyway, this is enough to simulate many of the actions and the gestures needed in VR (e.g. grabbing objects, throwing them, activating a button, etc…).

https://gfycat.com/immaculateshowyafricangroundhornbill

For fingers tracking Dexmo chose an original approach and went for mechanical tracking, that is there is a mechanical sensor that measures each tracked angle for each finger. Aler here made me a very long and interesting explanation of the difficulties that there are in performing fingers tracking in a glove and the various methods that they tried before getting to this solution (that in his opinion, is one of the best on the market). Maybe I will explain that in detail to you if we will ever drink a bubble tea together in Shenzhen, but for now, let me just explain briefly what are the tracking alternatives on the market and their pros and cons:

• Optical tracking solutions like Leap Motion are great because they can provide all 27 degrees of freedom of the hands, but the tracking is still not fully reliable and in case of occlusions, it just doesn’t work. And you can’t use a method that sometimes doesn’t work in industrial environments;
• Commercial solutions like Oculus Touch or Valve Knuckles are good and cheap, but only provide 3/5 DOF for every hand (and with a flexion detection that usually does not go all the way… it is mostly on/off detection) and no real force feedback;
• IMU tracking systems may seem the best approach since they can offer up to 10 DoF per hand, but IMU sensors tend to drift over time and do not perform well in a system full of vibrations, metallic elements and electro-magnetical fields like a force-feedback glove;
• Flex-sensors tracking is another common approach, and while it offers good measurements, the continuous bending of the sensors reduces their lifespan and so they tend to break fast;
• Rotational sensors are very reliable, but this comes with a cost: they take up more space to set up.

https://gfycat.com/unfinishedmealyeagle

Dexta went for the last one: the mechanical tracking approach, that is more reliable and resistant. Regarding the big occupied space, since Dexmo has force feedback units that would take up space anyway, this is not a big deal (it would have been if Dexta thought to produce thin tracking gloves).

Aler proudly highlighted that while 11 DoF is not perfect, it is still better than most gloves and controllers on the market that usually are around 7 DoF.

He put much emphasis on thumbs tracking. In Dexmo gloves, the thumb tracking alone takes 3DoF, and this is needed to make the grasping and all the other virtual hands’ interaction more realistic. He explained to me that they put extra emphasis on the thumb because their physics engine rely on accurate position of thumb to generate realistic force, while most data glove doesn’t have feedback, so accurate representation is less relevant.

https://gfycat.com/eagerportlycoral

Dexmo haptic feedback

Currently, Dexmo is able to offer some haptic feedback thanks to tactile vibrations applied to the fingertips and palm of the hands of the user. These vibrations are made possible by LRAs (Linear Resonant Actuators), that are installed in the exoskeleton.

Their vibration gets propagated to the fingertips, to simulate for instance the sensation of touching a very rough surface or a smooth one. Since LRAs allow a specific waveform to be generated with adjustable amplitude, the haptic system is very flexible and able to simulate different haptic sensations.

https://gfycat.com/mildfatalichthyosaurs

Dexmo positional tracking

Actually, the glove is not able to detect its position in space, and that’s why in VR it has to be used by applying one Vive Tracker to it. The Vive Tracker must be positioned next to the thumb articulation: this choice has not been made by chance: being in that position, the weight of the Tracker is less perceivable by the elbow joint and so it interferes less with the movements of the arm and makes the user less tired.

What is incredible is that the glove can work with WHATEVER tracking system. You choose… and Dexta even made a demo using the glove with Oculus Quest controllers! I couldn’t believe it until I saw the video: Quest with Force Feedback must be overly cool.

https://gfycat.com/equalimpossibleblackfootedferret

Whatever tracking technology you choose, you have to make sure that the system is well calibrated: in some demos, I had the real and virtual position of the hands having a little mismatch and so the force feedback was not coherent with what I was seeing in VR. This caused some weird sensations in me.

Connectivity

The gloves are completely wireless: they communicate with the PC via a customized 2.4GHz dedicated network. You establish this network thanks to USB wireless modules plugged into the PC.

Theoretically they could work also via Bluetooth, but Bluetooth communication would add extra latency to the haptics system.

Battery

The nominal battery duration for the gloves is 6 hours.

Price

Dexta actually revealed me the price of the device, but I can’t disclose it to you. What I can tell you is that the price is actually enterprise-oriented, and most VR enthusiasts won’t be able to afford it for now. But companies interested in having it, will be able to afford it and use it for instance in their training programs and actually spare money thanks to the more effective training procedure.

https://gfycat.com/importantincomparableboto

What else can I say? The main competitor of Dexta is probably the HaptX haptic system, and Dexmo is far cheaper than it.

Hands-on

I have tried two models of Dexmo: the older one only applying variable force feedback employing direct-drive motors, and a new one that gave me haptic sensations on top of the force feedback.

Setup

There are three different wearing methods for Dexmo gloves:

1. Swappable rubber finger caps. This is particularly comfortable if this is a personal glove. Once you pick the right caps, swap it for the first time like you would with a pair of headphones, you may wear them very easily afterwards. In this case, you can wear the gloves on your own;
2. Glove solution. This is much easier to wear, but it comes with different sizes. When the glove is shipping in large quantity, people can just pick ones that fit their needs. In this case, you can wear the gloves on your own too;
3. Strips method. This is the most complicated yet the most universal method. It is guaranteed to fit most people’s hand, so it is ideal for exhibitions and events. But it is also the most uncomfortable solution and it usually requires external assistance to wear.

For logistics reasons (I was not in their HQ), Dexta could let me try mostly only the third solution, the most universal but also most complicated one. The set-up was not that easy. I had to wear the glove and then close one big strap on the palm to secure it to my hand, and then I had to attach every finger to the glove. With the strip method, every finger has to be secured to the exoskeleton using a little strap on the fingertip. So, it is very difficult that you set up the gloves by yourself: you must have someone else assisting you to close all the straps well. Especially because once you’re wearing one of the gloves, it is difficult to use your hand to wear the other one.

With the other two solutions, the setup should be easy like wearing a pair of gloves instead… especially with the number 2. I managed to give a shot to the solution 1 employing the rubber finger caps: there were special silicone thimbles where I inserted my fingertips. I can confirm that it was faster to wear and also more comfortable… but unluckily they were not of my size, so I needed some help in wearing them as well.

On the software side, for the VR tracking management the software used was Steam VR, and for the management of the gloves, there was a special service running on the PC that managed the hand tracking that had to be set up as well.

Every new user employing the Dexmo may choose a prefixed hand model that is similar to his/her hand or to calibrate for one time. Engineers at Dexta said that using the prefixed hand model makes the operation faster, because it avoids the calibration stage, but the personalized calibration guarantees better results.

For my experience, I was asked to perform a special calibration phase, for a better hand model fitting. I put both hands in three special gestures that let the system understand what are the typical angles of my fingers when they are opened and closed. The calibration required 1 minute or such… and after I performed it once, I had not to repeat it again, even when I switched between the different models of Dexmo gloves.

Design

I love the design of Dexmo gloves. They look like exoskeletons taken from some cyberpunk movie. Wearing them, I felt a bit like Wolverine, a bit like Predator, a bit like Robocop. CEO Aler Gu told me that they spent some time thinking about the design because they wanted to give exactly that kind of sensation. He said that even if it is an enterprise device, the user must feel “cool” wearing it.

The fact that when you wear it, you can clearly also feel some kind of movement of the internal motors, gives even more the sensation that you are some kind of cyborg.

Comfort

Dexmo gloves felt incredibly light and comfortable for having so much technology inside. Wearing them for some minutes hasn’t given my arm any kind of noticeable fatigue.

Anyway, I have to highlight that the type of fitting method impacted a lot on the perceived comfort of the glove. As I have highlighted above, I tried two models. One used rubber finger caps for fixation, another one used velcro finger strips.

The rubber finger cap wearing method was rather comfortable: the fact that you can wear thimbles that have the same size of your fingertips makes sure that you don’t compress your fingers and this means that you can wear the Dexmo gloves for quite long period of times. It was not as comfortable as wearing a pair of velvet gloves, but I was really ok while wearing them.

https://gfycat.com/equalwellgroomedfreshwatereel

But when I used the finger strip method, the strips were worn tight on my fingertips and palm to convey better the haptic and force-feedback sensations. This was squeezing my hand, and I could really feel the pressure of my blood rising in my fingertips. When I removed the gloves, there was some signs left on my hands. This was not that good. You may think that the solution is loosening the strips, but if you don’t close the glove tight, the risk is that it slowly moves over time or that one of the fingers comes out from the gloves (this has happened to me once).

With this method, there was also the problem that sometimes the straps of two consecutive fingers shortly attached one to the other and this was a nuisance.

Dexta told me the strip method is only used in public demos because there are a lot of people with different hand sizes in line, and so swapping finger caps seems a bit inefficient. However, they said that for more personal usage, whether industrial clients or personal users may choose to use the finger cap method. This way, you can customize it to make it comfortable to you and you avoid all the problems that I have highlighted above for the strip method.

Regarding the movements of the fingers, I want to highlight that they were also slightly slowed up by the little resistance applied by the motors, even if they weren’t applying any force feedback. The hand didn’t feel 100% free, of course… but I have to say that this was not a big problem most of the time. With the new model, this problem was also less noticeable than with the old one.

The company is reportedly keep improving its comfort design.

https://gfycat.com/disgustingequatorialarawana

Force Feedback

I have tried various demos about the force feedback, like for instance:

• A demo that let me close my hand over a sphere, a cube and other objects of different shapes;
• A VR demo that let me operate with different objects like a faucet or a fire extinguisher, being able to feel the contact with the other objects;
• A VR and non-VR/demo that made me hold a pumping heart in my hand. The heart, pumping, was able to make my fingers pumping as if I was holding it in my hands;
• A demo where various water drops spilled from the faucet onto my fingers, making them to move;
• A demo where I could squeeze a rubber chicken toy to make it sound;
• A short experience where I could press buttons with my fingers.

Experiencing force feedback in VR is pretty interesting. It was the first time for me trying it and it was really cool. Finally I could hold an object in VR without seeing my fingers entering into it. And no matter how much force I was applying against the virtual object, the glove was very strong and always blocked me (I was told that the glove has anyway a security mechanism that if you apply too much force, in the end, it lets you win to both protect the user and to prevent the force feedback units from breaking).

The demos that I loved the most were the ones featuring active forces: the drops falling on my fingers making them to move or the pumping heart. I could really feel some magical force moving my hands, in VR and in the real world… it was a fantastic sensation.

https://gfycat.com/courageousripekittiwake

Also, the variable stiffness pressure was cool: I was able to squeeze the virtual chicken rubber toy and feel its resistance in my hand… and it was interesting that Dexta operator was able to modify the stiffness factor on the fly, changing the resistance that I could feel. In the buttons demo, I was able to recognize the different buttons by the different way they reacted to my touch, by their different stiffness… that was amazing, because it added a new sense to my ability to interact with the VR world.

I guess that now I should tell you how the force-feedback was, giving you my opinion… but it is hard to explain the sensation that I had… but let me do my best. The effect was cool, and for the first time, I understood how VR could be with virtual objects actively applying a force on my hand. The falling drops demo was amazing, and I could have spent the whole day just doing that. But realism is still far away.

https://gfycat.com/wideeyedcooperativechamois

My brain easily got that the force was not real, essentially for three reasons:

1. The force always originates from the glove, that can apply it only on the fingertip, while in reality the force can be applied in different parts of the finger;
2. The force was not accompanied by a sensation of pressure on my meat, as happens when something pushes against my fingers. When trying the glove with no haptic feedback, just force-feedback, everything felt a bit unnatural: I was holding a sphere in my hand, I could feel his presence, but I couldn’t feel any tactile information about it. With the gloves with haptic feedback, things were a bit better, though;
3. Sometimes the collision detection glitched and so my fingers started having a touching/non touching intermittent state with the surface of the object and this made the system apply and remove the collision force to my fingers. It was funny, but also weird. This was a glitch of the software layer and not of the gloves, though;
4. I could feel a very slight lag between the moment I saw the object applying the force to me and the moment I could feel the actual force. It was little, but noticeable.

The overall sensation is of a synthetic force feedback. It works, but it doesn’t feel entirely natural. I guess we’re really too early for technology to give us that.

Haptic feedback

For the haptic sensation, I wore the new glove and tried a VR demo where I could move my finger across various surfaces. One was a plasma screen, one was a piece of wood, another one a surface with little rocks on it and so on.

Just by moving my fingers over it, I could feel my fingertip vibrating following the part of the surface I was touching. This was thanks to the actuators of the fingers that vibrating at different amplitude with different waveform gave me the haptic sensations. A bigger amplitude of vibration and a different vibration pattern corresponded to a rougher surface, of course. I could detect the different kind of materials, but I could easily spot that it was not always a realistic sensation.

For the plasma, it was overly realistic! I could really feel my hands as going through an elecro-magnetical field! For the other surfaces, the sensation was really nice, but again felt a bit “synthetic”… I don’t know how to explain it… I was able to easily distinguish what was the surface I was touching, but it was not exactly as in real life, it was a bit “approximated”.

I could see how this can be used in places like virtual touch screen and other screen-based interaction in VR. This definitely opens a lot of doors for more natural interactions in immersive worlds.

Final impression

I was able to test Dexmo gloves only for like one hour, but I come out impressed by them. It’s the first time for me with such devices, so I can’t compare them with other similar gadgets, but I can recognize something when it is well made.

These exoskeletons are cool to be seen, are light and are able to give you a good emulation of the force and the haptic sensations that you can have on your fingers. They can work with and without VR and all the tracking systems that you want.

Of course, they are experimental hardware that is trying to push forward the boundaries of VR, so they have some shortcomings as well: they are expensive, not so easy to setup and can’t reproduce exactly the real sensations of haptics.

But I was impressed by the massive R&D work that has already been done and talking with Aler I could really perceive his deep knowledge of the topic, so I am sure that they will evolve well in the future.

In case you are a company looking for a force-feedback glove for VR, I would recommend giving Dexmo gloves a try. In case you may need it, I can also help in getting in contact with the company (just ask me).

Thanking Aler, Zizi and Tianshu from Dexta Robotics for having me try this glimpse into the future, I really can’t wait for this kind of gadgets to become the norm in VR.