One of the most interesting announcements of last week has been the release of the HaptX Gloves DK2, one of the most interesting haptic gloves for VR. I am very intrigued by these gloves that can provide quasi-realistic haptic sensations to the hands of the users and that have been praised by all XR journalists. I’m so very proud that I’ve been able to interview Jake Rubin, the CEO of HaptX, about his company. He was very kind to sit down with me and detail how his gloves work, how you can buy them, and also provides insights on haptics in VR, answering about when we can expect haptic gloves to become mainstream. It is an interview full of useful and detailed information, so fasten your seat belts and read it all!
(Little Disclaimer: Jake is a reader and Patron of this blog. This hasn’t influenced anything about this article, but it seemed correct to me to tell you 🙂 . And anyway… why don’t you join him and become a supporter of my website as well?)
Hello Jake, can you introduce yourself and your company HaptX?
Sure. First off, thanks for having me! I’m a long-time reader of your blog, so it’s great to be here. I’m Jake Rubin, the Founder and CEO of HaptX. I started HaptX a little over 8 years ago with my co-founder Dr. Bob Crockett (for those interested in more details on the company’s origin story, this profile provides a good in-depth overview).
For those who aren’t familiar with HaptX, we build technology that simulates touch sensation with unprecedented realism. Our flagship product is HaptX Gloves, but we’re also developing technology to bring a realistic touch sensation to the rest of the body. There are several other companies developing haptic peripherals for VR and AR, but what sets us apart is our focus on realistic, true-contact haptic sensation. Our unique microfluidic haptic feedback platform is the only technology capable of actually deforming a user’s skin the way a real object does when touched, delivering natural and intuitive touch sensations.
What are the HaptX Gloves DK2 that you have just launched?
HaptX Gloves DK2 is a major upgrade to the world’s most advanced haptic feedback gloves. HaptX Gloves DK2 are the first gloves with true-contact haptics available for purchase. Unlike other haptic gloves which are limited to vibration and force feedback, HaptX Gloves physically displace the user’s skin the way a real object would. They deliver unprecedented realism, with more than 130 points of tactile feedback per hand. DK2 is designed specifically for professionals in training and simulation, industrial design, and robotics.
Can you give us its specifications?
Happy to. Each Glove features 133 points of microfluidic tactile feedback that can displace the user’s skin up to 2 mm. Not only is this far higher resolution than other haptic gloves (which typically feature 1-6 vibrating tactile actuators), but each actuator produces a much higher quality sensation. Rather than being limited to an unrealistic buzzing sensation, each actuator can vary its displacement and pressure by minute increments, actually pushing against the user’s skin just like a real object would. The actuators can also be vibrated at high frequencies up to 250 Hz to replicate fine texture and vibration sensations.
Our gloves also feature the strongest force feedback of any commercially available haptic glove, with up to 40 lbs. (175 N) of resistive force. Finally, we have the industry’s most accurate hand motion tracking solution. We have developed a custom electromagnetic tracking system that captures 30 degrees of freedom with an average precision of 0.2 mm RMS (Root-mean-square deviation). Other gloves use mechanical, flex sensor, and/or IMU-based tracking approaches which are less accurate and much more noise prone.
How does the tactile feedback work? How do you provide the “haptic texture” to the user?
This is really the most important “secret sauce” behind our gloves. Tactile feedback is far and away the most important haptic modality for dexterous manipulation of objects with the hands. There are four different kinds of touch receptors in your hands. Only one of these four types, the Pacinian corpuscle, responds to the kind of stimulus that force feedback and vibration gloves provide – deep pressure and high-frequency vibration. The vast majority of receptors respond to more nuanced skin deformation that conveys object shape, movement, and texture. Looking at the number of each type of receptor and their distribution across the hand, this means that even the best force feedback and vibration glove will only be able to stimulate about 5% of the receptors in your hand. Our 133 points of true-contact tactile feedback boost this to around 75%.
The technology that delivers this feedback is our patented microfluidic skin. It consists of hundreds of tiny air channels and bubble-like pneumatic actuators, embedded in a flexible, silicone-based smart textile. When the user’s hand touches a virtual object, our software determines the contact force and object surface geometry and turns this into a picture of exactly how the user’s skin would be deformed at each of 133 points of contact per glove if the virtual object were a real object touching their skin. The software then converts this skin displacement information to pressure commands for each actuator in the microfluidic skin, using a sophisticated, physically-based model of the interaction between the user’s skin and the pneumatic actuator “bubbles.” Finally, an array of piezoelectric valves in our Air Controller precisely dispenses air into each channel within 30 ms, much faster than the blink of an eye, to reproduce the commanded displacement profile in the actuator array.
What are the improvements of this DK2 with regard to the previous DK1?
Almost every subsystem has been redesigned and reengineered from the ground up for improved quality and reduced cost. Some highlights:
- Completely re-designed force feedback system that offers significantly higher peak force (up to 40 lb. per hand), higher stiffness, and faster response time in a more compact package
- Redesigned tactile actuator panels with improved actuator distribution (133 actuators per hand) and improved haptic quality
- Brand new motion tracking system with completely custom electronics
- Improved fit, ergonomics, and donning/doffing of Gloves
- Air Controller has been reduced in size by over 3x, enabling room-scale mobility with backpack adapter
- Multiple major SDK improvements, including multi-user networking and full-body IK
- A brand-new demo that is much larger in scope and offers increased graphical and haptic realism relative to our old “Farm” demo. Can’t wait to show it to you!
Perhaps the most important improvement is that we can now manufacture our Gloves at commercial-scale, enabling customers to purchase them. We were hand-building our DK1 systems in small volumes, so we were only able to make them available for lease, despite the high demand from customers. Now, with our new manufacturing and go-to-market partner, Advanced Input Systems, and the streamlined design of the DK2, we are able to produce the Gloves at a much larger scale.
Let’s get nerdy: can you give us a technical explanation of how you managed to reduce the gloves so much?
The Gloves themselves are actually pretty similar to the DK1 in overall form factor. We’ve streamlined the design and made them more comfortable, easier to put on/take off, and more ergonomic. They weigh just ~1 lb. (500g) per Glove, which is light enough that you don’t really notice them when you’re using them. Making the Gloves tighter to the skin and “glove-like” will be a major focus of our next release, so keep your eyes peeled!
The big change from DK1 to DK2 is in the supporting equipment that’s needed to operate the Gloves. We’ve reduced the weight of our Air Controller, which contains the valves that control the operation of all of the actuators in the Gloves, by over 3 times, as mentioned above. It’s now light enough that you can comfortably wear it on your back, like a backpack PC, for room-scale VR sessions. This dramatic weight reduction primarily came from two design changes. First, we completely redesigned the data acquisition and processing electronics for our electromagnetic motion capture system. We were using an off the shelf data acquisition system that consisted of two boxes weighing about 9 lbs. (4 kg) each. Our engineering team, with help from our partners at AIS, were able to reduce these two giant boxes to a single printed circuit board weighting only about 1.4 oz (40 g) that fits comfortably in the palm of your hand, while maintaining the same level of performance. That amounts to about a 500 times decrease in volume in just one product generation! Second, we redesigned the manifolds and electronics for the valves in our Air Controller. This allowed us to reduce the size and weight of the pneumatic system by quite a bit.
Miniaturization has been a big focus for us since we revealed our first prototypes in late 2016. You may remember our “magic box” prototype that generated quite a bit of buzz back then. It delivered very realistic haptic sensation, but it weighed over 180 lbs. (80 kg)! That weight mostly came from the compressor, valves, and other pneumatic equipment necessary to support the operation of our microfluidic actuators. We’ve already reduced the size of our system by orders of magnitude since those first prototypes, and we have an R&D roadmap that we’re confident will deliver further, similarly dramatic reductions in size over the coming years.
For more technical details, check out the technology page on our website.
What are the features of your gloves you are the most proud of?
The true-contact haptics delivered by our 133 microfluidic tactile actuators per Glove. This is a completely unique feature among haptic devices. It hasn’t been easy to develop and commercialize this technology. HaptX and our partners have invested hundreds of thousands of engineering man-hours in developing novel manufacturing processes, pneumatic components, and wearable technology. I’m very proud that we’ve been able to commercialize such a new technology in just a few years.
Even though true-contact tactile feedback is the biggest selling point for our Gloves, we still manage to deliver the strongest force feedback and the most accurate motion tracking. Our realistic haptic experience comes from the sum of these three industry-leading technologies.
And what do you think are still their drawbacks?
Cost and size. We’ve been pretty transparent about the fact that this is not a consumer product. It is designed and priced for demanding enterprise applications. Most of our current customers are large companies using our Gloves for mission-critical applications, and as you can probably imagine they aren’t cheap 😊
Our pneumatic approach to haptics delivers much higher levels of realism than our competitors, who use electromagnetic actuators. Pneumatic components haven’t benefitted from the aggressive miniaturization of electronics driven by the computing industry over the last 50 years, so it is more difficult to achieve a small, wireless system, which is, of course, the ideal form factor for haptic gloves. We can’t just buy cheap, off the shelf actuators and drivers. We have to develop almost everything from scratch.
As I mentioned above, we are continuing to invest aggressively in R&D to reduce the cost and size of our system. We’re already working on a wireless version, for example. I believe that in a few product generations, microfluidic haptic products like ours will be similar to electromagnetic haptic products in terms of size and cost while continuing to offer much higher resolution and fidelity.
What types of customers do you have? And in what industries?
We work with world-class companies and government organizations across the automotive, aerospace, industrial, medical, and defense markets, among others. Our customers primarily use our Gloves for three applications: design, workforce training, and robotics. As you can probably imagine, given the type of customers we work with, most have asked us to keep their names and work confidential. Here are a few examples of projects we’ve done that are public:
- Nissan – used HaptX Gloves to test tactile feedback in the design of car interiors from existing car models and also models still under development
- Volkswagen – used HaptX Gloves to research the benefits of using realistic haptic gloves in design interactions compared to VR without haptic gloves i.e., using controllers
- General Dynamics Mission Systems – used HaptX Gloves to demonstrate telemedicine and remote surgical assistance in a project with the U.S. Navy
- U.S. Army – used HaptX Gloves to improve the training of medical technicians performing tactical combat care procedures like tying tourniquets, using needles for decompression, and more
- ANA – used HaptX Gloves in the development of an advanced robot with real-time teleoperation capabilities
I know you won’t disclose the price, but can you tell us if it is above or below $10K?
Above. The reporting from several outlets that it is in the “five figures” is accurate. I would note that all of our competitors are priced in the thousands or tens of thousands of dollars as well. Everyone in the haptics industry is laser-focused on the enterprise right now because that’s where the market is. There just isn’t a feasible consumer opportunity right now (I’ll address this further below).
Is there a loaner program for companies interested in having them just for a limited time?
Yes! We have an evaluation program for brief loans and a lease program for longer duration rentals. Companies interested in leasing or evaluating our gloves can submit a request through our website: https://haptx.com/quote/.
Why are these kinds of gloves so expensive? Your competitors have similar prices too. Is it because of the materials, the R&D, or what?
Like virtually every new technology, from computers to cell phones, haptic devices tend to start off as expensive enterprise-oriented products before filtering down to the consumer market. The biggest drivers for the high pricing today are high complexity and low production volumes. Our gloves contain thousands of individual parts, many of which are based on brand-new materials, technologies, and manufacturing processes. Over time, increased volume, improved design, and additional capital investment will allow us to significantly reduce cost, but it won’t happen overnight.
Our competitors face similar issues, which is why the whole market has gravitated toward the enterprise. The enterprise market is the laboratory that allows new technologies to be iterated, improved, and cost reduced before they become available to consumers. This is the way it has worked for virtually every new technology in the modern era. It’s not an issue that’s unique to VR or haptics.
When do you think that this kind of glove can become more mainstream? I mean, with good ergonomics and a price below $500?
I don’t want to disappoint any of your readers who may be anxious to own their own pair of haptic gloves, but I do want to set realistic expectations. It will take years of development and hundreds of millions of dollars of additional investment to bring a haptic experience of the quality of HaptX Gloves down to a sub-$500 price point. I anticipate that consumers will be able to use our Gloves in a location-based entertainment setting, like an arcade or theme park, within the next couple of years (once the LBE industry recovers from the pandemic, that is). High-end consumer/prosumer pricing may be feasible in less than five years, depending on the evolution of the consumer VR market. True mass-market consumer pricing is almost certainly more than five years away.
I have no doubt that consumers will see multiple startups tease “consumer haptic gloves” with enticing price points over the next 5 years (indeed, some already have). However, these companies will either fail to bring products to market, given the enormous capital requirements for mass-market consumer hardware, or they will produce a product that delivers a disappointing experience that leads to very little adoption. This is simply a function of the state of the art in haptic technology and the dynamics of the consumer VR market right now.
Finally, I will note that the “big guys,” like Facebook, are actively working on developing haptic gloves. Indeed, Michael Abrash has repeatedly teased Facebook’s R&D work in this area and discussed the enormous importance of high-quality haptics for his vision of the future of VR. However, he has also been clear that it will be a decade or more until Facebook brings a haptic glove product to market on their current product roadmap. Clearly, it isn’t just a resource issue (Facebook has near-limitless resources), it is a complex and fundamental interplay of technological realities and market dynamics that will take time to play out.
Tldr: a high-quality consumer haptic glove is definitely coming, but not in the next 5 years.
I remember the old times of AxonVR (Pepperidge Farm Remembers!). Why have you abandoned that vision of full-body haptics and will you ever return to it?
Ah, the good old days 😊. We certainly haven’t abandoned our original vision. On the contrary, we and our partners are investing more heavily in full-body haptics than ever before. This effort recently received a big boost when we were awarded a $1.5m, multi-year contract from the NSF to assist in the development of our full-body haptic feedback technology.
Like most ambitious startups, we found that we needed to start with a minimum viable product to find traction in the market before pursuing our full vision. Now that we have that traction with HaptX Gloves, we’re free to expand our product line while we continue to iterate and improve on our Gloves. I can’t share details of future products, but rest assured we have no intention of stopping at the hands. We will bring true contact haptics to the full body, perhaps sooner than you might imagine 😊
What are your future projections?
We have a detailed R&D roadmap for the next few years that involves two parallel paths. On one hand, we’ll be working to reduce the cost of our Gloves and increase production volumes. This will make our best-in-class haptic feedback less expensive and more accessible, enabling us to bring our unique true-contact haptics to a wider variety of customers. For the foreseeable future, we’ll be focused on enterprise; however, we do eventually plan to introduce some version of our Gloves to the consumer market. How soon will depend on the growth trajectory of the consumer VR market and the level of development resources available to HaptX.
On the other hand, we’ll be working to make our platform even more immersive, expanding true-contact haptic feedback to other parts of the body beyond the hands. We’re totally committed to our founding vision of fully-immersive “Ready Player One”-style VR, and we won’t stop working toward this goal until the virtual world becomes indistinguishable from reality.
If you had to teach to the community one lesson that you’ve learned all over these years, what would it be?
It’s kind of a cliché, but they don’t call it “hard”-ware for nothing 😊. New, innovative hardware is one of the most critical factors driving the VR industry forward, but most people don’t appreciate how hard it is to bring a new hardware product to market. It typically requires tens of millions of dollars of capital and years of development, and even then success is far from guaranteed. Even absurdly well-funded hardware startups like Magic Leap have struggled to deliver compelling hardware products to market as scale.
That’s the bad news. The good news is that there are so many passionate, brilliant entrepreneurs and scientists all over the world working on incredibly exciting VR hardware projects. So many smart people are dedicated to the vision of fully-immersive VR, and virtually all of the technology we need to realize this vision is out there (albeit much of it in an early, pre-commercial form). It gives me a lot of hope for the future of our industry.
One of my favorite things is talking with people who are working on a piece of this vision. Sometimes there may be a way for them to work with HaptX directly. Sometimes I can simply learn something from them or offer advice and connections. Either way, I encourage anyone who fits this description to reach out to me. I try to respond as quickly as possible to messages on my LinkedIn, which is usually the best way to reach me.
Anything else to add?
Thanks for covering HaptX, and thanks for doing such a great job covering the latest news in the VR/AR industry. Keep up the good work!
And that’s it. I thank Jake for the time he has dedicated to informing the community, and I invite you to comment here below or on my social media channels with your thoughts about haptics in XR!
(Header image by HaptX)