Today’s article is one of the most beautiful published in this blog in 2022. And in fact, it has not been written by me.
XR ergonomics expert Rob Cole published last year on this platform an article detailing Project Caliper, that is his experiments in building fully configurable controllers for SteamVR. The article soon gets viral, and many people would have loved to see these controllers go into production.
This year, as a Christmas gift for you all, he’s publishing the results of the year-long evolution of Project Caliper that lead to the creation of Project Modular. Project Modular is the design of XR controllers that are fully modular, extendable, open, and configurable. They are the dream of every XR tech enthusiast. In this post, you will find all the details about them, including the story of the long journey of the many experiments made by Rob, provided through a long descriptive text, and many interesting pictures.
Rob’s work is amazing and I sincerely think deserves better recognition than just being on this website. It deserves to become reality. I invite you all to try to push his work by either giving it visibility, or opportunities to make it come to life. I’m all in supporting him, and I’m sure you will be too after having read about his work and his vision. Happy reading.
Project Modular
by Rob Cole for Skarredghost
Introduction
This time last year, Tony a.k.a. Skarredghost, was kind enough to publish my long-read article detailing months of intensive work on a modular motion controller focused on input options, ergonomic adjustment, and ease of repair.
The article quickly went viral, and numerous meetings were held with different companies interested in my controller design. They offered several small amounts of cash, some dubious cryptocurrency, and parcels of virtual land in two different metaverses, but nothing that would fund the pre-production work and production tooling / assembly setup.
A lengthy Zoom meeting took place late one evening in January, with a group of legendary hardware developers from a highly secretive outfit. Out of respect for their privacy, my recollection of that meeting remains vague, though I did ask what everyone wants to know.
I bought another Index the very next morning, so there’s the answer to that over-asked question.
About two months after the article first dropped, an invitation was received to my final Zoom meeting, this time with a corporate representative. Excited about a potential offer, I agreed to attend, but unfortunately, I found myself facing an aggressive legal challenge over an aspect of the design.
After signing whatever they wanted, to avoid being sued into oblivion, the project was canceled and I was asked not to talk about the design anymore, so that’s all I can say about that.
Every cloud does have a silver lining though, and I suddenly found spare time to play VR games, with my new VR system and a ridiculous “Hall of Fame” 3080Ti graphics card I’d somehow acquired at a very reasonable price.
Being able to turn up frame rates whilst maintaining super-resolution was absolutely glorious, and it was great using lighthouse tracking again. Finally, I got to finish Arizona Sunshine, and played lots of other games that had been sitting unplayed in my Steam library, damn those tempting sales…
Reboot>
Despite thoroughly enjoying my newfound life of leisure (outside of my real job), an idea that had been gestating in my brain for months led to that all-familiar hardware itch needing to be scratched.
It was time to get busy in private, working up an idea for an unconventional way of building a motion controller. Keeping things under wraps would hopefully prevent further legal shenanigans, whilst eliminating the time burn that is promoting something through social media which during the previous project meant answering emails, tweets, and whatnot.
So what was this new idea then?
Conventional designs like the Oculus Touch, Valve Index, and HTC controllers are manufactured using relatively thin-walled, injection-molded plastic casings, usually split into two halves. It’s been said numerous times that once the injection mold tooling and assembly setup is paid for, the plastic is so inexpensive, it’s essentially free.
The remaining internal volume (empty space inside the casing) can be used to accommodate numerous components such as electronics (tracking, compute, and GPIO), a power source (battery), rumble motor (haptics), input mechanisms (switches, joystick), flexes and cabling to connect the components, and so on.
This allows smart designers to make cost-effective products using mass production processes: injection molding providing plastic casings, automated assembly fitting out layers of components, secured with machine fixings, adhesive tapes, hot glue, etc.
Injection mold tools are complex and expensive to cut but can be used to create many useful internal details such as reinforcements, mountings, bosses, struts, webs, and spigots to support the components inside the controller.
It’s a very neat solution designed specifically for mass production, but due to the high entry cost and need for factories to operate at capacity, not at all viable for smaller runs. On the flip side, 3D printing is too slow to support mass production, at least using current printing technology.
Injection molded casing controllers have one big flaw; due to the thin casing, complex assembly, and fixing adhesives they were not designed to be repairable. It’s not easy to replace a damaged joystick, battery, or trigger assembly: you’ll often find a small fitting screw is obscured by assembly adhesive, replacement of the joystick means losing capacitive sensing, or broken triggers are repaired using superglue as access is tight and spares are scarce.
Because it’s not easy, it’s not very cost-effective to repair. By the time the product is shipped to a repair center, a technician is paid an hourly rate to fix it, parts are used and the product is shipped back, it’s sadly often cheaper to supply a new controller and just recycle the damaged one.
Consider that a single Index controller only costs £139 brand new with free shipping, making repairing it a debatable activity for any commercial entity. Compare this to a $150 screen for a $1000 smartphone, and you can see why a plethora of “Smash” smartphone repair centers have sprung up in many high streets and malls.
However, the creation of technical guides and replacement parts from the recent partnership between Valve and I-Fixit is a great step forward, as it’s “cost-effective” to self-repair if you have the skill, and tools and can source authorized replacement parts. If you can solder competently and aren’t scared of opening your controller casing, it’s certainly possible to make repairs, with some users reporting success with fitting an alternative joystick, or a new battery.
From my experience as an Index launch day owner with multiple controllers RMA’s during two years of EU warranty coverage, it was typically a single point of failure each time. I’d return a controller with a drifting joystick, a sticking trigger, a cracked grip plate, and so on, but not with multiple failures on the same controller.
I always received brand new controllers (really easy to spot) as replacements providing a good customer experience. Later now during the Index lifecycle, there are users now mentioning receiving reconditioned controllers which is a big improvement in terms of E-Waste. My personal E-Waste allowance (if there was such a thing) was completely blown during those two years, making me feel bad about all the shipping and replacements from China via the Netherlands to the UK.
All of this soon led me to believe that a design that offered an easy replacement of parts that are prone to failure or accidental damage would be well worth investigating. If the parts could be easily replaced by the user, or by someone offering such a service in the country, it would drastically cut down the number of faulty controllers being shipped long distances to repair centers (in my case, from the UK to the Netherlands) or just scrapped if out of warranty.
The Core Concept
As an alternative to injection molded casing controllers, my idea revolved around a “core” containing a Tundra SteamVR HDK (hardware development kit) and a slim lithium-ion battery pack
I planned to find out whether it was possible to “package” these items inside a hand-held core which would form the mounting structure for hand grips, a tracking frame (to mount sensors at appropriate positions), and input components. Was it possible to have these components “hang off” the core, if the core was used as a structural entity?
I quickly discovered it was possible to mount the HDK and a Li-ion battery inside a structure that would be compact enough to fit inside a small size, closed hand. After some tests with heat-molded plastics and threaded metal inserts (to allow attachments), something more rigid, stronger, and lower profile was required so I switched my thinking, looking at military assault rifles for inspiration.
I thought about an idea I had used on a previous project of a “plastic grip” that bolts onto a “rail” aping the infamous ‘Picatinny’ rail used for setting optics and other attachments onto an assault rifle, as seen in the image above. The Picatinny rail standard allows compatible components to mount in seconds and has created a rich ecosystem of compatible parts from multiple manufacturers.
Building aluminum alloy rails and plastic bolt-on grips seemed to work, with the Tundra HDK and Li-ion battery snugly accommodated inside the hand grips. The problem now was working out how to attach the rest of the controller to the hand grip!
Working on this problem soon revealed the solution of lengthening the “rails” to provide attachment points at each end for mounting all the other parts onto, including the tracking system.
I then built an aluminum mule with plastic grips to get an idea of the size and shape of a tracking ring that could fit around the hand whilst having clearance, and how input components like a joystick and trigger could fit into the design.
The Grips would encapsulate the Tundra HDK and its multiple flexes, so I tried fitting it to the Mule to check for clearance, tight but would work, as shown below with the Upper Grip fitted and flexes coming back towards the tracking ring side.
Having recently tested Pimax’s new Sword controllers whilst making my review for Skarredghost, I developed a new appreciation for touchpads so decided to build a placeholder around that style of input, to keep it simple whilst the rest of it got worked through.
An unfortunate (brand new, boxed) Steam controller was sacrificed to the hardware gods, and stripped for parts including trackpads, buttons, and triggers.
I was back in the controller game with a new idea, and moved quickly, working on what would become the first-generation Modular controller.
Swapping the Touchpad input module out for a joystick module should be possible, at least on the hardware side. Was it possible to also adjust the distance between the grips and input module to accommodate different hands?
Reworking the Rail system made me think more about how to explain what it was, coming up with this description: “Modula Rail Frame”.
“Modular Rail Frame”
It seems to make sense: this could be called a “Modular Rail Frame” controller, in comparison to contemporary “Injection Moulded Casing” controllers we currently use in VR, like the Oculus Touch and Valve Index (for those who like abbreviations, MFR controller and IMC controller respectively).
An essential function of the Modular Rail Frame design is to provide the flexibility of a modular system that can accommodate many different options. Different size and shape hand grips bolt on in seconds. A longer strap can be fitted. Different input options should be available to suit the preference of different users… for teleport people a decent full size touchpad is much superior. And if you like fighting games, what about a D-Pad? If you want to have a touch (force) sensor on your lower grip, it can be done; but if you prefer a conventional grip button, that could be done too. Just bolt it in, and connect that cable!
To make this change on an IMC controller is not inexpensive as it requires the injection mold tooling to be substantially reworked, a new tool cut, or changes made to the assembly line.
With the MRF design, new modular parts could be quickly printed or manufactured at a much smaller scale with a shorter lead time for customers. New games could be released with specifically made MRF components to work in conjunction with game code, more on that later 😉
Anything that gets broken on an MRF controller can be removed and replaced without requiring complex disassembly; loosen some bolts and unplug a connector, then put it back together. Software tools would enable adjustment to render models of the controller to suit input options choice, color, mods, etc.
Rebuilding should be a less frightening proposition than taking apart an IMC controller which can involve heat guns, plastic picks, and tiny Torx drivers and require steady hands and strong nerves, especially during your first repair “adventure”.
Now that I was onto something tangible with my radical idea of MRF controllers, several months of intensive work went into figuring out how this could be put together, and most importantly thinking if it could actually be manufactured at any scale.
Using the MRF design concept eliminated a big initial setup cost for manufacturing which was a set of complex injection molding tools to make the controller casing. Was it possible to make this thing using aluminum rails with some nuts, bolts, spacers, and perhaps some 3D-printed plastic parts? If it was possible to hand-build a working prototype in my workshop, it might be possible to build some more, at least on a limited scale. It was time to find out!
Development
This time around, it became much easier to know how to build stuff quickly and generally get it right the first or second time. Some of this came from experience working with certain materials and tools, whilst also having the confidence to get around problems after having punched through many similar problems last year.
Time was always an issue, there just wasn’t enough spare time as my real job was always busy. Thankfully I managed to work some shorter weeks towards the end of the year, having accrued some vacation time, which saw a big acceleration in what I was able to accomplish.
Something else I managed was getting better at doing more in less time, often being able to break tasks down into small chunks, or figuring in time for adhesives or paint to dry whilst working on other tasks.
Since a picture is reportedly worth 1000 words, here are some images from Project Modular development during 2022.
Finally, by the end of this Summer, I had something ready for prototype assembly. The plan was to fit the Tundra Steam VR HDK, write “the JSON” and get the tracking system running; then start adding inputs through the GPIO system, and see what happens!
The first prototype was designed specifically around Tundra’s HDK, a slim lithium-ion battery, and commercially available input components that I had scavenged from numerous gamepad controllers. I’ve always been keen to use COTS (commercial off-the-shelf) parts as the parts already exist, reducing setup costs and risk, and as the saying goes: why reinvent the wheel?
Especially when there’s someone making else “great parts”, handily they already have a factory and they’ve ironed out the production problems of making their component long before you come along with an order. Unless something is proprietary, most factories are happy to take money from new customers or can provide a similar product with some small changes, for the right price.
Talking of components, the concern about the supply of Tundra’s HDK was always in the back of my mind, as it had been out of production for some time, and it was uncertain whether it was going to be available again. Tundra had been very busy with their well-received Steam VR tracker, showing that despite the component shortage that started during the Covid pandemic and since worsened, it was possible to get something built if perhaps the order numbers were adequate to make it worthwhile.
Without further delay, let’s introduce the result, starting with an unboxing of what is supposed to be the first working prototype of Project Modular.
Project Modular V.1 unboxing
Coming in a 100% recycled cardboard box, Project Modular V.1 controller arrives unassembled with each component in a separate, labelled bag to keep it safe during transit. Once the controller is assembled, the bags can be stored away and the box used to keep the controller itself safe.
Taking the eight different bags out of the box, we have:
- Tracking Ring
- Tracking Head
- Input Pod
- Haptic Hand strap
- Frame
- HDK on upper rail
- Upper grip
- Lower grip
Taking these items out of the bags reveals more detail, as to exactly what is what.
The image above shows each item in more detail, but let’s go through each item in turn, as it helps to explain how this strange-looking motion controller fits together.
You’ll need a couple of hey keys for assembly, in this case, 2mm and 3mm. If fitting out the prototype with a Tundra SteamVR HDK the main difference is the careful routing of the sensor flexes through the cable routing system, whilst doing the assembly. Here we will show it without the HDK, as it makes it easier to understand.
Rails
At the heart of the controller, these aluminum alloy Rails have different holes for mounting attachments – we have a “Upper Rail” and a “Lower Rail”, with a forward orientation indicated by etchings on the surface.
The magnified image below shows the Tundra HDK fitted into an early incarnation of the Rails, where the retaining nuts for the front and rear axle bolts can be clearly seen. The threaded holes further inboard of each axle is used to mount the plastic grips. Over time, I had issues with these threads becoming worn by constantly rebuilding the development controllers and later switched to steel inserts for long-term durability, with only a small weight penalty.
The image below shows the Tundra HDK mounted onto the first version of the top rail, using VRB tape, which makes for a semi-permanent fitting.
At the top and bottom ends of the rails are the Axle mounting holes – the axles (long threaded bolts) provide a vertical mounting surface for attaching different input / output components. Keeping it simple, we have some black aluminum spacers, and not shown in the image, some spacing washers and retaining nuts which can be nyloc or plain.
The Lower rail is not shown here, it’s part of the lower grip in the v.1 design, and has a grip sensor bolted straight onto the metal rail (more on this later).
Frame
Tying together the Rails is the Frame, providing a structural brace as well as a mounting plane for the tracking system’s optical sensors. The Frame is manufactured using a recycled aluminum alloy (7000 series) material which is formed into a skeletal structure, with mounting points for attaching the Frame to the Rails, and to the Tracking Head.
As can be seen in the image above, the aluminum Frame is then co-molded using special heat-molded engineering plastic which is 100% recyclable and non-toxic. This plastic provides a smooth surface around the user’s hand, and adds a noticeable degree of strength and stiffness compared to the aluminum skeleton on its own.
I arrived at this special shape through many iterations as it also houses flex cabling for the optical sensors of the tracking system, and could not interfere with the user’s hands around the input area, whilst allowing a wide range of ergonomic adjustments for the input controls.
Grips
Bolted onto the rails using four countersunk bolts are the Grips. The Grips are the human-hand interaction surface, allowing the user to comfortably find, hold and grip the controllers; and apply finger movements and grip force through the different controller inputs. The Grips also provide a secure casing for the controller internals, cabling and optical sensor flexes.
The Upper Grip is a simple heat-molded plastic piece, with two countersunk bolts. The front bolt is longer as it also acts as a mounting point for the front of the hand strap. The rear bolt is slightly shorter, providing an adjustment mechanism for the rear of the hand strap (more on this later).
The Lower Grip in the v.1 design has a pressure-sensitive grip sensor bolted on from the inside face, with a cable running into the controller interior where it can interface with the GPIO system.
Using the same style and size of bolts as the Upper Grip, the Lower Grip uses shorter length bolts so as to not penetrate the interior of the rail underneath where the battery is located. The Grip Sensor is bolted in place using two small countersunk bolts and can be removed for replacement or repair, although this requires opening the Rail Frame on the V.1 design.
Haptic hand strap
Still talking about Grips, these play an essential second role which is retaining the Haptic Hand Strap to the controller body, whilst providing angle and length adjustments for the Hand Strap to suit the user.
The Hand Strap is firmly elasticated, allowing the user to operate the controller without constantly gripping the body, either for comfort during longer sessions or during “experiences” requiring no input, or during operational actions such as throwing an in-game object.
The Hand Strap is a simple woven elastic fabric piece, one end heat thickened and punched through with a small hole for the Upper Grip front axle, the axle acting an anchor point and pivot to set Strap angle.
The front of the Hand strap is angle adjustable, by simply loosening the Upper Grip bolt a couple of turns, rotating the strap to adjust the exit angle, and then re-tightening the bolt which compresses it against the Upper Rail.
The rear of the Hand strap is secured by compression between the rear of the Upper Grip and the Upper Rail, making length adjustment as easy as loosening or tightening the corresponding bolt using a hey key. Pull it to length, tighten the bolt, and it won’t move again.
This makes for a super simple strap, that’s easy to adjust or replace if worn, but also a mounting point for a small Haptic Module that slides onto the strap, and wires into the controller interior using a snap in connector (not shown here). Boasting a very small but punchy ERM haptic motor (max 3nm), mounted onto a molded plastic body, with a carbon fiber safety shield on top, and a firm rubber skin contact base.
Discovered during last year’s development work, when I built a haptic test mule called “squirrel”, I noticed that firing even low-power haptics straight into the hand’s metacarpals provides a very powerful “in hand” stimulation, but only required a low-power budget (compared to using several larger, in-body motors). This is very interesting to prevent quick discharge of the battery during haptic-rich applications.
Another very interesting finding was that de-coupling the haptics from the tracking system improves stability, as some Index controller owners have reported problems with tracking when the haptics fire. As recently seen on the Index Reddit:
“I bought Full kit in November. Got it around 14th. Left controller was spasing out and kept losing tracking during strong vibrations (right was perfectly fine). Exactly the same problem that was happening for left controllers since the device has been released. I had to go through back and forth for over 2 weeks to even get offered a replacement. I only got that replacement last Wednesday. Right away I’ve noticed the controller was jittering slightly during vibrations but at least it wasn’t losing tracking completely nor was it hard enough to completely displace it by 10-15 cm so it was bearable.”
As the haptic motor is no longer mounted inside a plastic controller body (causing vibration throughout the structure) but floating on the Hand strap, potential improvements to tracking in all conditions, combined with lower power budget make it an interesting new development 😉
Input pod
To provide a useful range of inputs in a compact, adjustable package there is a self-contained Input Pod, this version boasting a high precision analog mini-stick, which can be quickly replaced after unscrewing five small bolts from the bottom of the Input Pod casing.
Shown here below as the design was being worked on (with a placeholder cable) the mini-stick also features a removable rubber joystick cover in concave (seen here) and convex versions to suit user preference and allow for wear and tear replacement.
With a glide-ring, metal construction and high specification pots (not Alps) you get a super responsive and precise input feeling, with increased return-to-center precision. It’s rated for 2 million activations so takes some time to wear out, and it is replaceable for around $20… not bad.
Most importantly, when testing this joystick in VR games like Aircar and Project Cars 2, the biggest difference was noticed in “stick feel” compared to current motion controllers, providing a much more accurate, direct, and less vague stick movement experience.
Four high-quality input buttons are also featured in the casing, with industry-standard “A, B, System” as well as an additional user-definable input button here just called “X”. This gives the controller an additional input channel, similar to the action of pressing the touchpad on Index controllers, which is used in a number of games.
With SteamVR boasting a highly configurable controller input system, it should be easy to remap controls as you require and find the best use for the X button.
Input buttons on glide rings compress high-quality microswitches mounted on a rigid aluminum alloy baseplate that bolts into the bottom of the molded plastic Input Pod body.
For definite tactile feedback, the use of high-quality microswitches provides a pleasing but not overly loud “click” when pressed. To the left of the Input Pod in the image above, is the Trigger and Input Pod’s Mounting Bracket.
The Trigger itself is a mechanical device with no electrical connection, featuring a robust pivot and travel adjustment screw which is adjusted using a 2mm hey key. This particular Trigger started out as part of a Steam controller and has been “adapted” to suit this application.
The travel adjustment screw in the base of the Trigger aligns with a microswitch mounted underneath the Input Pod casing. As the Trigger is worked by the user, the screw makes contact with the microswitch, providing an input action. By adjusting the screw height (with hex hey), the Trigger makes contact sooner or later with the microswitch, depending on your preference.
This allows the Trigger to be easily replaced or removed, without any vulnerable cable linking the Input Pod to the Trigger, or the need for any switching gear inside the Trigger.
And to satisfy my demand for ergonomic fine-tuning, the Input Pod and Trigger slide back and forwards 15mm on the steel bracket using single bolt adjustment with a Hex key, allowing users to fine-tune distance between controls and Grips to suit their fingers or personal preference.
15mm of adjustment distance seemed useful, though it would be possible to increase this distance by reworking the front of the controller to increase the distance between the Front Axle and Tracking Head at the cost of slightly increasing controller length and weight.
By slightly loosening the Front Axle bolt, you can rotate the Mounting Bracket around the Front Axle, letting you set rotation of controls relative to grips (before tightening the bolt to lock it in place), which feels like a really useful adjustment to have. This would translate to being able to set the Joystick angle just right for your thumb.
The only limit of rotation is the clearance between the sides of the Input Pod and the neck of the controller / side of the Tracking Head. The current design offers a good range of rotation adjustment, as further angle adjustment may be outside of a usable range for many hands.
The Mounting Bracket is a thick piece of strong Chromoly steel plate and can be carefully bent to your preference using a bench-mounted vice and grips, to allow users further customization. Alongside using longer or shorter Axle bolts, different spacers, and different Grips, you may want or need a slight tweak of the bracket to put your Controls in just the right place.
However, it will not bend in normal use, though could act as a “bending fuse” if somehow the user subjected the Input Pod to an abnormal impact load. (and then be replaced at a cost of $2)
Tracking ring
This molded plastic piece acts as a protective shroud that contains all the optical sensor flexes as they are routed from the HDK through the body of the controller and to their individual mounting locations as described by three-dimensional model Points.
Sensors are protected from environmental impact by robust surrounding edges and ridges molded around each Sensor location, designed to give the Sensor its full field of view whilst preventing direct impacts from causing any sensor damage or unwanted movement which can throw off calibration and therefore cause tracking instability.
As seen in the image above, the Tracking Ring bolts onto the Frame using four small bolts (only three shown here) which are fitted by hand using a Hex key. Once secured with these bolts, the Tracking Ring sits flush against the Frame, providing a neat solution for mounting sensors and retaining flexes away from the outside world, whilst presenting a hand-friendly interior surface and being robust in case of impacts.
Optical Sensors are secured using VRB tape or adhesive after threading the sensors through small “portholes” strategically moulded across the Tracking Ring surface. As seen in the image above, I’ve placed a detached (accidentally damaged) sensor where it would be located, the flex then runs back down through the nearest porthole and inside the Tracking Ring, back down to the Modular Rail area.
For production, it would use optical windows made from special plastic, and perhaps ultimately a “bolt-in Tracking Ring module” with a single connector, to make assembly and repair super simple.
Tracking head
Like the Tracking Ring, the Tracking Head is a simple molded plastic piece that provides a physical structure for the array of optical sensors serving the front of the controller.
Using a separate head allows the Input Pod to remain adjustable, as the design provides frontal space for the Input Pod to move back and forth on its sliding mount with up to 15mm of adjustment for hand-fit fine-tuning.
It’s possible to increase the adjustment range (for example 20-25mm) by reworking the front of the controller with a longer reach for the Tracking Head, at the expense of slightly increasing controller size; 15mm was felt to be a useful adjustment range in conjunction with different size Grips.
As seen in the image above, the Tracking Head is attached using two medium-sized bolts which simply pass through holes in the front of the Head, and then screw into heavy-duty, threaded steel inserts which are bonded to the skeleton, and then co-molded inside the Frame structure.
The v.1 controller has the addition of an impact bumper, made from a thick piece of lightweight aluminum alloy. This reduces frontal impact damage to the controller as well as protects optical sensors from individual damage. It’s cut in a shape to suit the optical sensors, hence the odd shape.
The molded ribs on the Tracking Head continue this theme of protecting optical sensors from user-caused impacts with play space walls or furniture. For production, like the Tracking Ring, it could be re-designed with optical window plastics, or the best option of a bolt-in “Tracking Head module” with a single connector for simple assembly and repair.
During hand testing and movement, it was quickly noticed that this new design of controller with a separate tracking head also protects the user from any play space impacts, having bloodied my hands a number of times over the years this is most welcome. I had no issues, or fear for the strength of the controller, but watch out for furniture or computer screens as it will put a hole straight through whatever it hits!
Cable routing
Finally, we have a bag with some cable routing parts to keep cables tidy across the controller and prevent unwanted interactions with user’s hands. The blue and yellow cables shown here are placeholders to help work out the cable path and what was needed to retain them in the correct orientation.
V.1 Assembly time?
Originally, I planned to show the assembly of the v.1 controller with the Tundra SteamVR HDK, until a horrible realization was made during a practice assembly run. I had made a small calculation error when originally working out how much forward space was needed to attach the USB-C cable 🙁
This calculation work was done months before, and I’d even had a sneaky feeling in the back of my mind that there might be clearance issues, but had failed to double-check until my error became obvious late in the day. Plugging a USB cable into the Tundra HDK was very different from plugging one into a production controller, which has its USB port located on the outside of the casing, usually at the bottom of the body grip
Tundra’s HDK featuring the USB-C port on its forward-facing edge (when building a left controller) and with the HDK mounted inside the controller meant the port was hard to access with an external cable.
Short by just 1.5mm (close, but no cigar), this clearance issue prevented a USB cable from being attached to the HDK for SteamVR integration and calibration. I couldn’t move the HDK further back as it would have squashed the GPIO flex against the rear rail and prevented any GPIO connections from being made.
I thought about doing it without the controller partially assembled (no front axle, to make space for the USB cable), but was immediately concerned as this would throw out calibration. The other option was shaving down the USB cable head, mounting the HDK at a slight angle and then leaving the USB cable hanging out of the controller, which wasn’t at all desirable for a wireless device.
So here we are, some images showing the Modular v.1 controller, as far as I could get before banging head-first into this show-stopping problem. A heavy lesson was learned in this instance, ouch!
At this point I felt like giving up, as it meant starting the entire build from scratch with dimensions to suit the USB cable access problem; but being a solo developer, there is no one to hear you cry into your third cup of coffee before breakfast 🙁
Following my horrible discovery, I quickly righted myself, as with every roadblock there was now an opportunity to make improvements going forward, if I put in the time and effort.
Several months of hard work later, involving way too many early mornings, many more cups of coffee, and long hours on top of working a very busy full-time real job, the v.2 controller prototype was created, and ready for unboxing.
Project Modular v.2 unboxing
Opening the 100% recycled box, we find a number of components inside different bags. I’ve put these out on the workbench for easy identification. If you were to buy this as a “kit” this is how it would come, but it could be possible to provide pre-built versions by using an assembly and repair service for each continent.
The second controller shares the same MRF concept as the first, but with further refinement, improvements for cable routing, better adjustment, and ease of manufacturing.
During the process of building the second controller, I started thinking a lot about how this could be manufactured, and if it was possible to open this idea up to everyone. The outcome could be a potential new open standard, for manufacturing this style of MRF controller…
The most immediate change is to the Rails which are longer, slightly narrower and now have threaded steel inserts bonded into the aluminum, with a serrated nylon spacer above each insert, which provides a solid interface for each Grip. The threaded inserts prevent damage from repeated tightening and loosening of the Grip bolts, even if the user is a bit clumsy.
The Upper Rail has an aluminum alloy mounting plate for the Tundra HDK, which simply sticks onto the plate using high-strength 3M brand VRB adhesive tape. This allows some adjustment following initial contact, before quickly hardening to create a near-permanent bond (it can be removed very carefully using a scalpel).
The mounting plate for the HDK is fitted using 2 small countersunk bolts so that the plate can be removed with the HDK still in place if required for maintenance or replacement.
The Lower Rail as seen in the image above has a small Touch sensor, with a ribbon cable for connection to GPIO. The Touch sensor protrudes through an open slot in the Lower Grip, allowing the user to easily work the input, whilst being independent of the Grip itself.
The touch sensor itself can be removed in seconds by loosening two small mounting bolts and unclipping its sensor cable from a GPIO module, allowing easy replacement if damaged. This particular unit is a placeholder* with an approximation of volume for currently available components, and a ribbon cable to allow it to be “plugged” into the GPIO module to check fit and clearance.
*The use of “placeholders” (inert approximation) for different modular components allows the construction of different elements at different times as parts become available and expertise gained, whilst allowing the overall project to continue moving forward. As I’m not a mechanical engineer nor an electronics expert there are elements of the Modular V2 controller that would be further built out or modified during the pre-production process.
Finally, the Lower Rail acts as a holder for the Li-ion battery, which is simply mounted on thin VRB tape and firmly secured with several narrow wraps of electrical tape around the battery and rail (this does not cause physical interference with anything unlike zip ties or clips).
Building the RAILS
Starting with the Rails, we find all the parts laid out on the box, as in the image below:
Starting with the Rail Axle bolts, the longer bolt is pushed through the corresponding hole at the outside facing the front end of the Upper Rail, with a thin washer under its head to protect the aluminum surface. On the inside face of the Rail, a spring washer and flat washer will keep things tight once fully assembled, and accommodate slight manufacturing misalignment.
The front “hard mounting” of the Frame is offered up, the bolt passes through the Frame’s eyelet and we add aluminum spacers, the Input Pod mounting bracket, some more spacers, and then the Lower Rail, with a flat washer and retaining nut. It can be tightened with just a Hex key, but it’s recommended to hold the retaining nut with a small spanner and give it a gentle snug tight (3nm if using a torque wrench).
A number of aluminum spacers and steel washers are used here to provide a simple but effective spacing method for the front input pod steel mounting bracket, allowing height adjustment of the Input Pod and Trigger relative to the Grips just by swapping spacers above or below the steel bracket… super simple!
Adding or subtracting spacers between the Rails will increase or decrease the height of the MRF controller, which is a super useful method to accommodate outlier XS or XL hands. Using shorter or longer axle bolts in that instance maintains clearance for the Grips and other parts.
The only limitation for the smallest Rail height would be the stack height of the internal electronics and battery. For most hand sizes the use of different Grip sizes will provide what is required.
We repeat the process with the rear axle bolt and spacers, quickly ending up with this metallic and plastic structure which is the heart of the Modular Rail Frame controller design.
As you may have noticed in the image above, there is a little cube hiding at the bottom of the controller between the Upper and Lower Rails. Whilst it’s obviously another homage to the legendary “Unity Cube” this widget has a very useful function. The image also shows the Touch Sensor bolted into place on the Lower Rail, and its ribbon cable connected by a socket to the little cube.
There’s a GPIO CUBE
As seen in the image above, a new addition to the second version of the controller is the GPIO CUBE, a small widget containing a microcontroller and multiple surface-mounted sockets for ribbon cables and a haptic strap connection. Like all MRF controller parts, it can be replaced if damaged or faulty, or if new input options become available during the controller’s lifetime.
The Haptic Strap connector faces backward, providing a smooth cable route for the power cable which follows the curve of the tracking ring. All other connectors face forward, with a very short ribbon connecting the GPIO Cube to the Tundra HDK’s GPIO connector on its bottom edge.
The current GPIO Cube is another placeholder, but contains the correct surface-mounted sockets and a microcontroller is embedded inside the Cube to provide a real-world approximation of the space required for such an item. The surface-mounted sockets allow the insertion of ribbon cabling so that the cable routing could be determined with great accuracy, ready for a working unit.
Construction of the GPIO Cube would require the assistance of electronics experts, it’s way beyond my skill set. Is it perhaps something that an expert company like Tundra Labs could provide along with a Steam VR unit and plug-in tracking sensors modules?
Fitting the Plastics
Laying out the kit on a handy wooden table, as in the image below, we can see a clearer picture of how Modular v.2 fits together. The GPIO Cube is already fitted, so not shown as a separate item.
The orientation of the Tracking Ring relative to the Frame is now much easier to understand, although there is a brand new molded plastic element called Neck.
The Neck provides both a shroud to protect optical sensor flexes routing into the front for the Tracking Head, and also two additional mounting points for optical sensors with an orientation and position not possible if just using the Tracking Ring itself.
But first, we will fit the Tracking Ring, as the Neck then locks against the front of that Ring.
Fitting the Tracking Ring
The Tracking Ring is really simple to fit, just hold it against the Frame, line up the middle mounting bolt and start to screw the bolt in using a 2mm Hex Key.
When fitting the Tundra HDK with flexes for sensors, each sensor is passed through portholes across the Ring, before the Ring is carefully lifted against the Frame to start the fitting process.
After the first bolt is fitted to the middle mounting, add the remaining three bolts, finally tightening each bolt to securely lock the Ring into the Frame. Threaded metal inserts in the Frame provide the mounting points for these bolts, providing a reliable mounting solution resistant to fitting and refitting damage.
Fitting the Neck
The Neck quickly secures to the Frame using two small bolts which thread into threaded metal inserts located on the side and top of the Frame. As with the metal inserts used for the Tracking Ring, these are resistant to damage when fitting and refitting.
When installing the Tundra HDK, sensor flexes for the Tracking Head (and the front half of the Tracking Ring / Neck) pass through a deep relief channel molded inside the Neck, to keep the flexes away from the mounting bolts or any restriction. The two optical sensors on the Neck are the last to be fitted, each passing through a porthole from the interior, and boasting molded impact protection for the sensor bays like the sensors bays on the Tracking Ring and Tracking Head.
The shorter M3 bolt is first used to hold the Neck in place, loosely tightened, then the longer M3 bolt is fitted to the top mounting; finally, both are firmly tightened to lock the Neck to the frame.
Fitting the Tracking Head
Fitting the tracking head is equally simple, this uses two larger M5 titanium bolts and a couple of washers. The sequence below shows the head being fitted with the bolts, then spring washers (and an additional spacing washer on the left side), the head is then placed on the front of the Frame, and the bolts tightened until they bottom out off hard stops.
The Sensor flexes can be either fitted into the Head before this step is done (being careful when lifting into place) or actually threaded through the access ports in the head once it’s fitted, as it’s completely empty inside with access at the rear.
Fitting the Input Module and Trigger
The Trigger is a brand-new development with a special shape and a trigger finger rest / onboarding ramp on the leading edge. This lets you easily find and maintain a good position on the trigger, or just rest your finger against the ramp when you play Beat Saber, avoiding any Trigger contact or unwanted movement.
During recent testing of the Pimax Sword controllers, I used an unorthodox hand grip during Beat Saber as holding them in the normal way made the Touchpads constantly click during rapid arm movements. Having somewhere secure to firmly hold a controller when not using input controls is a good advantage.
The new Trigger unit is fully rebuildable, with a steel axle pin that can be pressed out using a small Hex Key. The Trigger blade is replaceable too, for damage replacement or customization for different users. You can even easily cut it to shape using a metal file or some grit paper!
A stiff steel spring provides the return action, with adjustable spring tension using a hey key to suit user preference. Just like Modular V.1, the Trigger has a lower contact bolt that works a microswitch in the base of the Input Module (photos taken before installation).
Fitting the Input Module and the Trigger is not difficult, but can be tricky the first time. Offer up the Input Module to the steel bracket at the front of the controller’s Rail Frame. Press it home onto the slot in the bracket, and then lower the Trigger into place. The Trigger is retained with a single bolt and steel spring washer: it’s easier to pre-fit these into the Trigger before assembly.
Using a Hey key, the bolt is gently turned until it finds a threaded metal insert in the base of the Input Module. Keep turning until gentle resistance is felt, this is a plastic spacer compressing. Now you can adjust the fore / aft position of the Input Module and Trigger relative to the Grips.
Once happy with the position, lock the bolt down, it only needs to be 2-3 mm. For production, this attachment mechanism would be re-engineered to make it extra durable, but for the prototype, it works well providing a firm platform for the controls to mount on, but one that can be adjusted in seconds.
Once this is all fitted, you can push the Input Pod ribbon cable through a gap between the front of the Grips, so it runs inside towards the GPIO cube at the rear.
Fitting the grips and the strap
The Grips and Strap co-exist so are fitted at the same time. If you are not using a hand strap on your controller you just don’t fit it and fit the Grips straight into their mountings on the Rails.
When using a strap, start with Upper Grip and its longer bolt, and push it through the Grip so it sticks out inside. Take the front of the strap (which has a hole) and push it down over the bolt, so the bolt comes through.
Now lift the Grip onto the Rail, and slowly tighten the front bolt, but not securely yet. You can adjust the exit angle of the strap from the controller by simply rotating the strap, and then tightening the bolt.
Finally, lay the strap out so it follows the curve of the Tracking Ring, fits the shorter rear bolt but not securely. In this example, the Haptic Module is fitted so the power cable follows the same curve.
Take the loose end of the strap, and slide it through the flat gap behind the bolt, between the rear of the Grip and the Rail, as seen in the image above. Once you have decided on your strap length, just tighten the bolt firmly down to lock it in place. Larger hands can use the strap at full length, smaller hands may prefer to trim it down.
Assembled
After pages of text and lots of photos, here’s the partially assembled Modular V2 controller with a couple of images that should give you a good idea what it now looks like. This “tracked object” prototype is just missing the Tundra HDK and Li-Ion battery, which will be fitted at the end of this article.
Hand fit session
Quick run through of the ergonomics adjustment of the V.2 Modular controller
- Input Pod and Trigger: 15mm fore / aft, rotation and height relative to Grips
- Grips: bolt-on Grips for different hand sizes, different textures, and shapes, etc.
- Hand Strap: Strap angle (relative to the front of Grip) and strap length
- Rails: Height adjust with short / long bolts for outliers (XS and XL hands)
Taking the Modular V2 controller for a “test drive” or hand fit session, as everything is mechanically working – joystick, buttons, trigger, strap, all the adjustments can be tried.
Fitting a small block of wood with a similar weight to the HDK and battery between the Rails ensured it was handled in a similar way. It’s easy to get a good feel by trying out lots of different hand movements and hand poses that would be made during a room scale VR session.
Can the controls be reached? Is anything causing obstruction? How is the weight? What about inertia when quickly moved around? Is the Trigger in the best place? Can that extra button be operated with the edge of my thumb? How does it feel with an Index controller in the other hand?
Opening and closing feels really good, my hand naturally finds its way back to the controls. There is no obstruction, or any clearance issue between my hand opening and closing, which is very welcome. My fingers fall naturally on the Touch Sensor, though the surrounding Lower Grip makes it easy to avoid accidental activation (an adjustable threshold in software would be included).
The Trigger works well and feels aligned with my index finger, repeated quick pulls are firm but effortless. There is no side-to-side movement, wobble, or creaking, the Trigger blade feels very rigid.
I try adjusting the spring tension and it makes the Trigger noticeably firmer, which might be good for strong hands or just people that like the increased activation force. Reducing the spring tension to a minimum makes for an easier pull which would be less tiring during a long session.
My thumb feels well aligned with the joystick, unlike my Index controllers, and of course, the stick feels really precise with no float, wobble, or plastic mushiness. I try using the Input Pod and Trigger reach and rotation adjustments and notice the different positions my medium-sized hand takes. It’s easy to make the controller feel quite different by how it’s adjusted.
I ask my wife to try fitting it for her smaller hands and it easily accommodates her needs, much better than the V.1 which she found a little bulkier (she likes Index controllers without Palm boosters). One of my older work colleagues with large hands (son-in-law owns a Quest 2) says he finds the controller a better fit for his long fingers which get cramped using the Touch controllers.
Later I try swapping spacers on the Front axle, which changes the height of the Input Pod relative to the Grips. This allows extra fine tuning to suit the user, especially the joystick height.
I do some energetic throwing actions, but the controller stays put, which is surprising considering it’s got a simple elastic hand strap. For production, the strap could be a custom unit, with perhaps a bit more shape or some molded elements around the Haptic Module.
Compared to the V1 controller (which I did lots of hand fit testing with) the V2 feels more balanced with the weight centrally located. The V1 felt a bit lopsided in terms of weight balance due to the bulkier tracking ring, which I had slimmed down considerably for the V2, and it’s noticeable.
Overall it was a successful testing session: definite improvements were made over V1.
Next steps – SteamVR Integration
This starts with fitting the Tundra HDK to the Upper Rail using VRB tape, after cleaning contact surfaces with Isopropyl Alcohol, to degrease any contaminants. I also degrease my cutting tools, as I’m using the scalpel to both trim and place the VRB tape without getting skin oils onto it.
The sensor flexes are fitted out, the tracking components lifted into place, and everything bolts down. Next we would move onto SteamVR integration by writing the JSON, and scanning the controller in 3D to allow accurate generation of a render model with all model Points and Normals, including precise measurements of the optical sensor array.
I tried scanning using smartphone-based applications but the results were completely unsatisfactory so I’ve got an appointment at a facility in the New Year (it has been booked up for months) in East London: hopefully, they can help me create a “Digital Twin”.
Conclusions
Well, that was a super busy year, and despite two big setbacks (legal and miscalculation), somehow I managed to produce something that’s got some real potential.
Much of this year was spent figuring out the challenging puzzle of packaging a motion controller around the human hand, in hindsight I’d have done a headset as you’ve got a decent volume of space to work with, and everything projects outwards from the face gasket. Think about Pimax’s 8K series as a good example of how large in size, but still functional an HMD can be.
Trying to get a motion controller to work within the constraints of the human hand, whilst being adjustable to fit as many hands as possible, finishing with the start of a working prototype, well it’s been an interesting adventure!
It’s also been really tough working on the entire design as a self-funded solo developer, whilst working a full-time regular job. Products like motion controllers are typically developed by well-funded, multi-disciplinary teams working in full-time roles.
Even finding time to become competent in a CAD package like Fusion360 proved impossible this year, hence the sneaky move of using 3D scanning to generate a digital model for SteamVR integration. My primary interest and experience in XR is human factors engineering (ergonomics), as I mentioned in last year’s article:
“Perhaps the design is something that could be manufactured in a more refined version, if funding was actually available. Personally, I’d prefer to work on the ergonomics and adjustments and leave the mechanical engineering and electronics to people who have experience and more knowledge.”
The Modular Rail Frame physical form certainly works well in terms of putting controls in the right places and allowing a wide range of ergonomic adjustments. It’s certainly possible to build around this platform, and build at a larger scale to get it out to more people.
The next logical step would be to productize the design, with input from multi-disciplinary specialists to address mechanical engineering, materials, electronics integration, programming, production tooling, etc.
The money
Regarding pricing? Higher quality costs more money, but groups of consumers do exist in different equipment markets who are keen to pay higher prices for higher quality products. The healthy sales of $1000 Index certainly proved that VR enthusiasts will pay more for better.
A good example given during my industrial design training is: many consumers are happy with cheap headphones, but a healthy market exists for high-quality headphones for audiophiles. They appreciate the difference, and with many being time-poor, want the best performance during their limited time, and they want reliable equipment.
Perhaps it is not a stretch to think that many PCVR owners find themselves in the “time-poor” category: after all, something (good job with long hours ) has to pay for a VR-capable gaming PC and PCVR system. When sending many Index controllers back for RMA, top marks to Steam Support for always being helpful but due to the haphazard nature of production, it was sometimes weeks without replacement controller(s).
I even bought a second pair, but these soon required their own RMA. This downtime was frustrating as it stopped me from using a large number of VR titles that required motion controllers (I did enjoy some excellent seated and sim games though!) I‘d certainly be willing to pay Index headset money ($499) for a pair of Modular controllers that could be rebuilt, adapted, and upgraded over a number of years. $249 for one Pro level controller seems fair, but the market may decide otherwise if it ever went to production.
My closing arguments for the Modular Rail Frame design concept
- Rebuildable = long term ownership with minimal downtime
- Modular input choices = suit the user, not the controller
- Fully adjustable ergonomics = best hand fit, no compromises here!
- High quality input components = best “feel” (tight stick, no wobbling or creaking)
- Toughened to resist impacts = keep on playing, accidents happen
- Big reduction in E-Waste and shipping pollution = less plastic, less harm to the planet
- Future proof with upgrades = not obsolete by next year
- Different tracking systems = Steam VR, white LED (WMR style), IR LED (Meta style)
- AR use with pass through camera tracking of fiducial markers?
- Customise = do whatever you like or can put together, encourages collaboration 🙂
A closing thought, whilst working this year on the Modular Project, perhaps the future for MRF controllers is opening the idea up to everyone, so they can go build and have fun. If you can build the Rails (easy) and Frame (a bit tricker) it’s a good starting platform to build onto. It needs some expert support for electronics and software, but anything’s possible if enough people get involved.
Valve have commented on Nintendo in the past, from a widely reported interview (taken here from Nintendoeverything.com), Gabe Newell stated:
“We’ve always been a little bit jealous of companies like Nintendo. When Miyamoto is sitting down and thinking about the next version of Zelda or Mario, he’s thinking what is the controller going to look like, what sort of graphics and other capabilities. He can introduce new capabilities like motion input because he controls both of those things. And he can make the hardware look as good as possible because he’s designing the software at the same time that’s really going to take advantage of it. So that is something we’ve been jealous of, and that’s something that you’ll see us taking advantage of subsequently.”
Since the MRF platform allows easy customization, how about releasing a game with an MRF component, for example, a new sensor, input, or haptic? Same idea as Guitar Hero, and other games bundled with peripherals of the past, but much lower cost as it’s only a component, not a new controller.
An open standard?
Bootstrapping a new controller standard wasn’t my initial idea, but with both Meta and Pico reportedly selling loss-making hardware for their closed platforms, which has distorted the market, it’s tough for small companies, startups, and solo developers to make anything by developing and selling tracked hardware.
I’m using SteamVR because Valve licenses their technology to developers like me for free, through the Steamworks program. The only real cost is your time and $200 buying a SteamVR HDK.
Documentation is provided though a bit out of date, the SDK is downloaded free under license from Steamworks and contains a number of useful tools for prototyping. Thanks to Valve and Tundra Labs, it encourages experimentation because you can actually get in there and open it all up. You can design and sell hand straps for Quest 2, but you are not going to be selling custom controllers using their tracking hardware and code.
I looked at definitions of “open standards” and “open source” as ways of putting my work into the public domain for everyone to use. It doesn’t seem to be something that one person can ratify, but an Open Standard called OpenXRC would be my hope, sparking an ecosystem of MRF-compatible parts from different manufacturers big, small and independent! We can only dream…
As the Project comes to a close, I’ve become aware that Tundra’s HDK is at the end of the line. I made some enquiries (to get a second unit to build another controller) but nothing came of it. Ultimately, it wasn’t designed for production units and wouldn’t have been cost-effective to build production controllers with, but its purpose was to allow prototyping.
In this respect it did its job beautifully, forcing me to make some creative design choices with radical thinking to accommodate its physical restrictions (i.e. the length of flexes and positions of sockets). The outcome was that I ended up with something very different from what I originally intended. It definitely worked out better, so thanks Luke! [Luke Beno, founder of Tundra Labs]
Whether this controller ever becomes a product that people can buy depends on many factors. There is still a lot of work to do and it would need a big pile of cash. There are also doubts about the long-term future of lighthouse tracking, though the Modular controller design can easily adapt to different tracking systems, maybe even some small on-board cameras like those used on the new Quest Pro controllers.
If this article gets industry people and users thinking, then it’s all been worthwhile. The Nintendo hardware and software integration has some real value and could be supported by the modular nature of NRF. Imagine the next VR game stimulating your hand with a mild electric tingle through an MRF add-on, whenever you replenish your health! What about an extra button to use a new game mode? Or a unit that provides hot and cold sensations for the hands?
In some ways, it was never about going to production, but seeing what could be built by going down an unconventional route. Hearing people talk about wanting “Modular controllers” after last year’s article was published was a very satisfying feeling.
Thanks!!
Many thanks to everyone who has helped, advised, and encouraged me, including people who invited me to different meetings at the start of the year. Special thanks to those legendary hardware guys for the Zoom call, I’m still stoked about that 🙂
Thank you to Tony a.k.a. Mr. Skarredghost for publishing this, my twelfth article, and putting up with an endless barrage of excited emails and Steam voice chats during the past year, I promise no more (for a while…)
Huge thanks to my long-suffering wife Sara Ramos for her unwitting role as a hardware widow, and for taking some great photos.
And of course, my genuine thanks to you, the reader, for getting this far. I hope it was useful and shows what you can do with just an idea. Let’s see what the future will bring…
(… and if you want to get in touch, you can find me here www.immersionmechanics.tech)