How to design a proper bimanual UX in your VR experience

In some of my reviews, I praised the experiences that offered appropriate bimanual interactions (e.g. Monzo VR) and criticized the ones offered inappropriate ones (e.g. Archangel VR). But how do I judge when a bimanual interface is good and when it is not?

We have to go back by some years when I was reading a lot of scientific articles for my thesis on bimanual hands tracking. One day I came across to this article by Yves Guiard. It has been published in 1987, but I think that every person should read it even nowadays since it is a must-read article if you want to design a UX. It states the basic principles of bimanual interactions and its conclusions have been proved by various other experimentations (I’ve read a lot of other articles that confirm its results).

So, what does this super-important text states?

It talks about how the hands should interact to perform a task. We have 3 ways to perform a task with our hands:

1. Single-handed task.
2. Symmetrical hands task.
3. Asymmetrical hands task.

In the first case, there is not much to say. You have a task, and you perform it using only one hand. For instance, you are in VR and have to shoot a bullseye using a single gun that is in one of your hands: you just take the gun and shoot it with your right hand. (Notice that in this article I will assume you do your tasks with your right hand… there are little changes if you’re left-handed). Our brain will find this interaction perfectly normal.

In the second case, the two hands perform the task with a symmetrical movement. For instance, if you have to clap your hands in VR, you just move the left hand towards the left hand and viceversa (same movement, mirrored); if you have two guns and you have to shoot the same object, both hands will aim towards the same target (same movement, non mirrored). For our brain, this is perfectly ok and interface will be well designed.

The third case is the most tricky and the one I’ll discuss in deeper depth. The two hands perform different movements to accomplish the task.

One naive approach would be to just assign different subtasks to each hand. In Sneaky Bears VR, the two hands have two different guns which have two different purposes. The left hand has a water gun to extinguish fires, while the right hand has a standard gun to kill enemies and you have to use both guns contemporarily to win the level. This is bad on so many levels and in fact the gameplay becomes very challenging. If you’re designing your game to make your players go mad on purpose, this is ok… otherwise you have to rethink your interface. In this example, what resulted is that my brain split my tasks into two parts, so either it controlled the left hand or it controlled the right one. Controlling both at the same time was just impossible. The reason? Our brain doesn’t treat our hands the same way.

According to Guiard, our hands have different purposes while accomplishing a task. He talks about three principles:

1. The left hand provides a frame of reference for the right hand. Guiard takes handwriting as an example. When talking about handwriting, we usually say things like “I write with my right hand”. This is only partly true. While it is correct that you actually hold the pen with the right hand, left hand and arm are there to give you a frame of reference. You can write straight on each surface, provided that your left arm is there to give you a frame of reference of what is “straight”. If you don’t use your left arm, the brain has to infer a frame of reference for your writing lines using the elements of the environment (sheet, desk, etc…) and this results in a less precise evaluation and so a worse writing. The left hand is there to give a frame of reference to right one.
2. The left hand moves on a macrometric scale, while the right hand on a micrometric one. The right hand can move very fast, offering little and precise movements, while left hand moves fewer times and at a bigger scale. In the case of handwriting, the left hand and arm perform a big movement to put themselves on the desk and adjust the sheet, then the right hand can perform those little and fast movements that represent the writing. Sometimes (seldom) you’ll re-adjust the sheet and you perform this again with the left hand and then write again with the right one. Left hand finds very hard making little and precise movements: it is just not its job. Right hand instead operates at a finer temporal and spatial resolution.
3. The left hand moves before the right hand. Since it has to provide a macrometric frame of reference, it has to move first. It is obvious: it would have no sense that the right hand moved first to write on a paper, without having a frame of reference. Left hand provides a frame of reference, performing a macro-motion and then right hand can make its tasks with its finer motions.

So, no one writes “with the right hand”. We all write with both hands, with the left hand having a completely different task from the right one. And it is not true that the left hand is inferior to the right hand: they’re equally important and serve different purposes. Every hand is specialized in what it has to do and does it well.

Why is this so important for VR? Because in VR we can use our two hands thanks to controllers like the Oculus Touch. And as developers, we should take in mind that we have to design a bimanual interface that abides the 3 Guiard’s principles.

Sneaky Bears VR is bad because the two hands do not cooperate: left hand does not provide a frame of reference for right hand; it doesn’t move first; it is required to shoot at stuff, which is a movement in a micrometric scale and this is not what left hand is designed to do. Archangel VR does the same error, treating both hands as if they were independent and this results in problems in shooting enemies when they’re too much. Monzo VR, on the contrary, uses your left hand to rotate the car you have to assemble, so that the right hand can put the car pieces in place: left hand moves first, provides a rough frame of reference and then right hand does the task of assembling the pieces. This is perfect.

Have you wondered why all artistic program (like Google Blocks) in VR have the tools on your left hand? Because if you want to draw, you move your left hand somewhere in your field of view, then you move your right hand to make the action of taking a tool (on a 3D menu whose frame of reference is defined by your left hand) and then you make the micromovements of drawing with your right hand. It follows the Guiard’s principles.

So, next time you want to develop a VR application, remind the Guiard’s principles. Either you make the interaction uni-manual, or you make it bi-manual and symmetric, or you make it bi-manual and asymmetric, following these 3 principles.

I’ve simplified a bit the original article, that I advise you to read since it provides you a deeper explanation of the concept and also Guiard’s idea of the Kinematic Chain. Anyway, here I have expressed the basic rules you have to follow next time you’re designing an AR/VR application. Remember that your two hands have different purposes and that they work together to accomplish a single task. Your users will appreciate that a lot.

And I also would appreciate a lot if you can share this article on social media and also subscribe to my newsletter…