If you are a VR enthusiast, for sure in the last days you’ve found your social media feed full of videos and animations of people playing around with a cute cat in virtual reality using only their bare hands. This is because Leap Motion has announced its v4 version of the tracking runtime and with it three demos to showcase the new tracking functionalities: Cat Explorer, Particles, and Paint. And Cat Exploer got a lot of interest from people, because… well, cats.
I’ve tried them all: I was so curious about them that I took out my Leap Motion and used some tape to attach it to my Oculus Rift CV1, that in the end, was something like a mummy headset. The duct-tape job worked, so I can give you my personal description of all of them.
Cat Explorer
Everything that regards cats becomes a success on the web and this little experience is no exception. My Linkedin and Twitter timelines were full of people playing with Cat Explorer or sharing videos made by others with this app.
Cat Explorer is an educational app made to show you all the anatomy of a cat and it obviously employs Leap Motion as the only medium of interaction. As all these demos, it is a short app, it is more a showcase than a real practical app, but it is a great one: it is really well made. The cat is very cute and all the few functionalities are implemented in a masterly way. The only problem is that it is slightly disturbing since you can see all the interiors of a cat. Bleah.
Here you are a video where I showcase all you can do with this application.
The first thing that you notice when you enter the app is that the cat Whiskey, that has been inspired by Keiichi Matsuda’s cat Donna, is very cute. Then you notice that it follows your hands, and since it is very cute, you want to pet him, but he’s not interactive in this sense. He just watches you asking why you are disassembling his body.
In fact, if you pass your hands through his body, you can see all these body parts slightly exploding, showing with a caption which parts of his body they are. You can this way learn all the cat anatomy: the organs, the bones, the vessels. Then you can also:
- Rotate the cat, to investigate its organs from another point of view;
- See a section of the cat’s body just by selecting the feature and moving some panels that “cut” the cat;
- Select which layer of the cat anatomy you want to see: muscles, vessels, skeleton;
- Perform a fast explosion of the cat internals to see how it is made.
This is a nice educational experience. I admit that I’m more interested in Grumpy Cat memes than in the anatomy of cats, but I found the app intriguing anyway. I spent some minutes playing around with the cat and see all its anatomy from different points of view.
But the true gold nugget of the app is the user interface. It’s marvelous: Leap Motion has shown in all these years that they are masters in designing user interfaces and all their long articles on Road To VR about UX in VR are a proof of it. They’ve also written a post about how they have designed Cat Explorer on their blog: you can read it and understand all the process they’ve gone through to design such a great demo.
Everything you can do in Cat Explorer is simple and intuitive. The cat starts in an ideal position calculated so that it is next to you and you feel the desire to pet him. But as soon as your hands go next to him, you start seeing the disassembly of its body: you automatically discover that with your hand you see the internal of Whiskey, you need no explanations on how to use that. Even the rotating pedestal when the cat is on is intuitive: in real life, you rotate such objects by using your index finger and in this app, you do exactly that.
https://gfycat.com/gifs/detail/naughtybiggecko
When your hands go next to the pedestal, the menu appears and you can easily select what view of the cat you want to have. This is the only moment when I found the UX a bit wrong: as you can see from the video above, I tried to click on the view I desired to have, but actually, I had to move the slider. Anyway, I understood what I had to do after a short while.
The panels to perform the section of the cat are carefully studied too: as you can read from Leap Motion’s blog, they spent a lot of time designing the handles of such panels, so that they convey you the idea that they are interactive. The handle is magnetic and goes towards your hands and this makes you understand immediately that you have to grab it… and it is also insensitive to little grab gestures errors, so it is very easy to be used.
In the following GIF by Leap Motion, you can see all the iterations they performed until they arrived at the best handle possible:
https://gfycat.com/gifs/detail/raggedmeanbassethound
The little handle to perform the fast explosion of the cat that you see at the end of the video is nice, but it comes from nowhere so it would have had more sense to add a little label to it.
You can see from my description that the interface is fantastic because it is very easy and intuitive: everything has been carefully studied in this sense. You need no tutorial for that, you just discover everything about the app just by using it. As someone said a lot of time ago: “A user interface is like a joke, if you have to explain it, it is not good“. And this is a great UX, that is fantastic to be used with the bare hands, that worked great with Leap v4 Tracking. A great job by Leap Motion engineers! If you’re interested in designing UX for virtual reality, you should play with it and also read the related article.
Particles
This is the demo that I liked the least. This is because I’m not interested at all in those particles’ life simulators, the game of life and such things. But if you’re into it, I’m sure you’ll love Particles.
You can set the initial conditions to start with and see a lot of colored spheres floating around you: then these spheres start moving and aggregating following some simple rules. You can also grab these aggregations and also destroy them with your hands to see what happens. You can also change the parameters and restart the simulation, if you wish.
While I don’t love this kind of apps and I also think that it doesn’t showcase at all the improvements of Leap Motion’s v4 tracking, I’ve still been able to find an interesting thing about it as well and it is again the user interface.
If you move your head towards the left, you can see the two dialogs that you can use to tune the particles’ parameters. The nice things about them is that below them there is a sphere with a hand drawn on. Since there is a hand drawn, I immediately grasped that I could take them with my hand and I discovered that this was the handle to move the two windows in the position I preferred. While taking the handle, I could see that the parts of the hand and the object I was grasping that were close to their intersection points became colored. This is something that Leap Motion discussed in an article on Road To VR and that helps in making the interaction with the hand and an object it is trespassing more natural. This way you understand better how you are grabbing the object and also you feel better about the fact that you’re trespassing an object, even if this shouldn’t be possible (in real life you can’t penetrate objects with your fingers). A little UX trick that you can see in the GIF below.
https://gfycat.com/gifs/detail/HiddenOldfashionedApisdorsatalaboriosa
Paint
If you have ever dreamed to paint in VR using your bare hands, Paint by Leap Motion is what you were waiting for. Again, it is not a complete program like Google Tilt Brush, it is just a demo, but it is enjoyable nonetheless.
With Paint, you can draw just by pinching your thumb and index finger and drawing in the air using current color and brush width. Then, of course, you have all the tools to select the settings of current stroke and to load and save your creations. In my case, since I’m terrible at drawing, nothing I created was worth saving 🙂
As you can see from the video above, Paint has some very interesting functionalities and even in this demo, the most interesting thing to analyze is the user interface.
This time the app starts with a tutorial since there are things that are not immediate to grasp: for instance, painting by pinching is not natural (even if it resemble a bit the pose of holding a brush in the hand). The tutorial is super-useful to learn all the mechanics of the app and it is basic but very well made. It guides you in understanding everything. What I loved of the UX of the tutorial is how you signal to go to the next step: there’s no OK button or such. When you got the concept, you have just to look at your master that is explaining you everything (that, weirdly, is just a grey cube made in Unity… WTF Leap Motion, you could download some model from the Asset Store!) and perform a thumbs-up gesture with both hands. This is a genius idea: thumbs up is a very natural gesture to make people understand that you understood something and Leap used it in the right way. The final genius comes from the Master Cube doing a thumbs up himself to make you see in a very natural way that the system detected your gesture.
https://gfycat.com/GreedyUltimateKakarikis
You can find all the tools of the app (e.g. the options to set colors and width of the stroke) attached to your left hand, as soon as you rotate it towards your face. This is in line with Guiard’s principles of bimanual interaction: you give context with the left hand and then perform the tasks with the right hand (if you are right-handed, of course).
https://gfycat.com/ElatedMammothCrocodileskink
Having the controls attached to your hand has two problems:
- They must be little panels, otherwise you have something enormous attached to your hand and this is unusable;
- It requires you to keep the left hand in a tiring position to show the controls.
That’s why Leap had the genius idea of letting you extract the various panels from your left hand and move it into a fixed point in the world. The paint panel expands and gives you even more features and also lets you rest your left hand while you continuously switch colors. I found that very comfortable: left hand controls were fantastic, but if I had to perform some operation continuously, I always moved them to the world.
https://gfycat.com/MedicalSneakyHoatzin
When moved to a world position, the color panel also lets you mix colors, something that people that actually draw always do. This is another cool idea and happens in a very natural way as you would do in real life: using your fingers to take colors and mix them on a white portion of your palette.
https://gfycat.com/InnocentGrandGoose
There are three panels attached to your left hand:
- One to select the current color;
- One to select drawing options (stroke width and undo/redo);
- One to select program options (change environment and load/save drawing).
The environment change is implemented in a very basic way (you can only select if you want a light gray or dark gray environment), while the load/save functionality is more interesting. You can especially use it to load some drawing already made by Leap Motion’s artists, so you can see some interesting things that are possible with this paint demo.
My final opinion on Paint is pretty mixed. From one side, I loved the UI, on the other side, the program shows some objective issues. First of all, this is a very basic painting application, it can’t compete in any way with Tilt Brush or other similar applications. Of course, this was not the purpose, but it is a pity anyway.
Then, the app suffers from the not-perfect tracking of Leap Motion v1. I don’t know if designing a Paint demo to showcase v4 tracking has been a smart idea, since when drawing we all want to move our hands fast and move them everywhere, but Leap Motion v1 has not a great FOV and so it is easy to see the hands get lost during the drawing operations. Furthermore, during hands fast movements, the pinch gesture detection was intermittent and this resulted in the drawing coming out not as desired. Sometimes it was frustrating pinching my fingers and see the system not drawing. As Abrash has said, an input mechanism can’t work 90% of the time, it should work 100% of the time. In this case, Leap Motion wasn’t, in my opinion, a good input mechanism for this app, at least for my office setup. Most probably with Leap Motion v2 these things should be far better (it has a FOV that can encompass all the hands’ movements), but with v1 we have various limitations.
v4 Tracking
These were demos to showcase v4 Tracking of Leap Motion… so, how is this new tracking? According to Leap Motion, it should be far better than v3, adding:
- Better finger dexterity and fidelity;
- Significantly smoother hand and finger tracking, with motions that look and feel more natural;
- Faster and more consistent hand initialization;
- Better hand pose stability and reliability;
- Improved tracking fidelity against complex backgrounds and extreme lighting;
- More accurate shape and scale for hands.
Honestly, from my tests with the 3 apps and the Visualizer, I can say that these improvements are not easily noticeable. From tracking v2 to tracking v3, I noticed a huge leap forward, an impressive improvement. Trying this v4, I noticed that it was a bit better than v3, but it is only a step forward and not a big leap forward. Maybe with the new hardware (that we’ll never have as an accessory, sigh), these improvements are more noticeable, but I can’t tell. My mediocre experience with the Paint app shows that there’s still a lot of work to do… or most probably we’re at the maximum that the v1 version can do. I still hope that Leap Motion will change its mind and will sell the v2 version to us enthusiasts, because that is a game changer.
Anyway, I think that Leap Motion is doing a great job and I encourage you all to try all these demos if you have one Leap Motion device, because as I’ve told you, they can showcase you how a great UX in virtual reality is done.