How to develop a mixed reality experience on the Vive Focus

Ho ho ho! Christmas is coming and it is time for me to make a gift to the whole XR community, letting everyone create augmented and mixed reality experiences for the Vive Focus (Plus or the original one)!

More than one year ago, I published a post entitled “How to create an Augmented Reality app for the Vive Focus in Unity“, where I shared with everyone a plugin to develop AR applications on the Vive Focus, that is actually a VR headset. The plugin was usable, but had some limitations, like the fact that it was compatible only with Wave SDK v2, it had problems with Unity UI elements, and didn’t let developers modify the cameras stream.

In all this time, I have evolved this solution, because I used it in various experiments I performed, and especially because we at New Technology Walkers have used it to develop our game HitMotion: Reloaded, the first mixed reality fitness game.

We at NTW all agreed that after the launch of the game, we would have shared the plugin with the community so that to let everyone create AR and MR experiences for the Vive Focus Plus. And what day is better than Christmas for a gift?

I have so updated the repository of ViveFocusAR with the new version of the plugin, that is much better than the previous one because:

• It is compatible with the latest version of Vive Wave SDK;
• It tries to compensate for the low refresh-rate of the cameras;
• It presents a quasi-undistorted camera stream;
• It is very easy to be used: you just import the plugin and then add a prefab to your scene;
• It doesn’t let you only do AR, but also MR.

The last point is what we are the proudest of. Thanks to this, you can play with the cameras stream of the Focus, and not only show a black&white feed but do with the images whatever you want by applying a shader. Do you want to see some cool effects that you can do with it? Well, for instance, you can give people Predator vision

or make them feel like in The Matrix

Of course, you can not only add some fancy effects to the images but also add virtual elements, as we do in HitMotion: Reloaded. We exploit this feature to let you play a boxing game without the risk of punching people and objects around you since you can see your surroundings (check out our game on Viveport, if you haven’t done it yet!).

The possibilities are endless, in the tech and creative worlds. And it is cool that in these months some people have contacted me to tell me about the uses they were doing or were thinking about doing with the plugin: I’ve heard about IoT things, artistic things, healthcare things, etc…

Now you can do all of this, for free. The project comes with MIT License, that is an educated way of saying “Do the f**k that you want” :D. Of course, every mention in the projects you will create with it will be highly appreciated, exactly as every check of 1M dollars… but they are not mandatory. And if you need help in developing an MR application, feel free to contact me. We of NTW will be very happy to help you in creating it, especially if it is something creative, innovative, and a bit impossible. We are the jedis of MR 😀

I hope you like this Christmas gift and you will do amazing things with this MR plugin for the Vive Focus Plus. In Q3 2020 we should see the first AR/MR standalone headsets that will exploit the XR2 reference design of Qualcomm, and I think that using this plugin with the Vive Focus Plus is a nice way to prepare for that moment. It is the only headset currently enabling something similar, and after having tried it, I can tell you that the possibilities for the MR future will be endless.

I, HitMotion:Reloaded‘s game designer Max Ariani, the super-creative mind Enea Le Fons and all others taking part in this project wish you a wonderful Mixed Christmareality!