How HitMotion:Reloaded transitioned from an AR to a VR game

Two weeks ago, we of New Technology Walkers have launched HitMotion: Reloaded on SideQuest. This game has had a very long and interesting story (maybe one day I’ll write a long post about it), and it has lived various transitions, with the last one being from a passthrough AR game to a standard VR game. In this post I want to tell you what changes we had to apply to the game in the transition, and what were the pros and the cons of both approaches, hoping that it can be useful for other devs (yes, Julien, especially for you 😉 ).

From HitMotion AR to HitMotion VR

HitMotion: Reloaded has been one of the 20 launch titles of the Vive Focus Plus, and we can never thank enough HTC for the support that has given us. It’s been a true emotion to launch the game in Shenzhen, China as probably the first mixed reality fitness game. While the other titles were all in virtual reality, HitMotion exploited a plugin that we of NTW created ourselves to exploit the front cameras of the Focus to offer passthrough augmented reality. It was an AR game that could be played on a VR headset, in 2019. Just to make you understand how pioneer we were: Facebook will probably release the SDK for passthrough AR for Quest in the second half of 2021.

Unluckily the Focus Plus has been sold mostly as an enterprise headset here in the West, so our game was targeting a very niche market. That’s why we decided to port it to other devices, especially the Oculus Quest that is selling so well among consumers in the last months. Since passthrough AR is not available on Quest yet, we decided to transform the game into a full VR game: we kept the same main mechanics (move inside a space, punch, dodge), but we changed almost all the rest. Now the game is available on SideQuest as a free demo, and it is getting great feedback from all its users (download it now!). We hope to get traction so that to get the funding to develop the full version of the game that we have in mind.

This transition from an AR game to a VR game of course has required some choices, and it has taken some pros while also carrying with it some cons. Let me tell you everything about this conversion that required a lot of effort from the game designer Massimiliano Ariani, the artists Lapo Germasi & Victor Pukhov, and of course, me, the developer.

The setting

In AR your room is your environment, while in VR, you have to craft an environment around the user.

Every one of the two scenarios has its share of problems. When you are in AR, your big problem is that you don’t know where the user is playing. On the Focus+ we had no kind of environment understanding, so we just put elements on top of the real world, without knowing where the user could find them. He could be playing on the toilet, or he could be on a vast football field, we had no idea. And it’s very difficult to think about the shape of the enemies, the targets, even the lore of the game if you don’t know where the user could be. A target that is perfect if the user is in the gym may look ridiculous if he is playing in the toilet. That’s why we avoided crafting a story around the game, even a basic one, and we created targets with a simple spherical shape that could fit well in whatever environment.

In passthrough AR there is another problem: since you don’t know where the user is playing, you can only hope that he set correctly his play area and there are no objects inside it. If there are, he can see the virtual elements always on top of the real ones, breaking the magic of the game. This is a problem common to many AR experiences, especially the ones that you enjoy with the phone (think also to Pokemon Go), and while this is not a big issue, it is still a nuisance.

In VR we have not these problems anymore, so we could define everything about our game. The artists proposed a street-fighting mood with a cyberpunk touch, and we all liked it because it was so different from the other boxing games that are either set in a gym or in a Tron-like environment. This made our game stand out when compared to competitors. We could draw the environment, the enemies, the trainer, all in a very coherent way, and define ourselves the mood that we wanted to convey to the player. You know that if you play HitMotion, you are fighting in the streets against some targets, you’re like Rocky Balboa in the Blade Runner world (I’m just imagining how trashy would be this kind of movie lol).

The problem with the VR scenario is that you have to draw everything, and this is more expensive to do. If you are an indie team, it’s much better for you to make an AR game, because in it you have just to draw the enemies. When you are in VR, instead, you have to draw all the environment, the skybox, set the correct lighting, etc… This means more time invested, which is more money spent. It is true that the Unity Asset Store is your friend, but as our creative artist Lapo Germasi says “If you want to make a good job, there is no asset that you download and you just use as is. There is always some work that is necessary to adapt it”.

Colors

Passthrough AR on the Vive Focus Plus and in most other headsets is still black and white. That’s why every element that we designed in the AR version of the game, both the UI windows and the game elements, featured very bright colors. The UI in particular was made with strong cyberpunk fluorescent colors like fuchsia and yellow. This way the bright virtual elements could counteract the boring monochromatic camera stream.

When we switched to VR, this was not necessary anymore, so we decided to keep a slight cyberpunk touch, but to make everything look less strong and vivid. It’s interesting to notice that the original AR menu was not looking bad at all even in the new environment, and for a moment we even thought about keeping it, but then we decided to change everything so that to make both the UI and the 3D elements completely coherent.

Tracking

Passthrough AR was made through a plugin that we invented, and it was not officially supported by HTC. Since the plugin was built on top of Unity, it was not as performant as something integrated into the OS, and paired with the nonperfect tracking of the Focus, the result was that when using AR, you could clearly see the elements wobbling in the space when you moved around your play area. The result was that if we tried to attach a virtual element to an exact physical position, the user would have seen it shaking around every time he moved, causing a disturbing effect. That’s why in the original HitMotion: Reloaded, all elements were floating in the air: the menus, the UIs… everything was floating, so all the movements of the virtual elements were more easily tolerated. Everything was like a flying cloud, and if a cloud moves a bit up and down, this is not a problem, while if you put an object so that it looks like a static thing on the floor, and you see it floating, it breaks the magic. You should remember this piece of advice also for when you develop an application for all those AR glasses that are becoming popular now (e.g. Nreal Light) that have not perfect tracking yet. Flying objects are always a safe bet.

In VR, given the high quality of the tracking of the Quest, and the fact that there is no matching of real position and virtual elements to take into count, we have been able to avoid all these problems. That’s why we have been able to put the initial menu fixed in an exact position of the world. This has also been good to increase the sense of presence: we thought about keeping the floating interface, but in VR, a floating interface, while you are in a virtual environment, is a bit an immersion breaker: in real life, you don’t have interfaces that float around you. That’s why Max made sure that most of the UI windows of the game stay in places where they seem emitted by displays that are in the world. Also, we made sure that you start the game outside the ring, and then enter the ring only when you are in an active play session so that you have the impression that you have gone on the ring to fight, as it would happen in real life. Little details that I’m sure all the players have appreciated.

Safety

Safety was one of the selling points of HitMotion: Reloaded on the Vive Focus Plus: since you could see your surroundings, you had no risk of punching the people and the objects around you and so you could “stay fit while having fun in full safety”. This has been very important for instance for the demo that the game had at the Vive Ecosystem Conference in Shenzhen: the demo space was very little, and the fact that people could see what there was around them was important to make sure that no has got hurt. With the other boxing game that was showcased there, sometimes the players had to be careful of what was happening around them. AR offered a great advantage in this sense, and it offered us an edge when we compared our game to alternatives like BoxVR/FitXR.

When we switched to VR, we had to lose this advantage that we offered. It was a painful decision, also because we had no alternative to offer safety to the user. Luckily, modern standalone headsets have a reliable guardian grid, so the user most of the time can anyway play safely without much risk of punching things (unless you play hectic games like Gorn or Echo VR). As a palliative, we also offer a new tool to adjust the play area, so that the user can put the ring in the position and orientation that best fits his play space. Also, in this free demo, we are only providing static levels, so that if the user has set correctly the play area, there is no risk of punching stuff because there is no need to move in the space. Dynamic levels are coming in the future full release.

Play Area Setting

Making the user set his play area in AR is incredibly easy: since he sees his surroundings, you have just to show him the borders of the area required by the game, and he can immediately see how the game area fits in his physical room. This is incredibly handy. Again, regarding this, AR was a great advantage of our game: think about how it is easy to set your guardian in AR on your Quest and you get immediately why.

Porting this to VR was a bit more tricky: how to make the user see how the virtual play area fits the real space without requiring him to constantly put on and off the headset? Here our game designer Massimiliano Ariani had a good idea: we show the Guardian contour on the floor while you set your game area: since more or less you have drawn your Guardian so that to match the shape of your physical room, by looking at this contour, you can have a sense of where you are on your physical space, and you can fit better the 2m x 2m virtual ring of the game, without having to constantly peek with your eyes outside of the headset.

https://gfycat.com/formalvigilantinganue

Performances

If you think that the Quest 2 is a device with limited horsepower, it is because you have never used the Focus+ with our passthrough AR plugin. If you wanted to keep a good framerate, you could draw very few polygons every frame, and you had to forget about post-processing effects or particle systems. You had the rendering capabilities of the original Game Boy probably. That’s why we used very simple geometric shapes for the targets, very lightweight shaders, and a low poly monochromatic figure to represent the trainer Bob.

On Quest 2 now we don’t have these problems anymore, so we have been able to add some fancy effects, like the holographic shields on the targets, or the animation when an enemy gets generated. Our graphical quality has improved a lot, and also the poor Bob now has been able to evolve in something better, like a cute robot that reminds a bit Robo Recall. This is great news, but anyway, the Quest 2 has still very limited capabilities, so we had to work a lot to optimize the graphics of the game for it, making some sacrifices on the visual quality too.

Motion Sickness

I have already told you multiple times that our plugin for passthrough AR was experimental, and for this reason, it may cause some dizziness to some users. That’s why we added a big warning message at the beginning of the application.

Now, being it a standard VR game, we have removed the warning. We also used standard tricks to improve the comfort of the users: for instance, during every scene transition, we perform a fade to black, so that the user doesn’t realize that Unity may have some stuttering while loading a new scene. Notice that this was impossible to do in AR: you can’t do a fade to black in AR, because the user must continue to see the environment around him, so the little jumps due to the scene loading were clearly visible in the previous version.

Easier cross-platform development

Passthrough AR is available on very few headsets at the moment, so it limited HitMotion: Reloaded to be available only for them. Making it a VR game makes it immediately compatible with the Quest, Focus, and all PCVR headsets (and potentially also PSVR).

I have also to mention that HitMotion had been originally developed with Unity 2017/2018 and we had to make a lot of work to create a cross-platform development environment. I’m sure that other game developers can hear my pain: we had to create a system so that we could use the native SDK of every headset when building for it. All of this considering that SteamVR conflicted with Android builds, so our system had to hide SteamVR directories when building for the Focus and Quest. We created a complex series of Unity scripts + PowerShell scripts to make everything work (kudos also to the other dev Federico Gallucci for having helped me with this). It was a complete mess, and also could give us some problems with Git, because if two devs pushed on the repo by having the system configured for two different headsets, some conflicts may happen.

Now, thanks to Unity 2019 and the Unity Interaction Toolkit, we have been able to make the game fully cross-platform in a very easy way. We have released it for Quest, but we could potentially deliver it also for many other headsets by just making little tweaks. Recently, in my review of the Vive Focus 3, I highlighted how we made an alpha of the porting to HTC’s headset in just 30 minutes.

https://gfycat.com/leadingoblongindianpalmsquirrel

Notice that some little changes are always necessary when porting between the headsets: for instance how the boxing gloves fit your hands depends a lot on the shape of the controllers you are using. In this case, we are detecting at runtime what is the current XR system in use (Unity.XR allows that pretty easily) and we change some settings accordingly.

And I’m sure that when OpenXR will be embraced by all companies, cross-platform builds will be even easier…

Bigger market

Being available on a bigger number of headsets (in particular the Quest, which is the most successful device at the moment), the VR version offers the opportunity of reaching a bigger market. Since we as a development studio want to survive (and even better, to succeed!), the VR version offers us much more opportunities for our future. This holds for the present and short-time future: when AR glasses will be widespread, we will wonder again what to do with our game.

WOW Effect

Passthrough AR was something unique of our game HitMotion: Reloaded on Vive Focus Plus, and it made our game special. When Max and I took the game to the WCVRI in Nanchang (China), every person we demoed HitMotion to was surprised to see his surroundings. Sometimes I just put the headset on my head and waved at people passing by, and they were amazed I could see them. This was something I personally loved about the game, and that I will miss about it. Well, for sure until Facebook will release its passthrough SDK…;)

via Gfycat

---

I hope this journey in the differences between a VR and a passthrough AR game has been interesting for you. Don’t forget to play HitMotion: Reloaded on SideQuest, and if you have any questions or considerations, write them in the comments here below!