Viveport Video “6DOF Lite” mode review: move your head in 360 videos

At VEC 2019 HTC announced the “6DOF Lite” mode for Viveport Video, promising the fact that we would have been able to watch standard 360 videos with the possibility of actually moving the head in the video. But… does it work? Is it good enough? Well, yesterday I’ve tried it at HTC China HQ in Beijing, so let me tell you my hands-on impressions.

What is Viveport Video 6DOF Lite Mode?

The idea of the 6DOF Lite Mode for Viveport Video is quite simple: resolving one of the biggest issues of 360 videos: the fixed point of view. When you are in a 360 video, you often want to move your head to reach out something that there is close to you (especially in porn, a friend told me), but actually, nothing changes and the video appears always the same. This breaks the magic of VR and makes the viewer frustrated.

Furthermore, the fact that moving your head in the real world leaves the point of view in the video unchanged causes virtual sickness: the brain can’t understand why you are moving but actually you are always seeing the same visuals and so it triggers its defenses.

The solution to this is volumetric videos, that is full 6 DOF videos with the complete 3D reconstruction of the scene that is around the camera. But they are damn big and expensive.

Trying to create a compromise to start solving this issue, HTC has so created the 6DOF Lite Mode for Viveport Video. You can take a standard 360 video (so even one just recorded through a Gear360), play it through Viveport Video and it magically gives you some kind of volumetric effect, in the sense that you can move your head up to 1m in all directions and actually see your point of view in the scene changing.

6DOF Lite Mode is already available on Viveport Video for the Vive Focus Plus and it will be distributed for Vive and Vive Pro soon this month.

Viveport Video 6DOF Lite Mode review

At HTC China Headquarters in Beijing, I have been able to try the 6DOF Lite Mode on a Vive Focus Plus, with 4 videos in total. The videos were of different kinds (animation, dancing, movie), but all were standard 360 videos.

When I put the headset on and started the first video, I have not noticed any difference with a standard 360 video. Then I started taking some steps forward and I saw that something was happening. The video didn’t appear fixed as always, but it started distorting itself to show some visuals that should be more coherent with my new position.

I expected some AI magic able to reconstruct a full volumetric video from just a 360 one… maybe I just want too much :D. Actually, the 6DOF Lite video is more like a “smart zoom&distortion” to give you the sensation that the video is reactive to your head position. So, for instance, if you have a person in front of you in the video, and you take a step towards him, he will look a bit bigger, and this is coherent to your movement. But if you keep going forward, you will see just him becoming always slightly bigger, but you will never reach him, and this breaks the magic again. The reason is that actually in the video there is no depth info to tell the system where is that person, so it just does what it can.

This illusion works well for some things and worse for other ones. For the floor, it is like black magic. If you look at the floor and take a step forward, you actually see the movement that you have made reflected in the video very well and it’s mind-blowing. For objects that you have in front of you and that you try to reach going towards them, it works a bit if you don’t move too much. For objects that you have in front of you, and that you try to move perpendicularly to them, so to look what’s behind them, the illusion doesn’t work (the video has not the info on what there is behind every object that has been pictured). Other objects, especially in very complex scenes, will look like a bit distorted while you’re moving.

Just to make an example: in one video, I was in the second row of the audience of a performance by some cute Korean girls (eh, the hard work of the VR blogger…) and walking forward towards the girls, I was:

• Disappointed because I could never reach them. Walking towards them, I had the impression that they were always “too far away”, with the visuals just “zooming a bit”;
• Pleasantly surprised because I could actually see the guy of the first row going behind me as soon as I started walking towards the stage.

So at the same time, I had something breaking the magic (I could never reach to the girls) and something else increasing it (the guy of the first row disappearing behind me).

The final results depends on many factors, like what you are looking at (for people, we are able to spot easier that something looks weird), how you are moving, how much you are moving, the quality of the video (the more resolution the video has, the better is the resulting quality of the “distortion”), how much complex is the scene (the more objects in the scene, the higher the probability that something will look distorted), etc…

Basically, the system tries to do the best that it can with the data it has available. And since it has just a monoscopic 360 video (that has so not even any depth info), the only thing that it can do is distorting it to accommodate it to your viewing position. And it has to run locally in realtime on a mobile processor, it can’t use super-complicated AI algorithms. Maybe in the future, with 5G streaming, it will be possible to do something like that in the cloud. For now, since everything runs on the device, it is hard doing something that works that much better. For this reason, this new mode is called “6DOF LITE” and not “Full 6DOF emulation”.

In the end, I can say that “it is a nice beginning for 6DOF emulation”. It isn’t perfect, but it is nice to have. In the same video, while moving, you will have the impression that something is reacting well and other things less well. When it works, it gives really a nice sensation, it seems that the video is “more real”, more credible, and so the sense of immersion is increased.

It doesn’t look at all like a volumetric video, but it looks way better than a monoscopic 360 one in a standard player. For sure, it helps a lot in fighting the motion sickness: the visuals change when you move, and so you have not the effect of being in a sphere attached to your head. I think that it is good especially if you don’t try to move the head too much. I am very curious to see its next evolutions after this first version.

(Header image by HTC)