All the most interesting AR and VR news from MWC 2018

Some days ago, the Mobile World Congress 2018 has finished. I expected a lot of AR/VR news from there, like the announcement of the price of the Vive Pro or the release of the Oculus Go by Xiaomi and Facebook… but actually I’ve been disappointed by everyone. The number of announcements or important news has been close to zero. When I talked about this disillusionment, Mr. President (Alvin Wang Graylin) told me that it is an event about smartphones, so of course, there hasn’t been much attention towards VR. He has a point.

Anyway, there have been some things worth attention and I’m writing this article to report them all to you. Let’s start!

HTC Vive

HTC has maybe been the virtual reality company that has shown the most interesting things at this MWC. And for sure it has helped the fact that it is also a smartphone company, so it had all the interest of having a big booth at an exhibition targeted at mobile phone vendors.

The CEO of the company, Cher Wang, has made a long speech about the future vision of the company, that encompasses mobile, AR, VR and AI. From the talk, it is clear that HTC is betting a lot on these new technologies and envisions a future where all of us will wear these devices every day. The smartphone division is not going in a stellar way and the company has just re-organized itself so to merge the smartphone and VR divisions. IMHO all of this means that for HTC the XR technologies are a natural evolution of smartphones and will substitute them in the long term, so the company needs to unify its two divisions so to have a more seamless evolution from one business to the other.

In Barcelona, various journalists have been able to try the new Vive Pro and the Vive Pro with the new Wireless Adapter. There was a setup that let them simulate a fly inside a ballon and it worked completely wireless. Feedbacks on these two devices have been very positive, even if everyone said that to express a more reliable opinion, they would need to stress-test the devices more.

Who wants to go on a balloon ride? @htcvive #vivepro and @janimationinc pic.twitter.com/sdkM47NCCe

— Dan O'Brien (@obriend17) February 26, 2018

Ben Lang of Road To VR has also been able to try the Vive Focus and to write another awesome review of his: the feedback is that it is an awesome headset, very comfortable, with a great positional tracking and with fantastic visuals. But of course it has also some shortcomings: first of all it is very expensive (more than $500) and so no general consumer would buy it, and then the controller is only 3DOF and its emulation mode that should be able to somewhat allow it to simulate a 6 DOF controlling scheme doesn’t work well at all.

Company Mi Hiepa Sports has showcased a soccer training simulation that used Vive Trackers v2.0, that is the Vive Trackers that are compatible with the new SteamVR 2.0 tracking technology. It has been the first time that they have been showcased publicly and it has been a way that HTC has used to say “chill out, we have them and they work”. But we still don’t know anything about price and availability.

The booth of HTC was full of installations: for instance you could play at Arizona Sunshine with the Vive Pro; you could see a collaboration tool called Engage; you could feel the fear of height with Richie’s Plank Experience; you could try and experience where a little avatar of yourself could help you in overcoming your fear of heights; and especially you could see the new VR experience developed by Innoactive for Volkswagen: it is a training and collaborative environment that Volkswagen will start adopting in its industrial environment. It plans to train 10000 employees in VR with this framework this year. Do you now understand why training is currently considered the most important sector for VR?

Samsung S9

Samsung has released its new flagship phone: the Samsung Galaxy S9. It has a 5.8-inch 1440 x 2960 display and features the new Qualcomm Snapdragon 845 chipset and it is, of course, the most powerful Samsung phone ever.

This is surely an advantage in virtual reality: and the new S9 is both Gear VR and Daydream compatible, so you will be able to live a new awesome level of mobile VR graphical quality with the headset of your choice. What has surprised me is that a new Gear VR headset not been released: the Galaxy S9 will work with 2017 edition. I think that Samsung has realized that releasing a copy-paste new device each year had little sense and so kept the previous edition.

Qualcomm has hinted that thanks to the powerful AR/VR features of its chip, this phone would potentially be capable of performing 6 DOF virtual reality, but it would need the development of a new headset by Samsung and Qualcomm together: the standard Gear VR is not enough. As I said in my post about ARKit, for sure with ARCore and Gear VR is possible to hack a 6 DOF headset, but its performances would be poor for a comfortable VR experience (where every little latency and tracking error lead to motion sickness of the user). So, it is ok for an experiment, but it is not ok for a commercial solution. A new headset is needed.

Regarding the AR features, of course, the Galaxy S9 will be ARCore compatible… but it will also introduce AR Emojis, that are…. ehm, the cut-and-paste of the Animojis of the iPhone X. You have a 3D avatar that resembles you and that can follow exactly the movements of your face using the phone frontal camera so that you can send your emotional state to your friends. While I find it completely useless, I find interesting the find that this means that the phone is able to track your face using the frontal camera in a reliable way and this can be very useful for a lot of AR applications (e.g. to let you try some glasses before buying them).

After having read its feature on the official website, I have to say that this is not the only feature that has been copied from the iPhone X.

Varjo Bionic Display

Varjo has showcased its Bionic Display to the public. Varjo is that Northern European company that is aimed to create a virtual reality headset geared at professionals that has a high-resolution area of the display in the center of vision. They have an approach a bit like the foveated rendering: inside the headset, there are two displays: one is like the standard screen of the Oculus Rift and the other one is a microdisplay that has a very high pixel density. Using some optical magic, they plan to show the content of the microdisplay in your center of vision, so that you have a super-high resolution at the point you’re looking at and a decent resolution in the zone outside your fovea.

Currently, the microdisplay output is fixed in the center of the display and this means that you can exploit the magical high-res area only if you look straight in front of you. The company plans for the future to use eye tracking plus some opto-mechanic device to move this high-resolution area so that to follow your eyes. Current feedbacks are very good: the area of high resolution is highly impressive. The problem stands in the evolution: we are all very concerned about how they will be able to merge the content of the two screen so that:

• You can’t see the seam between the two areas;
• The moving high-resolution image doesn’t jitter;
• You don’t notice that there is a moving image at all.

Varjo says that they have prototypes in this sense, but after having read the articles of Karl Guttag on the topic (this one and this other one), I’m not that convinced about it.

Anyway, Varjo has also announced that thanks to a technology supplied by SocioNext, it will also offer a see-through feature, letting you see the real world through the display. This would let Varjo offer also augmented and mixed reality features and this is surely a good news. It is not clear if this functionality will be offered from the headset itself or thanks to an add-on, though.

Vuzix Blade AR Glasses

Vuzix has showcased its Vuzix Blade AR Glasses, that are some augmented reality glasses that can project some useful info into the corner of the lenses (so, they’re a bit like the Google Glass).

The coolest features of this glasses are:

• The battery, the projectors, etc… are all contained in the frames of the glass;
• The glasses are powered by Amazon Alexa and this means that you can use them with vocal commands;
• You can also command them with swipes on the right side of the frame;
• They have a nice design, so they are not awkward to wear.

The price to have such a nice device is €1080.

5G

If there is a technology that will empower both smartphones and VR headsets, that is 5G. This new technology standard reduces a lot the latency of communication and increments a lot of its speed. As UploadVR makes you notice, the improvement over 4G is impressive:

It can cut response time down to 1-2 milliseconds (0.001 or 0.002 seconds) from 4G’s average 50 ms (0.05 seconds). Similarly, it can increase the data rate from 4G’s average 0.02-0.03 gigabits per second to between 0.1-5.0 Gbps.

This could finally allow for streaming virtual reality to become a reality. Thanks to the reduced speed, the latency will go below the one that causes motion sickness and so you would be able for instance to play a VR game completely in streaming. This is currently not possible because once you rotate your head in a direction, the system should be so reactive to show you immediately the image you should see, but because of the streaming latency, this is not possible and only happens after various milliseconds. 5G would allow exactly fast reaction times, letting you rotate your head and move as you wish without feeling nauseous.

This would allow the standalone and mobile headsets to have finally the computational power of the tethered ones, unleashing completely their power. Currently, the only advantage of tethered headset over standalones is the computational power, that is far superior on a GTX-1080 powered PC than on a Vive Focus. But if the Vive Focus is able to stream a game rendered on the cloud, it could become even more powerful than the PC. Do you now get why Mr. President has told me that he believes in standalone headsets?

Furthermore, 5G would allow for higher quality of the streaming of VR videos, especially the new upcoming 6 DOF videos, that should come at the end of the year. But don’t get too excited yet: 5G still needs some time to come (it is expected to start this year, but to roll out completely in 2019-2020).

At the MWC there have been various company showcasing 5G demos, with or without virtual reality. For instance SK Telecom has showcased a 5G powered social VR demo.

I hope you enjoyed this round-up. If this is the case, subscribe to my newsletter to show your appreciation!

(Header image by MWC)