All the best AR and VR news from CES 2018

The past week has been the week of CES, the exhibition where a lot of companies showcased innovative projects or announced disruptive news. Of course it has been an interesting event for AR and VR too: in the first two days, we had some key announcements that will shape how the XR market will be in the upcoming months.

I’ve not been at CES, but I’ve followed the CES news thoroughly every day and I want to collect here in this article what have been in my opinion the most important ones. This way, if you hadn’t the time to check your news feed this week, you know what you’ve missed and you can get on par with the rest of the XR innovators. And if you have already read the news about CES, you can scroll this article and see the ones that you missed. (And, by the way, if you like this way of mine of collecting news… I do it every week in my weekly newsletter… so, subscribe to it using the form at the end of this article!)

Are you interested? Well, let’s start…

HTC Vive news: Vive Pro, Vive Wireless Adapter, Viveport VR

This has been for me the most important XR news of all the CES: HTC has announced a new upgraded version of the HTC Vive, the Vive Pro, which features some evolutions over its ancestor:

• an improved 3K resolution
• integrated audio
• SteamVR v2 tracking
• much improved comfort
• 2 frontal cameras for possible mixed reality applications

This is not a disruptive evolution of the Vive, but surely a great incremental one, that makes it on par with other headsets as the Samsung Odyssey. Regarding the controllers, there is some mystery about if this will ship with Valve’s Knuckles controllers or an upgraded version of Vive Wands, but it seems that the latter will be the most probable solution. The device will be sold starting Q1 2018 as an upgrade kit with the headset only and in Q2 2018 with all the bundle including Lighthouse stations and controllers.

Together with the Vive Pro, HTC has announced an official wireless adapter for the Vive and Vive Pro, made in collaboration with Intel. It will be released this summer (so Q3 2018) and first tests at CES report that it offers good comfort, a great quality of the image and great stability, but it has some latency issues. We all hope that this problem will be solved in the upcoming months.

The last news is that HTC is investing a lot in its Viveport software platform: it has just re-invented its interface, that now is ideal to be used inside VR and offers the innovative feature of letting the user enter the environment of a VR game before deciding if buying it or not.

I’ve covered in details this news in a previous article of mine, so if you want more details, click here.

Lenovo Mirage Solo and Camera

The image above shows you the Lenovo Mirage Solo, a standalone VR headset that Lenovo has made in collaboration with Google. It was since Google I/O that we were waiting for this and finally it has been announced. This headset uses Google Worldsense technology to offer you 6 DOF tracking (that is, you have room-scale), but the controller is just a 3 DOF remote with some keys and a touchpad. Apart from Worldsense, the other signs of Google collaboration for this device can be found in the use of the Daydream platform.

Regarding the main specs:

• 2560 x 1440 resolution
• 75Hz FPS
• 110° FOV
• 7 hours battery duration.

The first review reports that it is a good headset, even if it may have some issues with comfort, also because it is very heavy (it weighs more than 600g). The comparison with the Vive Focus shows that the Focus is more performant and is more comfortable, but it is also more expensive.

Lenovo Mirage Solo will be available in Q2 2018 for a price below $400.

Beside the Mirage Solo, Lenovo has also announced a camera made in collaboration with Google, that lets you shoot 180° 3D video in 4K very easily. It is like a standard compact camera, but it has two photographic lenses, so it can record a stereoscopic hemisphere of what is in front of you. This way, you can easily record memories and then see it inside your headset and live them again. Or you can easily share them on Youtube, where there is the new VR 180 video format. We still don’t know the price of this device.

If you want more info about the hardware that Lenovo has announced at CES, check out my related article.

Oculus partners with Xiaomi

During the CES, Oculus has published a news on its blog announcing a partnership with XiaoMi regarding the Oculus Go. If you read my previous post about the Go, you would notice that I’m not surprised by that, because since the first time I heard talking about the Oculus Go (when it was still codenamed Pacific), I knew that it would have been made in collaboration with XiaoMi.

But what has surprised me is that actually Xiaomi will make a Chinese clone of the Oculus Go: a device that is completely identical, but uses Mi VR as the software platform, so that it can run happily in China, where Facebook is not very loved by the local government. The device will be called Mi VR Standalone and is the Oculus way to enter the Chinese market, where currently is not allowed. This is a very interesting choice and lets Oculus compete in an enormous market where basically HTC has the full control: I’m wondering if they’ll make another partnership to sell the Rift CV2, when it will be released.

Hugo Barra, the “CEO” of Oculus (I use the quote because actually there’s not such a role) worked previously at Xiaomi and while he says that his role is not a part of the agreement between the companies, surely he has helped a lot in creating this partnership. He also has helped a lot with this idea of an affordable VR hardware.

So, while HTC is aiming at attracting professionals with its high-quality Pro device, Oculus tries to take the most number possible of people inside VR with its cheap headset.

Pimax 8K disappointment

Since its successful Kickstarter campaign, we’re all very interested into the Pimax 8K, the virtual reality device designed by the Chinese company Pimax (that already shipped a 4K headset) that should feature an amazing 2*4K screen resolution and 200° FOV. The device should be shipped in January to backers, but the company is having some problems in manufacturing it (the project is very ambitious, so it is not easy…) so there will be delays.

The company tried to take its v5 prototype to the CES nonetheless and people that have tried it have come out pretty disappointed. Particularly there are issues with:

• Comfort
• Optics (there are some distortions on the peripheral part of the vision);
• Display (not crisp quality, mura artifacts and the framerate is only 80 Hz);
• Tracking (SteamVR tracking is not implemented well).

Furthermore, some people also complained about the booth itself, where for instance there were people eating in front of journalists and the overall impression was not elegant.

The issues of the device don’t seem so bad that they can’t be fixed, but they show that the road to have a final HMD that can be used by us all is still long. Pimax has answered with a letter where it shows its roadmap, but the bad feedback from CES has already hurt a bit the company reputation inside the VR community. Anyway, I still think (and hope) that they’ll be able to ship what they’ve promised: the problem is in understanding when this will happen.

TPCAST improvements

TPCAST, the Chinese company that has offered for the first time the ability to make the Vive wireless, has announced some news. I think that they knew that Vive was going to announce its cool wireless gadget, so they decided it was time to answer properly. So they have unveiled:

• The TPCAST Plus: an improvement over the current TPCast device, that has been made better following the feedback of the customers: so there is a new design that should make the device more comfortable; an improvement over the stability of connection and interference reduction; a new battery design. It is great seeing such improvements, even if a quick test from Upload VR was far from enthusiastic from it;

• The TPCAST Universal: a reference design for a wireless adapter for Microsoft Mixed Reality headsets. It is strange that, in this case, TPCAST won’t manufacture the device itself, but it is looking for OEMs interested in producing it as a standalone device or a piece integrated inside VR headsets. Creating such a device is far from easy: if standard wireless adapters are already difficult to design, imagine the problems of engineering something suitable for inside-out tracking, where you have also to transmit  all the data read from the tracking cameras from the headset to the PC;

• The TPCAST v2: the new generation of wireless devices, featuring support up to 8K headsets! As the press release states:
  

  The TPCAST 2.0 technology includes three main features. The first element is the ultra-low latency codec, capable of compressing VR content on a 50:1 ratio which significantly reduces the bandwidth required for video data transmission while bounding the latency to 1ms. This easy-to-use, stable, and ultra-low latency codec enables TPCAST’s product family to deliver consistently high-quality real-time video transmission. The second feature of the TPCAST 2.0 technology is its scalability making it easy to adapt to any headset including 3k, 4k, and 8k video resolution HMDs in the market. The third feature is the capability to preserve the user experience compared to a wired connection. TPCAST’s proprietary technology, and accumulated VR knowledge base, gained by its long-term leading position in the Wireless VR space, enables TPCAST’s products to deliver a stable and commercial-grade VR experience.

So, TPCAST has answered to HTC by offering a wide range of products, that aim at making wireless every kind of VR headset, including the Rift, the Windows Mixed Reality ones and maybe even the Pimax 8K, considering the high resolution supported by the v2 version. Anyway, I think that having the competition of HTC and of Intel (with DisplayLink) won’t make things easy for this Chinese company, especially regarding the Vive market.

Pico Neo hands-on

Do you remember the Pico Neo, the Chinese standalone headset with 6 DOF tracking and two 6 DOF controllers tracked using ultrasonic technology? Well, at CES various people have been able to test it, so we have now some first-hands impressions about it.

It seems that the device is good, but it has some tracking problems: the headset sometimes have some hiccups, but the controllers are the true problem. The tracking has still to be implemented well, because it has noticeable delays and problems regarding tracking precision and reliability. Pico has stated that it is working hard on fixing this, but surely it is something that you have to take in count when preordering such device. It seems that tracking reliably the controllers inside a standalone headset is far from easy and this may be the reason why Vive Focus has only one 3 DOF controller.

Anyway, the other interesting news regarding the Pico Neo is that it will feature Viveport platform even in the Western market edition: this means that HTC has plans of using its store not only in China, but in the whole worldwide market.

uSensAR platform

Continuing talking about Chinese companies, we had another interesting announcement by uSense. This Chinese computer vision company, after having stated that it wants to offer a hand-tracking module (something like Leap Motion) for the Pico Goblin headset, has unveiled its AR plans for the future.

And these plans are very interesting. You surely know about Apple ARKit and Google ARCore, don’t you? These are amazing mobile AR framework, with which you can make amazing mobile AR apps. But they have the problem that can be run only on some enabled devices, like the iPhone phones (from 6S to iPhone X) and 5 Android smartphone models. There is the ARCoreForAll hack that can help in this, but it only enables 3 more phone models. Well, uSens plans to make something like ARCore, but that can work on every smartphone in this world: this platform is called uSensAR.

No one has been able to perform a public test of this framework, so we don’t know how it works well (ARKit and ARCore have already thousands of people reporting enthusiasts feedbacks)… we only know that in Q1 2018 uSense should release the SDK, with both support for native C++ programming and for Unity, so we will all be able to test it. I admit that this is one of the news that has excited me the most about the CES: if this platform will work well, it could be a game changer, enabling millions of people to use and develop mobile AR applications.

You can register for the uSensAR platform SDK on its official website.

Intel NUC is VR-ready

Intel makes since years very compact PCs called NUC that are very useful to be embedded inside kiosks or other enterprise installations: we used these little boxes for instance at Immotionar, when we just needed PC stations that had to take data from Kinects and stream them to the main VR PC.

Well, it seems that the latest NUC made by Intel, the NUC8i7HVK, is VR-ready and is able to let you run your quality VR experience at 90 FPS. This has been possible thanks to the collaboration of Intel with AMD: the use of the latest Intel processor with Radeon Vega M graphics chipset enables enough computational power to run VR.

While this announcement is almost useless for consumers (almost no one uses a NUC at home… you just use a desktop PC or a laptop), it is very interesting for enterprise users: NUC is so small that it is easily portable; furthermore it is also easily embeddable inside kiosks or other kinds of installations. It is perfect for every kind of headless VR installations, where you just need a computational core and you don’t need a keyboard and mouse attached every time.

LooxidVR headset

You all know how I’m interested in Brain Computer Interfaces, since they’re the final paradigm of human machine interactions, the one that will let us become cyborgs as Robocop. So, when at SIGGRAPH 2017 Neurable has showcased its mind controlled game The Awakening I become very excited by that.

At CES 2018 another company has taken the stage regarding the use of BCI with virtual reality and this is Looxid Labs that has showcased is LooxidVR headset.

This is a mobile 3DOF headset (so you use it with your Android mobile phone) that embeds EEG sensors and eye tracking modules inside it. While you use it, the system uses some proprietary algorithms that mix data from the EEG waves read from your brain and data regarding the eyes position to analyze your emotions in response to the VR experience that you’re having. So, while Neurable seems more interested in using BCI to offer a new UX paradigm, Looxid wants to analyze what the user is looking at and what he/she feels while looking at various elements of the experience.

The headset is obviously not aimed at consumers at the moment, but more at professionals (as for instance psychologists, doctors or trainers) and enthusiasts. We don’t know its price or have any hands-on, but it will be available for pre-orders starting from the 1st of February.

NextVR 6 DOF streaming

We all like watching 360 or 180 VR videos, but we all hate one thing about them: you can’t have positional tracking. So, you are in the video and you would like to move your head to come closer to one of the characters or to see what’s behind him, but since the video is just a recording, you’re not allowed to do that. There are some companies working on that, like for instance Lytro, but their videos require very complicated hardware for the shooting and the resulting file is so big that it is almost unusable.

NextVR, a company whose job is offering the streaming of various worldwide events (it has just signed an agreement with the WWE) in virtual reality, is planning to offer 6 DOF tracking in its future streams. This would allow you to watch that event in a more natural and immersive way, because you would be able to move your head like you do in the real world. This would require some technology that not only records the event using lightfield technology, but that also makes the resulting file compact, so that it can be streamed over the web. The company hasn’t detailed how it plans to do that and this makes me cautious about its statement, but if they managed to do that, they could revolutionize VR streaming.

Intel Studios

Of course, NextVR is not the only company working on something like this and among its competitors, there’s also Intel, that is working since months on a similar technology of volumetric videos. Intel said that wants videography to evolve, so that the basic unity of videos changes from pixel to voxel (so videos become a 3D experience, as a video game) and dreams a future where everyone can experience the various events from all over the world in VR, exactly as if they were there. Currently, they have a working tech for recording volumetric videos using multiple cameras , but it is not that feasible for streaming since it employs 3 Terabytes for every minute of recording (Yes, you’ve read it well).

At CES it has also announced Intel Studios, an enormous warehouse full of technology (e.g. bazillion cameras and servers) where there is a 50m x 50m dedicated space for volumetric recordings. I think that they’ll rent this space for companies wanting to shoot 3D VR videos with an optimal quality (exactly as Microsoft offers his services for 3D avatar recording, as we’ve seen in Blade Runner 2049 Memory Lab). I’d really like to see a movie shot inside this futuristic place…

Tactical Haptics controllers

Tactical Haptics is developing some VR controllers that have a unique feature: they can be combined between them to obtain a form factor that is perfect for the VR experience the user is living. Let me explain that better: you have these big controllers in your hands and these have some sockets with magnets inside. You can use these sockets to attach one controller to the other in a way so that the combined shape is similar to the one of the object that the user is handling in the virtual world. So, for instance, if you attach a controller one in front of the other, you can simulate a machine gun, while if you attach one on top of the other, you can simulate a stick that you hold with both hands. Besides this feature, these innovative controllers also feature a special grip mechanism that is able to give you force feedback that can make you feel torque forces in your hands.

https://gfycat.com/uncommonspiritedaustraliancurlew

(GIF from Road To VR)

Tactical Haptics controllers are still in development and currently they have Oculus Touch controllers attached to them to track their pose and this hybrid configuration is being used for fast prototyping. In the final version, the Oculus controllers will be removed and the devices will be ready to be used inside some VR arcades.

The review from Road To VR of these devices is positive, even if they’ll still need time for the final refinements.

Contact CI’s Maestro Haptic Glove

Continuing speaking about hands controllers, we come to one that has astonished an experienced VR journalist as David Jagneaux: the Maestro haptic glove, developed by company Contact CI.

As you can imagine from its name, this is a glove for VR interaction and this glove is not made only to give you finger tracking, but also haptic feedback. The idea behind the Maestro is replicating the exact anatomy of the hand, with tendons and muscles and applying to your hand a counter-force related to the haptic sensation that you have to feel. Let me explain that better: if you’re in VR and you want to press a button on a wall, most probably you’ll see your hands pressing the button and then penetrating it, since there’s no feedback from the virtual world at the moment. This unrealistic behavious breaks the immersion. With Maestro Haptic Glove, instead, in the moment that you are pressing the button with your finger in VR, the glove will give you a touch sensation on the tip and if you wanted to go beyond the button, the glove would start giving your finger a force that is exactly the opposite of the one that your finger muscles are applying, meaning that your finger won’t move, so can’t trespass the virtual object. Of course, your body could be able to go beyond the force of the actuators of the device, but the illusion is so realistic, so well made thanks to special configuration of the glove, that the brain believes that there is really an obstacle that prevents your finger from going further and so believes in its existence. The journalist says that when he touched a virtual desk, the touch sensation was so real that he thought he had touched a real object. I’m really amazed by that.

The system is still a prototype and finger tracking is not optimal yet… but it is one of the most promising projects of this CES. I hope to be able to try them in the future, because I really want to feel this realistic touch sensation, too!

RealMax AR glasses

A company based in Hong Kong, Realmax, is trying to enter the AR glass market. The device is still a prototype, and looking at its photos, it seems just a collection of devices glued together by some maker, so it doesn’t seem able to compete with HMDs as HoloLens, Meta 2 or Magic Leap. But RealMax has an ace in the hole: its impressive field of view of 100 degrees, that is the biggest FOV of all the AR headsets out there (more than 2 times the one of HoloLens!). Quoting the article on The Verge where I found this news:

The company claims it can achieve this vastly superior FOV only by using some proprietary optics, which involve a combination of so-called waveguide and freeform technology to control how light is beamed out from a source and then reflected back onto the lenses a user looks through.

The problem of RealMax is that it has amazing optics, but lacks fundamental features of an AR headset, like hands tracking and environmental recognition. For this reason, we’re not sure if they can compete with the big guys, but for sure they could survive in some niches (or be acquired by a bigger company).

Tesla Suit

Tesla Suit is a product whose name is around in the VR ecosystem since a while and it is a suit that you wear to have your full body inside VR experiences. It is a very expensive device and it is obviously targeted to enterprise usages, as for instance arcades or military simulations.

At CES 2018 Tesla Suit has been added a new interesting feature: a set of electrodes, distributed on all the vest, that can give you a little electrical shot to your body parts. This is useful to give a sense of touch distributed on all your body, but also a sense of pain if the shot is made more powerful. This could be fantastic to play VR action games inside arcades: imagine the possibility of really feeling the sensation of having been shot while playing a war game!

The company is going to ship dev kits to interested developers this year and is planning “consumer” release for 2019.

Ready Player One Sansar experience

Intel, Warner Bros, Sansar, and HTC have unveiled a digital place that you can experience inside Sansar and that lets you enter the garage of Aech, the best friend of the main character of the movie (and the novel) Ready Player One. This is one of the most visited places of the whole movie and for sure fans of RPO will be eager to be able to enter it, maybe with their friends. Now it is possible: if you log in to Sansar, you’re able to feel inside the Ready Player One movie.

I admit that I’m not that hyped by a static environment of a movie, but if you are a fan, I think you would be delighted by it.

Mayweather Boxing

Floyd Mayweather has decided to enter VR. During the CES he has announced Mayweather Boxing + Fitness VR experience, a virtual reality application that will let you train with your boxing idle. This program is not a game and is made as a complementary boxing training that over 3 months should increase your skills; or as an experience to let newbies enter the world of boxing.

This application won’t be available in the stores (even if I hope that later it will be), but you’ll be able to try it only inside the “Mayweather boxing + fitness” gyms. What a pity.

Other news

Some other news that I’ve found that deserve a little mention are:

• Peraso has announced that it is developing a reference design for wireless VR. But differently from other ones in the field, this startup doesn’t to work in the Wireless adapters field, but does want to design wireless modules to be embedded inside VR headsets;
• NVIDIA in his press conference has said that it predicts that in 2021 we’ll have 50 million VR headsets. I think it is referring to tethered and standalone headsets, because if we include cardboards we’ll probably reach that number at the end of this year. 50 M is a great amount, but I really hope that in 3 years will be beyond that mark;

• Zeiss is releasing Zeiss VR One Connect, a software and hardware solution that lets you use your mobile headset (even a non-Zeiss one) to play SteamVR games. The system features a runtime plus two 3 DOF controllers thanks to which you can interact with your VR experience. To play the VR games with your smartphone, you have to run the Zeiss application and then plug your mobile headset into your PC using a USB cable, so that SteamVR detects your headset and controllers. The system is very similar to VRIdge, with the difference that you have to plug your mobile headset into your PC and this is in my opinion a killer for usability. Basically, it serves to transform your cheap mobile headset to a low-tier tethered headset;
• Flex is entering the mobile AR headsets with Flex AR, a reference design for augmented reality glasses for enterprise usages that will use Atheer’s AiR Enterprise productivity solution;
• Black Box VR has presented a program to get fit in virtual reality while having fun. You can use this application together with some gym machines, like for instance a chest press machine, so that you do the standard exercise, but without just doing some dull movements, but actually having fun inside a game. How much you do correctly the exercise while playing is not clear, but for sure this kind of application can motivate more people to stay fit;

• Reliefband Technologies, a company whose business is trying to avoid people throwing up, has announced the Reliefband, a device that should help all the VR innovators in not having motion sickness, by the stimulation of the vagus nerve. I was pretty excited by the news, but a fact check by redditors has revealed that actual studies on the device have found that it works as well as a placebo.

Further references

I tried to give my best to summarize the most important news taken from different sources, but if you want a more complete and official roundup, you can refer directly to the two most important magazines of VR, reading their official roundups:

• UploadVR (all CES news)
• Road To VR (Day 0, Day 1, Day 2, Day 3)

And that’s it! Hope to have reported all the most important news of this amazing event… but if I forgot one let me know in the comments. And also let me know what has been the announcement that has excited you the most: if I had to choose, I’d go for the Vive Pro for virtual reality and uSensAR for augmented reality. 

If you liked the article, please subscribe to my newsletter, this would make me super-happy!

(Header image by CES)