All the most important AR and VR news from the GDC 2018

As with every big event, I’m writing a post to try to summarize the most important news that have come out from the GDC 2018… so that if you have lost something, you just need to read this post to get in par with the rest of the world.

So, are you ready? Let’s start!

Vive Pro price and release date

HTC has finally revealed the price of the Vive Pro: it will cost $799 just to have the headset (no controller and no base stations, so you must already have a Vive 1 kit) and will be available for pre-orders since April, 5th. Because of this new headset, the old Vive will get a $100 discount and will so then cost only $499 for the full kit.

The price of the Pro received a lot of critics and angry comments from the VR enthusiasts, that were expecting something like $799 for the full package and $499 for only the upgrade. The problem is that HTC has decided to target this headset to professionals only, as HTC President for the Chinese Region Alvin Wang Graylin has told me both during an interview with him and after my hands-on with the device. And once you decide to sell a premium product, of course, the price becomes super-high and unaffordable for all normal people. This business strategy of selling to few people for a high price has completely sense, considering that all the device has been conceived to add all the features that could make it compelling mostly for professionals. But, as Road To VR has stated, maybe HTC should have clarified that in all its marketing material. Furthermore, it doesn’t even come with an embedded business license, so the confusion IMHO continues.

In my opinion, the true error is with the Vive 1, that will get a price that is higher than the one of all the competitors… and that will cost as much as a Samsung Odyssey, notwithstanding its far lower features.

Vive Focus comes to the west

I loved the Vive Focus since the first moment that I’ve read the news about it: a 6 DOF standalone headset with a wonderful light blue color and a great comfort seemed intriguing to me. Having had the chance to play with it for some days, I can tell you that it is really an interesting product (I’ll write a review about it here on this blog, so register to my newsletter to not miss it!). The problem is that we had no idea about the release date for us western people because the device had been announced and released only in China.

At GDC, HTC has announced that this great device will come for sure to the rest of the world and that this will happen “later this year”. I know, this is not a very definite time, but it is better than nothing: before the GDC we weren’t even sure if it would ever come here.

If you are a developer, you can fill out a form and try to request a devkit to HTC, specifying what is the project that you’re going to develop for this new peripheral. If this may interest to you, here you are the link for the form to be filled (note: I’ve provided you the link for Europe… if you’re from UK, just substitute “eu” with “uk” in the address; if you are from US, use “us”, etc…).

Vive Trackers 2.0 now on sale

This is a news that went almost unnoticed and that I discovered thanks to Upload VR. On Vive website is now possible to buy the Vive Trackers 2.0, that is the ones that are compatible with SteamVR tracking 2.0. The price is still the same ($99) and the trackers feel the same, with the only difference that they use the new tracking sensors. To make them different from the previous ones, HTC has made the HTC Vive logo on the device blue instead of white. If you want the new Trackers, when you buy them just be sure that the caption is “VIVE Tracker (2018)”.

LG ???

At GDC 2017 LG showcased a virtual reality headset that was SteamVR compatible: it was a game-changing moment since for the first time a new non-Vive headset would have been compatible with SteamVR tracking and ecosystem. Everyone was very interested in the project, but then… ehm, it almost vanished away. We had some new showcases and some little news about it (like for instance a possible name of LG Ultragear), but nothing more.

After one year, we continue ignoring everything and LG weren’t at GDC showcasing it. I think that, after a year, its specifications don’t sound exceptional anymore and maybe it is better that LG jumps directly to a second generation or that enters the standalone market (as it seems, since there are some rumors about a new revolutionary headset screen developed by Google and LG).

Oculus Go showcased

At GDC, the Oculus Go has been one of the main characters. First of all, Oculus has showcased all its main technical characteristics: I advise you to read the post dedicated to that on Oculus blogs, where Chris Pruett introduces you to them. Basically, they are:

• Hyper-optimization: since this is a standalone headset, Oculus has been able to optimize the hardware and software for VR usage. With Gear VR, you use your phone to experience VR apps, but your phone is a device that has to do a bazillion things (handle calls, messages, videos, VR, etc…), so it has to be good for lots of use. If you make a custom device for VR only, you can design it from day zero to be optimal to do only VR: this means that even if the Go is based an old VR 821 reference design, it has more power than a Gear VR mounting an S7 phone (and dissipates better the heat);

  

• 72 Hz mode: the headset can run not only at 60 Hz but also at 72 Hz. 72 Hz may not be useful for your app and you should activate it only if you think that your game can run at that frequency (so the graphics should be simple, for instance). The advantage of using 72 Hz is that everything appears smoother, plus the display becomes brighter and the colors warmer;
• Fixed foveating: we all know what foveated rendering is and we all know that to be implemented it requires eye tracking. But we also know that when in VR, most people just look forward with their eyes. So, exploiting this statistic and knowing that due to the various distortions, usually, the peripheral regions of the rendered images are the one made employing more pixels, Oculus has decided to apply a Fixed Foveating technology. This means that the peripheral regions of the rendered images are rendered with a downgraded resolution. This technology, that has been developed together with Qualcomm, guarantees a great boost of performance of the device with no noticeable loss during standard usage;

  

• Dynamic Throttling: the system auto-detects at runtime the CPU and GPU computational power needed by the application and can continuously tune the level of usage of the CPU and the GPU so that there is the littlest number of frames drop possible.

I’ve read a lot of hands-on reviews of the Oculus Go and they are all very positive: you can read for instance the ones of Upload VR, Road To VR and Ars Technica. All highlight how the device works like a charm, is very comfortable, has a great design (like all the Oculus products), a great LCD display with crisp visuals, great integrated audio and especially awesome lenses with very few god-rays (this is a new generation of lenses created by Oculus). The controller too is very comfortable and does its job very well.

I’ve asked Ben Lang of Road To VR also some details about the mura of the display and it seems satisfying as well.

A little preview just for you then! The mura was a little worse than I expected (because LCD in my experience is usually far better for mura than OLED). Without being able to do a side by side, I would say it was probabaly slightly better than the Rift mura, so not too bad.

— Ben Lang ᯅ (@benz145) March 23, 2018

Of course, the drawbacks are that this is a 3 DOF device with a 3 DOF controller, so its functionalities are very limited… and this is especially noticeable when playing a multiplayer game between Rift and Go owners. But this is also a very cheap device ($199), so it is normal that it can’t allow everything. Everyone has said that it is absolutely worth the price and that it will surely help a widespread adoption of VR.

Oculus Go should launch at F8 2018 in May, according to some unconfirmed rumors.

Oculus Santa Cruz new info

While Oculus has finally released almost every info about the Oculus Go, there’s still a lot of mystery about the Santa Cruz headset, about which we are all very curious after having heard so many cool things during the latest Oculus Connect. During GDC, Oculus has just:

• Released two new images about it, so that we have some new materials for our presentations on VR;

  

• Said that the Santa Cruz will be able to run at 72 Hz as the Oculus Go (even if honestly I hope that it will be able to reach even 90 Hz);
• Talked about its controllers: while at OC4 the Santa Cruz prototype controllers showcased trackpads on them (as the controllers of Oculus Go and Gear VR), for GDC 2018 they have been removed in favor of a stick and some buttons. The reason is to offer an experience that is the most similar possible on the Rift and the Santa Cruz so that the developers can easily develop cross-device games for these platforms. The reaction of the community to this news has been overly enthusiastic… I think no one loved the idea of trackpads.

Oculus Santa Cruz is getting new controllers, with joysticks and buttons for better compatibility with the Rift Touch. pic.twitter.com/XjLOvu7QlQ

— Olivier Palmieri (@Alkerion) March 21, 2018

Apart from the publicly shared info, Oculus has made selected developers to try the device behind the curtains (notice that journalists have not been allowed to that). The few of them that decided to talk about this experience were all super enthusiastic about it. The tracking seems very solid: both the one of the headset and the controllers and this makes me think that Oculus is on the right path to producing a marvelous VR device.

I had a chance to demo the Oculus Santa Cruz today at #gdc18! Quick take: this is the standalone headset the world needs. Clear unknowns around what kind of performance devs can expect, but really solid progress on SLAM, controller FoV, and ergonomics. pic.twitter.com/DkJND4m0QX

— Dimitri Diakopoulos (@ddiakopoulos) March 24, 2018

Vive Focus is heavier and the optics didn't feel as good. The inside-out tracking was more "swimmy" on Focus, but I tried it in an environment with a lot of dynamic features (i.e. people walking around), so I don't have an apples-to-apples comparison against Santa Cruz.

— Dimitri Diakopoulos (@ddiakopoulos) March 25, 2018

Can’t wait to buy a Santa Cruz!

Oculus still believes in the PC platform

Hugo Barra has said at the beginning of this year that his goal for 2018 was to see the standalone Oculus Go become successful in the world. In 2018 the whole Oculus seems completely dedicated to standalones, with the upcoming release of Go and Santa Cruz, so a lot of people started wondering if we’ll ever see an Oculus CV2 or if Oculus is now only dedicated to mobile headsets.

According to Jason Rubin (quoted on a Reddit post), PC is still an important Platform for Facebook:

I want to be really really clear here: PC is vital to the VR business. We think PC will lead the industry for the next decade or more.There’s so much more we’re going to be able to do with VR, and PC is where we figure that out. So Oculus is investing just as much in PC as ever. We’re in all theee categories [PC, mobile, standalone], and plan to stay there.”

This seems to confirm that we’ll see in the future (maybe 2019) a new tethered headset, as I had already suggested in a previous post of mine.

Oculus Home will get multiplayer and user-generated content

I like the new Rift Core 2.0 with its epic Home and Dash. But this new environment still lacks a lot of features and one of them is that it is not possible to host other people inside our virtual home.

Oculus is conscious about this big issue and is ready to solve that in the upcoming months: in summer 2018 will be released a new update that will allow people to meet their friends directly inside their VR homes, so that the Oculus Home will finally become our virtual home where we can host our VR friends and have fun with them. With the summer update will come another feature, that is the possibility to decorate our virtual houses with 3D models directly sculpted by us. The 3d models that can be imported have to be in the GLB file format, that is part of the specification of the glTF file format… so basically, the same 3D file format that is becoming popular in virtual reality.

Oculus is going to found big games

There are enough indie games on the Oculus Store, so Oculus is now planning to fund bigger games. It is something that they already announced a lot of time ago: they’re climbing the ladder of the games. They started funding little games and as soon as there were enough, they decided to switch to funding medium-sized games; now that the indie games are enough, they’re going to fund the big ones. Their goal is continuing this way until they fund AAA games. VR needs AAA games to appear compelling and Jason Rubin knows that.

What we think consumers want, and what’s going to work, is AAA content. And we’re going to invest in that.

Of course, indies won’t be super-happy of this decision, but unluckily this is how business works.

Oculus announces Hardware Reports

What is the most used Gear VR device? Do users use the Gear VR remote or not? What is the typical processor of PC VR users? These and many others are the questions that every VR developer asks when developing (and optimizing) his game. Today finding the answer will be easier, since Oculus has announced Hardware Reports, that is a place on its website where it is possible to read statistics regarding the usages of both mobile and PC Oculus headsets. From there you can discover interesting details about the most used hardware and software systems by Oculus users so that you can design your app so that it works well for the majority of VR users. One of the most interesting findings, that have been discovered by Upload VR, is that only 40% of the Gear VR users currently use the provided remote!

This is Oculus’s answer to Steam Hardware & Software Survey.

Magic Leap releases its SDK

We still haven’t seen a Magic Leap device, but at least now we can see its SDK. The Florida company has released during the GDC the Lumin SDK, that is the development kit that has to be used to develop for LuminOS, the custom operating system of the Magic Leap device. The SDK offers native APIs, but also supports popular game engines like Unity and Unreal 4. Together with the SDK, it is also possible to access the developer tools as for instance the Magic Leap emulator that I’ve used to infer the Field of View of the device. It is so possible to start creating your Magic Leap application. I have started digging around the SDK and the tutorials and I can say that until now it seems well done. (I have to perform further analysis and then I’ll publish a blog post about that)

From the docs of the SDK, it is possible to get how Magic Leap is able to understand the environment around it, to offer marker-based AR a la Vuforia, to track the eyes and the hands, use audio input, detect the light of the environment, and do a lot of other interesting things (it can even detect eye blinks!).

In the future, it will be possible to distribute the apps that you are going to create on a dedicated marketplace called “Magic Leap World”. Upload has also spotted the registered trademark “Mag!c shop”, but we don’t know how it will be used. While we wait for that, there’s a Twitter account, Made With Magic Leap, that showcases all the best works made with this SDK.

The last news regarding Magic Leap is that it has partnered with Mozilla to experiment with WebXR on such an AR glass.

We’re delighted to be able to share that we’re teamed up with @magicleap as an early access dev partner to explore spatial computing on the immersive web! #WebXR

— Mozilla Reality (@mozillareality) March 19, 2018

IMHO the release of the SDK is a good sign because releasing the SDK and then not releasing the device would really be a nonsense decision for Magic Leap, so now I’m a bit more confident that we’ll see something before the end of this year.

Realtime Raytracing is now possible

NVIDIA has announced RTX, a framework that enables real-time raytracing in its latest generation Volta GPUs (the architecture of the upcoming NVIDIA graphics card, already implemented in some enterprise products by NVIDIA). If you don’t know what is ray tracing, well, Wikipedia may help you, but basically, it is a rendering technique that tries to reconstruct the full path of the light that goes from the light sources to the pixels of the camera that is viewing the scene. This is a technique that is able to create photorealistic images and videos, especially for what regards reflective surfaces, but it has the disadvantage that is damn expensive on the computational side. When I went to the university, Ray Tracing was described as something that could be used only in offline productions and that a real-time implementation was impossible with state-of-the-art technology.

Well, some years after that statements of my teachers, it is possible to have real-time raytracing. Thanks to RTX, that is supported by Direct X’s Ray Tracing API DXR, that have been announced by Microsoft in the same days, we can have real-time photorealistic renderings. Unreal, together with ILMxLAB and Nvidia have produced a fantastic Star Wars demo to showcase how a scene rendered in real-time through ray-tracing is cool. You must see this video… and while you look at it, remember that everything is rendered in real time.

I have to warn you that this cool stuff can’t run on consumer hardware, yet: according to Road To VR, the demo runs on a Nvidia DGX Station running four Volta GPUs connected with NVLink. So, this awesomeness can’t enter inside our house now, but only in the near future.

Of course, AMD couldn’t stay there watching NVIDIA getting all the glory, so the same day AMD contacted Polygon to say that it is working on a similar technology, too.

Epic Siren demo

Talking about cool graphical stuff, Epic Games has showcased Siren, a marvelous demo developed in collaboration with CubicMotion, 3Lateral, Tencent, and Vicon. This is a demo of a realistic female character rendered by Unreal Engine at 60fps using an actress as a reference: the body motion of the girl is recorded thanks to Vicon motion capture system and her facial expressions are detected using the computer vision magic of CubicMotion and implemented using 3Lateral facial rigging features.

The demo is just… impressive, even if it is easy to spot that she is not a real person.

OpenXR is continuing its work

The Khronos group, the association including almost every player in the AR and VR sector to create a standard for AR and VR is continuing its job and has chosen the GDC to showcase publicly the current status of its hard work.

After one year and a half, we are finally close to a first provisional release of the standard and this is surely a good news. I admit that I’m not a huge fan of digging into standards, so I will just provide you a link to the various resources offered during the GDC, that you can find directly on the Khronos group website. In particular, you may find interesting this PDF where there are detailed some parts of the future implementations of OpenXR. This is the PDF that has been used for this presentation regarding OpenXR

If you don’t have the time to watch one hour of video, I suggest you read this little summary made by the fellow Redditor Redmercuryvendor. The part of this summary that, as a user, I found the most interesting, is this one, that would simplify a lot the launch of a VR application:

To be explicit (because I know the usual suspects will be out in force): this means if a game implements OpenXR, it should run on any runtime that implements OpenXR and can support the API features the game requires (i.e. if the game requires leg tracking and all you have is handheld controllers, expect to be SOL).

For example, if you were to run an OpenXR game bought through Oculus Home and you had a Vive, you would get the game, but none of the Oculus runtime functions (e.g. no ASW) and you would get the SteamVR environment rather than Core 2.0. Vice versa if you were to run an OpenXR game bought through Steam using a Rift; you would have it show up with Core 2.0 rather than the SteamVR environment. This is different to how ‘cross compatibility’ (official or otherwise) is currently implemented, where the API translation occurs at the other side of the runtime – i.e. one runtime’s device layer feeds into the application layer of another through a translator – so you end up with two runtimes stacked on top of each other.

One of the most curious things is that the OpenXR has started its work taking as a starting point an initial reference made by Oculus to create a common overlay between the Rift and the Gear VR. And the name XR has been taken because they wanted to create something that worked across AR and VR, so they put a V over an A and that seemed an X, so they used that letter. It’s actually funny 😀

Google AR apps

Google has showcased some interesting AR apps made using ARCore by its partners or by Google itself: eBay, The Walking Dead Our world, Just a Line, My Tamagotchi, Pottery Barn 360 Room View and TendAR. If you want to read a short description about these VR apps, I suggest you read the related article on Upload VR.

I found particularly interesting two apps:

1. The eBay app that exploits AR to let you understand what is the right package to buy when you have to ship a package to your customers. It is a great use of AR;
2. The Just a Line experiment, that is an AR app that lets you draw in AR a 3D line with your smartphone and then shoot a video with that creation. While very simple, I think that this app may give people some funny moments.

Games

A lot of games have been announced and/or showcased during the GDC:

• Owlchemy Labs has showcased Vacation Simulator, that is the sequel to Job Simulator. The game is more complex than Job Simulator and surely represents its evolution, but we can’t say that it is an original experience… it is just a great improvement of the previous one (read the review on Upload VR);
• Oculus has announced some great games for the Oculus Go, like Catan VR, Anshar Online and They Suspect Nothing. The last one is the title that intrigued me the most: in a world dominated by robots, you are a human but you have to behave exactly like a robot so they don’t discover your secret. Catan VR is a board game that can be played both by Rift and Go users and that may introduce that problem about different abilities of players wearing different headsets: of course Rift players are able to move faster and better and are so advantaged during the game (read more on Oculus website);
• Google and Squanch games have announced a humorous exclusive game for the Daydream platform: Dr. Splorchy Presents: Space Heroes (more on Road To VR);
• It has been showcased the current state of the stealth game Budget Cuts that a lot of people are waiting. The reviews of the hands-on are all great and the game is ready to be released in May (more on Road To VR);
• Creed: Rise To Glory is an interesting boxing game (more on Upload VR);
• Warhammer is going to get an AR game based on collectible cards (more on Upload VR);
• Pixel Ripped, a game that I tried years ago on my Oculus DK2 and that is developed by Brazilian dev Ana Ribeiro is close to launch. It is interesting because it lets you play with a Game Boy as if you were in real life… so for instance, if you play with it at school, you have to be careful of the teacher not watching you (more on Upload VR).

Other news

Some other interesting news bits:

• Massless has presented a stylus to be used inside VR: so, instead of using a VR controller, you can write in the air or on a surface just holding this special tracked pen. The review by Road To VR is quite positive, even if honestly I still don’t get in which scenarios this should be particularly useful… maybe in the graphic sector (more on Road To VR);
• Half-Live 2 VR will be compatible with the Hardlight suit, that is a $299 full body suit for virtual reality that is also able to give you haptic feedback. Playing Half Life this way with full immersion should be awesome (more on Upload VR);
• Disney Research has developed a solution called Cardinal that is able to transform a script of a movie in a storyboard and then into virtual reality animations thanks to some AI magic. This means that it is possible to have a taste of how the movie would look like without doing resource consuming tests: the filmmaker has just to give the script to the program and then preview the scenes inside VR, deciding what has to be changed to make them perfect. This is an awesome idea. Of course, the prototype has to be improved, but it is very promising (more on Variety);

  

• Unity has added support for Oculus Go and Lenovo Mirage Solo (more on Upload VR);
• Sketchfab has announced some APIs that are very similar to the ones of Google Blocks and that allow for fast searching and downloading of 3D models of the Sketchfab database without leaving the program you are in (more on Upload VR);
• The Snapdragon 845 VR reference design will have native support for the Vive Wave platform (more on Road To VR).

Wrap-ups

If you want the links to the wrap-ups made by Upload VR and Road To VR, here you are:

• Upload VR (link)
• Road To VR (Day 1, Day 2, Day 3)

And that’s all for today! I hope to have helped you in getting to know all the most important news from this GDC event 🙂

(Header image by GDC Conference)