NVIDIA talks about RTX 2080 graphics cards and virtual reality

When NVIDIA announced the RTX 2080 graphics cards, I was there. I watched Jensen Huang announcing the new graphics cards with amazing features like real-time raytracing and variable rate shading. Of course, after the announcement, a lot of questions started coming to my mind, especially regarding the use of these new graphics cards with regard to VR and AR.

So, I sent some questions to the company and after some time, I finally got some answers from NVIDIA China! Of course, they haven’t revealed to me any secrets, but it has been interesting reading some details anyway. Here you are the short interview I had with them, to which I added some personal notes.

BVH stands for Bounding Volume Hierarchy and basically means that the whole scene gets represented by a hierarchical tree of simple bounding figures that define what is the scene structure. At the leaves of this tree, there are the actual 3D elements of the scene. The fathers of these leaves are simple 3D objects (e.g. cubes) that enclose each one of them completely and then traversing the tree from the bottom to the top, you get always bigger boxes containing the elements that are below them.

Why do we need to represent the scene as a hierarchical set of boxes? Well, the answer is that computations using boxes are much easier than the ones regarding complex meshes. So, when doing ray tracing, when you cast a particular ray to see if it intersects with some elements in the scene (lights or other objects), you just verify the collision of the ray with the bounding boxes: if the ray doesn’t collide with a big bounding box, you are sure that it won’t collide with all the elements contained in that box and you spare a lot of computational time (you avoid checking the collision with all that branch of the tree). If it collides, instead, you start checking the collision of the ray with the children bounding boxes, applying the same fast-rejection process. In the end, you only do heavy calculations if the ray collides with the final leaf containing the true mesh of the object. Doing ray tracing means so traversing the BVH tree for each ray.

Basically, NVIDIA is hinting to the fact that the optimizations given by BVH and by fast BVH traversal have been the key to be able to offer real-time raytracing and that RT Cores have been the right hardware to accelerate that algorithm based on BVH. (In this post series from 2012, NVIDIA already talked about BVH and its implementations)

Microsoft has already released an SDK for ray-tracing, supporting NVIDIA algorithms. You can refer to this Microsoft blog post, with a final link to the SDK and tutorials on Github to have more info about it. Reading the post, I also got a confirmation on the statements above regarding BVH. In fact, DXR APIs introduce a new concept:

The acceleration structure is an object that represents a full 3D environment in a format optimal for traversal by the GPU. Represented as a two-level hierarchy, the structure affords both optimized ray traversal by the GPU, as well as efficient modification by the application for dynamic objects.

The Microsoft Windows 10 October update, that includes official RTX support, is already being rolled out by Microsoft… but for instance I’ve not received it yet (if you want to verify if you have already upgraded to it, check your Windows 10 version: if the bulid number is 1809, then you have the October update)

After the update will be released, games and applications will be able to exploit the DXR APIs and so offer raytracing rendering. That’s why the games supporting RTX have not been released yet: they are waiting that the operating system will support realtime ray tracing.

In the previous link, there is, in fact, a reference saying that both Unreal Engine and Unity will implement DXR APIs. According to online forums, UE4 should support DXR APIs in its 4.22 version. We are currently at version 4.20, so Unreal developers should wait a bit before playing with real-time raytracing (unless they want to use directly the DXR APIs from code). 4.22 is expected to be released around the end of the year, someone says. Regarding Unity, I haven’t been able to find any news on a possible release date.

So, some changes are required to transform your game into one that supports real-time raytracing. We are not talking about massive changes, but for sure on ones that will require some time to be implemented. Reading the above linked Microsoft DXR post, I found that apart from the BVH hierarchy that has to be constructed, there are new specific RTX shader types that can be used:

A set of new HLSL shader types including ray-generation, closest-hit, any-hit, and miss shaders. These specify what the DXR workload actually does computationally. When DispatchRays is called, the ray-generation shader runs. Using the new TraceRay intrinsic function in HLSL, the ray generation shader causes rays to be traced into the scene. Depending on where the ray goes in the scene, one of several hit or miss shaders may be invoked at the point of intersection. This allows a game to assign each object its own set of shaders and textures, resulting in a unique material.

This has been basically a non-answer. And I can understand… the framerate depends on what we are going to render. I guess that even in VR if we just render a dull cube in the center of the void, real-time raytracing can guarantee 90 FPS.

But I found anyway interesting that the answer has not been a complete no: it seems that the company is hinting that thanks to DLSS (Deep Learning Super Sampling, a new innovative technology for anti-aliasing), Variable Rate Shading (the tecnology that lets you decide which parts of the frame to render at better quality and which ones at lower quality, and that, if used with eye tracking hardware, is able to offer you foveated rendering) and Texture Space Shading (that can help a lot in virtual reality because it can make the graphics card render the right eye view by recycling a lot of pixels already shaded for the left eye view), it may be possible that some VR applications can work with real-time raytracing. What is not clear is how complex a VR experience can be to work in realtime with raytracing. I hope that when the DXR support will be completely rolled out, some people will start making experiments in this sense.

For sure, in the future eye tracking can help really a lot in decreasing render times and will be a game changer in this sense.

So, even if we don’t consider real-time raytracing, these new graphics cards are much better than the previous ones.

And that’s it for this interesting interview… and I really want to thank a lot NVIDIA for its kind answers.

These new graphics cards seem very interesting and I hope to get one of them one day! The new technologies they offer, like real-time raytracing, are really new and we will still need time to understand how to exploit them properly in virtual reality… at this time they are not implemented in the most popular game engines, so they are really at the cutting edge. But I really hope that in the future they will help a lot in having more credible virtual reality scenes, that we’ll offer us a greater sense of immersion.

And while we wait for that moment, why don’t you subscribe to my newsletter to receive in your inbox interviews like this one? 😉

(Header image by NVIDIA)