All you need to know on Oculus Connect 6: the most important news in one single article!

This week has been the week of the Oculus Connect 6, the biggest yearly event by Oculus where the VR “company” (actually, now it is a division of Facebook) announces all its most important news, both on hardware and software sides. And as any year, Oculus has really amazed us: for sure you have already read my short recap published after the first keynote of OC6 where I told you about amazing stuff like Oculus Link, Facebook Horizon and Hands Tracking. But now, in this article, it is time to go deeper.

I read (and watched) lots of materials about the Oculus Connect 6, and it is time for me to merge all this info into a super-big-fat-article with all you need to know on the Oculus Connect 6, so that all of you in the communities can stay informed on what has happened without reading more than 60 articles as I did!

Let’s start!

Oculus Link

Oculus Link has maybe been the most important news announced at OC6, and the one that has also created most debates in online communities.

It is a software solution that lets you connect your Oculus Quest to the PC so that you can “transform” your Oculus Quest into a PC VR headset. To use it, you must just connect your Quest to a PC by plugging a cable into the USB C port of the Quest and into an USB A (or C) port of your PC.

So, basically, you connect with a USB cable your Quest to your PC, and you can play PC VR games, on the Oculus Store, Steam or Viveport. You can even access to Oculus Home and Dash. For the PC, the connected headset is more or less like a Rift S.

That’s a piece of fantastic news for all the Quest owners (like me) because this means that we have finally a super-headset that can be both standalone and tethered. Oculus has always told that the Quest is like the Nintendo Switch of VR, and with this new feature it may actually become close to it: you can play a PC VR game on Quest using Oculus Link, then when you have to go away, you can save your game progress data on the cloud thanks to Oculus Cloud Storage 2 (if the dev has enabled it), then detach the Quest, and play the same game (supposing it is available on both platforms) from where you left it in full portability on Quest. This is exactly what you do with the Switch (even if with far more “if”s). Amazing.

Oculus Link will be available in beta in November 2019.

How does it work?

Oculus hasn’t given many details, but basically, it is a streaming solution, exactly like ALVR or Riftcat VRidge, only that it works only via cable, and not also via Wi-fi.

So, the computer renders the frames, then encodes and compresses them in a “fixed foveated way”: the central area of the frame is not much compressed, while the peripheral regions of the frame are much more compressed (that is, the peripheral regions are compressed with lossy compression, so more data is lost from them, while the central area appears in great quality). The frames are then sent via USB to the headset. The headset decodes the image and shows it on the screen, not before having applied some timewarp/spacewarp magic to update it to the new actual pose of the head of the player. According to Ars Technica, the frames don’t get sent as a whole, but in little horizonal slices there are continuously streamed, so that to reduce a lot the perceived latency.

In the meantime, the headset sends the pose of the headset and the controllers to the PC, so that the PC can render the frames and execute the logic of the game for the next frame.

As you can see, there is a lot of decoding and compression. This is needed because the device is connected via USB, that has not a bandwidth as big as the one of DisplayPort, so the images have to be compressed to be sent to the Quest.

According to John Carmack, when the Quest is connected to a PC, theoretically its displays can run at 90FPS, but this mode can’t be enabled because the device has not been certified at FCC for running at that speed. We all hope that someone will release an unofficial crack for this (come on, John, put it on the deep web yourself 😉 ).

Does it work?

According to the reviews, very very well. All journalists report that thanks to the Oculus Link, the Quest becomes very similar to the Rift S. Of course, there are some little issues, like for instance:

• There is a little latency for positional tracking and tracking of controllers (even if it is hardly noticeable);
• Sometimes you can spot some artifacts, especially if you move very fast;
• The images do not appear as crisp as on the Rift S, because of the compression stuff.

Notwithstanding these issues, Link was praised by most as a great solution: it has been showcased with Asgard’s Wrath, a very good-looking and computational-heavy game and it worked surprisingly well. It is much better than ALVR and other similar solutions (of course, because Oculus can operate at a deeper level on its hardware and runtime).

The cable

Oculus says that Link can theoretically work with every USB port and cable that respect some requirements. What are these requirements is not clear, because Oculus has not shared them yet.

But Oculus will share after November an official cable, for the cost of $79 (according to PC Mag), that will be ideal to implement this solution. The official cable will be:

• 5 meters long, exactly as the cable of the Rift S, to guarantee optimal room-scale movement;
• Made with optical fiber, so to guarantee the best data transmission speed between the PC and the headset;
• Made so that to not ruin much the balancing of the headset;
• Manufactured so that to make sure that the headset will charge while using it. This way, it won’t turn off just after two hours of play time.

Oculus stresses that it is not fundamental to buy its cable, but it seems that finding other unofficial cables at the moment with that specs is hard (but I hope that some Chinese companies will start producing them soon).

Why not wireless?

Why hasn’t Oculus gone for the jackpot releasing a fully wireless streaming solution? HTC has actually released that for the Vive Focus Plus…

Well, according to the god John Carmack, it is something about which the company has discussed a lot and has also experimented a lot. In the end, it seems that the cable seemed the best solution to offer better visual quality and also the simultaneous charging of the headset. Carmack has also said that there has been the idea of selling a Wi-fi dongle to attach to the USB of the PC so that to offer a dedicated high-quality Wi-fi for the Quest, but in the end, the idea of the cable has been the one that has won. But Oculus is still experimenting on this, so I guess that sooner or later, with this Quest or the next, Wi-fi streaming will become a thing.

Regarding cloud streaming in 5G, instead, Carmack said that it is too early to think about that.

Rift S vs Quest

This decision by Oculus has created a lot of drama in Oculus communities. If the Quest can now connect to the PC and become something like a Rift S… what is the purpose of the Rift S? The Quest has also better resolution, better colors (thanks to the OLED display) and IPD adjustment!

Some people argue that the Rift S has still its place in the world, considering that:

• It is more comfortable than Quest;
• It has a better fill-factor;
• It operates at a slightly higher frequency;
• It doesn’t show compressed frames;
• It has a bigger controllers’ tracking FOV and this is fundamental for games like Echo VR;
• Developers of PC games can decide to exclude to Link users from playing the game (but I wonder what developer could be so stupid to take the decision of having fewer users).

And that’s true. But if for $399 you can buy a headset that can work connected to a PC, and for $399 you can buy a headset that can work as a standalone, and that is also a decent PC VR headset, what would you buy? I think that most people would buy the second one…

Consider that the most sold headset in the market is PSVR, that is maybe the most mediocre of all: this shows that if you provide an affordable product with great content, it sells well. Specs count up to a point. So, for most people, Link+Quest experience will be good enough as a PC VR experience. And the other ones, well, most probably are people that want a high-quality experience and so they will buy a high-quality device like the Valve Index, Vive Cosmos or HP Reverb, and not a Rift S.

I foresee tough times ahead for the Rift S… I think that it will lose at least 50% of its sales after this decision.

https://twitter.com/Damo90001/status/1177321808615227392

And there is even more: during the whole conference, Oculus has not mentioned updates for the Rift S and has not talked about the headset at all, if not to announce the new AAA PC VR games, that will be anyway playable also via Link. This shows how the Rift S is not the main interest of Oculus, and probably all PC VR is not the main interest of Oculus anymore…The Quest is selling way better, and companies like Facebook must chase money, not the willing of us VR enthusiasts.

People that have just bought the Rift S went crazy of course, and I agree with them. Releasing the Rift S and “killing” it after 4 months has been a very weird idea.

But actually, it may still have a minimum of sense: VR is a new field and it is all about experimentation. Jason Rubin has told TESTED that they want to see what customers will choose in the next months. From the stats, they will see how many of the PC VR users will use a Rift S and how many will use a Quest+Link and then they will decide how to go on. I think that if it will turn out that most of them will be Quest users, Oculus will say goodbye to PC VR headsets forever.

Hand tracking on Quest

Lovers of natural interfaces, rejoice! Oculus, thanks to its genius engineers like John Carmack, has been able to implement hands tracking on Quest! Starting from early next year (2020), in beta for consumers and developers, it will be possible to use your bare hands within the Oculus Quest. It is very interesting to notice that the hands tracking won’t be only limited to custom applications, but it will work also in the system UI of Quest! You will be able to start a game, or choose a movie to watch in VR without having to use a controller anymore!

Hands tracking will guarantee 25 tracked points for each hand. For sure Oculus is exploiting all its AI expertise and the acquisitions made in all these years (e.g. Nimble) to offer this advanced hands tracking solution on a mobile headset. It uses a new method of deep learning to reconstruct the pose of the hands of the user. It’s impressive: consider that on the Vive Focus Plus, you can only have gesture tracking and not full fingers tracking, because of the mobile platform limitations. Oculus has managed to offer what seemed impossible. And this also thanks to a DSP by Qualcomm that does lots of the heavyweight computational CV work directly in hardware, without impacting the performances of the software running on the VR headset. Oculus claims that using hands tracking, the battery drains only 7 minutes faster in a full charge cycle: that’s a negligible amount of time.

Hands tracking won’t replace controllers

Hands tracking is amazing because it can give you a more natural way of interacting with VR content. You can use your bare hands to interact with the VR world, exactly as you do every day in real life.

It has many advantages, like for instance the fact that you can finally watch a 360 video on a Quest without the need of grabbing every time a controller; you can demo VR stuff without giving every time controllers to newbies that don’t know how to use them; you can spare a lot of batteries on controller you won’t use; and then yeah, a friend told me that having the hands-free is also better to watch in VR “a certain category of videos” (IFYKWIM).

But notwithstanding all these pro sides, Oculus has clearly stated that hands tracking won’t replace Quest controllers. Touch controllers are still necessary to have an optimal VR experience because they let you feel something in your hands, they give you haptic feedback, they make you press buttons and such. So, hand tracking is just something more, a nice addition, not a substitution.

For now, the choice between the use of controllers or hands is exclusive, you can’t have both working together or having a hybrid mode: either you use two controllers or two hands. But Oculus is investigating more choices for the future.

Reviews

The hands-on impressions on the hand-tracking on Quest have been pretty positive. The system correctly tracks all fingers and this is very impressive. It also lets the developer specify in the application a custom 3D model to use for the hands of the user, that can so have human hands, monster hands, robot hands, etc…, something that is very useful in games. You can also use it to detect some gestures, like for instance pinching.

Anyway, there is still a lot to do: the system can’t handle occlusions at all, and if the two hands cross each other, the system loses the tracking. The same if one hand is in front of the other. There is a bit of latency (80ms) and it can be perceived. The tracking FOV is reported to be quite good, but not as good as the one of the controllers, so you can use your hands only more or less in your vision FOV. Then there are the problems that are inherent to all hands-tracking solutions like Leap Motion: no haptic feedback, virtual hands that trespass objects they are interacting with, and such.

It is also interesting to notice that journalists reported that if the hand tracking is not good enough on a particular hand, the system makes it to disappear, to avoid you seeing hands in weird poses.

The good news is that Oculus is continuously improving it: in this video reported by journalist David Heaney, you can see some hands tracking performances that Oculus has obtained in the lab, including hands interacting the one with the other and external objects held in the hands.

No hands tracking on Rift S?

In my predictions, I predicted hands tracking for Rift S, because it seemed a more natural destination platform. Actually, Oculus has only announced hands tracking for the Quest, because it is their device for consumers. After some protest by the PCVR community, it has answered to Upload that it will “look at tackling the technical challenges” of porting hands tracking to Rift, but it is not clear what kind of technical difficulties can be in a device that is attached to a beefy PC. The only one seems to be that the Quest has a special DSP on the Snapdragon SoC that takes care of all the computations in hardware, while regular CPUs don’t have it.

Anyway, also from the interview to TESTED, it seems that it is more a business decision, since they wanted to release something for consumers and Quest is the device for consumers. In the same interview, a Facebook representative opened the door to a possible future use by developers on PC.

Passthrough+ on Quest

Talking about all the new features of the Quest, you can add Passthrough+ to the list. Passthrough+ was a special feature of the Rift S, that now is being transferred to its smartest brother, the Quest, and it consists in an optimized screenthrough vision of the world around you.

When you go outside your boundaries on Quest, you can clearly see your surroundings on the screen of the Quest, but these visuals are terribly distorted and also the framerate is not optimal. From next week this is going to change: thanks to Passthrough+, finally you will have undistorted fast visuals of the world around you. And later on, you will also be able to tap twice on the system menu of your two controllers to activate passthrough on demand, maybe to have a sip of water while you play Beat Saber without removing the headset. All of this is fantastic.

This passthrough won’t be open to developers at the moment, though. So you won’t be able to develop AR apps for the Quest, yet. I imagine Oculus enabling this in the future, but not now.

At the moment, if you want to do some crazy AR/MR stuff on a standalone, your only option seems the one of buying the Vive Focus Plus. I’m developing a MR game with it, and I can assure you that doing MR on a standalone is pretty cool, so I hope that Oculus will release a MR SDK in 2020.

Oculus Quest running Go apps

Oculus Quest will have Oculus Go emulation. And Oculus is selecting 50+ Oculus Go apps, among the most successful and high-quality ones that regard gaming, and will port them to the Quest store. Before the end of the year, people can decide to use those apps on the Quest and have them free if they already bought them for Go.

Oculus claims that not all Go apps can become compatible with the Quest: only the ones that are made with the latest APIs are eligible for a possible porting.

Has the Oculus Go a future?

We have seen how Quest is cannibalizing the Rift S. And this piece of news seems to show that it is doing the same with Oculus Go. The Quest seems the only device on which Oculus is investing, because it is the one selling the most.

I’ve already reported on this blog how Xiaomi has disbanded the team working on the Oculus Go and this seems to point that there won’t be a Go 2. Oculus not announcing new features for Go as well during OC6 and barely citing it seem to hint that it has no future.

After a discussion with some journalists (like David Heaney) out there, I am still convinced that the Go as form factor has died, but its spirit may survive in a second edition that looks like a low-tier Quest. A device that is very cheap and it just works for media consumption: something like a low-end Quest but without controllers, that works through hands tracking. It would be a good entry-point for VR. This is possible, even if I personally think that it sounds more like a sibling of the Quest than a successor of the Go.

Forward compatibility for Quest

Virtual Reality is in a moment where there is a lot of confusion, and things are evolving so fast that it is difficult to keep pace with them. Especially for us developers, having to provide compatibility with all the devices on the market is a big nuisance.

So it is a fantastic news that Zuck has announced that Facebook will invest a lot in creating a full-fledged platform for Oculus Quest and that it will make sure that there will be forward compatibility for Oculus Quest content. This means that if I develop a game for Quest today, it is guaranteed that it will work also on the Quest 2. This is great, it gives us devs some solid foundation onto which develop for the Oculus ecosystem.

Facebook Horizon

We were all expecting Facebook announcing a metaverse of its and in fact, it has arrived: it is called Facebook Horizon, and it really looks like Rec Room running on Facebook infrastructure.

Horizon will feature a big square (the Skyway Hub), where people will be able to enter and meet other people, making new friends or having fun with people they already know. From the hub, they will be able to jump into new worlds, that will feature different kind of activities: playing games, chatting, exploring, painting, etc… It is like a social world made of connected worlds., a bit like VRChat

The graphical style of this social VR world is very cartoonish and very pleasant. Personally, I like it a lot, it looks very informal and funny. Actually, the style that you see in the trailers may not be the final one: according to John Carmack, there is still some work being done on the avateering system on Horizon. And it seems that at the beginning it won’t support hands tracking… a weird thing, since it could improve the sense of realism a lot. What TESTED points out as cool, instead, is that the system tries to understand from your voice and your actions (e.g. what gestures you are making), what are your current facial expressions. So there is emulation of eyes’ position and also the mouth moves following your voice… and this emulation works quite well and it is very cool.

Facebook encourages people to create new worlds inside Horizon using the World Creation tools, that let people create easily new experiences for Horizon from inside virtual reality in an easy way and without knowing how to code (there is a visual scripting system). TESTED people report that Facebook wants people to develop stuff for Horizon from inside Horizon, and not by importing stuff made with other tools. But they claim having also seen a Unity experience running from it (maybe it will be possible to develop for Horizon with Unity, as it is possible with VRChat), so developers may still be able to create more complex stuff. In the demos, Facebook has showcased a plane game call Wing Strikers. Tested also reports that in Horizon it is possible to make some puppettering, that is moving elements following your body gestures.

Facebook claims that Horizon will be made to guarantee inclusion and the wellbeing of all its inhabitants. To do that, heavy customization of avatars will be possible. It will have moderation tools, and also real people dedicated to being there and support new users while blocking trolls.

Horizon will be available somewhere next year in closed beta. Interested people can register at this website.

We’ll see if finally, Facebook will be able to release a really compelling social VR experience. It is pretty embarrassing for such a big social company not being able to create a good VR social network. The good news for now is that finally, it is unifying all its social offerings in only one platform. And as an effect of this decision, it will close Oculus Rooms and Facebook Spaces on October, 25th.

Oculus Store sales numbers

Oculus claims that from its opening, the Oculus Store has made $100M in apps sales. The good news is that Oculus Quest, in just 4 months, is already counting as 20% of that big amount made in 4 years (the Oculus Store opened for Gear VR in 2015, as Road To VR remembers us).

These numbers are cool, but to put it in context, Fortnite alone has made $2.8B in 2018 (according to SuperData). So, 100M is good, but far from the numbers big companies are interested into.

And Oculus, according to previous claims, is investing $250M every year in funding content. You can do the math alone: in one year, it is investing in content the 250% of the money that it has earned in 4 years. Sum to it the fact that it is almost sure that it is selling the Quest undercost, and you get why it seems the only company that cares about VR for consumers: because it is one of the few companies that can burn hundreds of million dollars every year without going bankrupt.

Announced Games

We all bet that Oculus would have talked about content in this OC6, and we were right!

Medal of honor: Above and Beyond

First of all, it has finally announced the war game that it is being developed by Respawn Entertainment: it is called “Medal of honor: Above and Beyond” and it is a realistic war game set in the WWII times. This is the official description:

You’ll play as an Allied agent of the Office of Strategic Services (OSS), tasked with infiltrating, outgunning, and outsmarting the Nazi war machine. And as you complete your tour of duty across Europe, you’ll collaborate with the French Resistance, working to sabotage Nazi operations from behind enemy lines.

The game recovers the “Medal of Honor” franchise, that according to many, created a new genre that has led to many successful titles like Call of Duty. Respawn has worked a lot in providing realism, in conveying into this experience what really means being in war. Anyway, as also Valve has pointed out in a recent talk, they are not aiming at perfect realism, but at something that is realistic enough, but that also guarantees that the player has fun and doesn’t get frustrated. So, for instance, the weapons return to the inventory in the belt without the need of putting them exactly at their place, for the convenience of the player. Anyway, it is cool that Respawn has also involved Veterans to know how the war was so that it was possible to recreate it with high fidelity.

The game will feature 50 actions segments, for a total playtime of more than 10 hours. According to Dean Takahashi of Venture Beat, the development has taken 2-and-half years, and it involved 180 people working on it. You make the calculation and see how much money it is worth.

The game will feature a single-player narrative campaign, and also a multiplayer mode. There will also be special narrative content (photos, videos, etc…) to show you how the war really was.

Some selected journalists have tried it (read Upload’s review here) and told that this game really feels like an AAA title. The quality is very high, and there are so many things you can do (be on a plane, on a train, boat, etc…). There are some glitches at the moment (like the AI characters sometimes getting crazy), but they will for sure be solved before the launch that will happen somewhere in 2020. Of course it will be compatible only with Rift (and Quest+Link).

Stormland and Asgard’s Wrath release dates

Sci-fi adventure Stormland and fighting game Asgard’s Wrath have finally a release date: the first one will launch on November, 14th and the second one on October, 10th. Pre-orders for both titles are now open for €40.

It is funny to discover that Asgard’s Wrath features a whopping install dimension of 121GB, because of the dimensions of the high-quality textures. The world feels so good and detailed there. I hope it will be sold in bundle with an external hard drive 😀

Vader Immortal episode 2

Oculus has surpised us all announcing the immediate release of Vader Immortal episode 2! In this episode, you will learn how to use the force from the Sith Lord himself, and so you can both fight with lightsabers and move objects in the air with the force! That’s overly cool.

The first reviews are good, but I’ve also read non-enthusiastic ones. According to Upload VR, the use of the force is not fully exploited, and sometimes it seems that using lightsabers would be better than using the force. Anyway, everyone agrees that the fighting Dojo of this second episode is amazing since you can kill your enemies with creativity using both the force and the sabers. So you can for instance throw a lightsaber towards enemies, slicing them and then attract your sword back. Wowow!

The game is available for Quest and Rift for 10$.

Beat Saber

People from Beat Games took the stage to announce that the 360 mode of Beat Saber will finally be available on Quest starting from December of this year. It is not clear if it will arrive to headsets as well, but I doubt it, since Beat Games has some kind of exclusivity with Quest.

For now, people can enjoy a new DLC, the “Panic! At the disco” from next week on all platforms.

Other titles

Many other titles have been teased or announced for Quest:

• A game in the Boneworks universe (coming to Quest in 2020);
• A Fisherman’s Tale;
• VRWERX’s Paranormal Activity: The Lost Soul;
• Space Channel 5 VR;
• Five Nights At Freddy’s VR;
• The Room VR: A Dark Matter (hands-on);
• Echo VR. This was already confirmed for Quest, but finally journalists have been able to try it. The game feels great, but the graphics is greatly simplified, and since the Quest can’t track the controllers when they are behind you, some mechanics of the game don’t work well anymore;
• Solaris: Offworld Combat (coming for both Rift and Quest);
• Last Labyrinth;
• Pistol Whip (from Cloudhead Games)

Futhermore, during OC6, former Owlchemy Studio Director Cy Wise and CEO Alex Schwartz announced the creation of a new indie game studio named “absurd:joy“ that has immediately got $4.4M of investments. I wonder what they have shown to investors to have all that money! :O

Future perspectives

Jason Rubin has told TESTED that Oculus is already funding new high-quality titles that are not disclosed yet, and these future titles will be released for Quest and/or Rift S.

Developer tools

Facebook has also announced new tools for the developers creating content for the Oculus platform, and they are pretty cool.

https://twitter.com/russellholly/status/1176914084559613953

Oculus has already done a good job in summarizing them, so let me just do some copy/paste

RenderDoc Updates
We’ve contributed to RenderDoc, the open-source frame debugger, so that it supports all of our extensions and uses less RAM. That means it will run more smoothly on Quest. We are also improving RenderDoc to report more performance data.

OVRMetricsTool
Later this year, we’ll be enhancing our OVRMetricsTool, so you can
view more metrics in the headset while your application is running—or view on a computer and save the information for later.

Vulkan News
We’ve added support for Vulkan Multiview and Fixed Foveated Rendering in our platform and in Unreal Engine, which can boost performance significantly. Later this year, we’ll expand our Vulkan support on Quest to include Unity and Vulkan validation layers for easier debugging.

Unity GPU
We recently enabled Unity’s GPU Profiler on Quest and Go. We’re also working with Qualcomm to develop a low-level GPU profiling API. This will give you more detailed information to increase accuracy in pinpointing and resolving performance issues.

New Purchase Funnel Analytics, Coming This Fall
We’re bringing you new metrics for Quest in the Developer Analytics tool, to help you understand how your app is performing in the store. You’ll be able to explore the dashboard to view conversions, as well as metrics related to browsing, searching, and engagement with your product detail page. Now you can measure the quality and impact of your art and product pitch, so you can test, refine, and improve.

Mixed Reality Capture for Quest
We recently shipped the ability for developers to make mixed-reality videos, with support in our Unity and Unreal Integrations. Today, we announced that we’ll be adding native support later this fall, along with a tool to make it just as easy for content creators to produce mixed-reality videos.

Oculus Link: Remote Rendering for Developers
When Oculus Link software ships, developers will be able to iterate more quickly by running on a PC, while streaming the output to Quest. Just hit the play button for the Unity or Unreal Oculus plugin, and you’ll be able to preview right on the device—no need to compile and transfer a new APK first. It’s development at PC speed, for a mobile device.

I am mostly excited by the last two news for which I want to write a little comment.

Developing for the Quest thanks to the Link is an amazing idea. Developing for a standalone headset is a PITA because every time you want to test your application, you must deploy it on the device, and every deploy requires minutes (not to mention all the nuisances that adb creates) and it is so a big waste of time. Having finally the possibility of testing a Quest application on the fly thanks to Link is a big time savior.

The native Mixed Reality capture feature is very interesting. Basically, if I’m playing with the Quest, someone can point an iPad towards me and see my real silhouette playing in the virtual environment I am in, as in all MR videos that you see on the web, like the ones of people playing Beat Saber. It is possible to create mixed reality videos on the fly, without the need of dedicated hardware or green screens, just a tablet. This has huge implications for streaming (Twitch or Youtube) and for marketing.

The quality of the resulting stream is not the same of the one that you may obtain using a green screen, but it is good enough to be used inside exhibitions, or for the marketing of little indie studios.

Oculus for Business

Oculus has no intention of leaving all the enterprise sector to HTC and it is so finally launching all its new offering for the enterprise, dubbed Oculus For Business, in November. Until today, it was available in closed beta, and from November on, it will be open to anyone in the enterprise market. The two available headsets will be the same of the beta, Quest for $1000 and Go for $600, but there are a lot of new platform tools.

First of all, companies will finally be able to buy big batches of headsets and manage of all of them from a simple app or web portal: it will be possible to update them, to install apps on them remotely, launch applications, etc… Finally, companies will have the possibility of controlling all their headsets from a single access point.

Also inside the headset, the view will be different: there will be a kiosk mode or in alternative a simplified interface that won’t show Oculus Store, but a menu from which it will be possible to start only the experiences approved by the companies.

And then there is hands tracking, that is a great addition for companies as well: training experiences can become more natural with the use of the hands and hand tracking will let also companies avoid the use of controllers, that sometimes create problems.

Thanks to the experience already gained through Workplace, Oculus will guarantee strict data safety protocols, so companies will know that all the secret data handled by Oculus will be safe.

The importance of VR in training

As a consultant, I find that one of the most important moments of the whole keynote has been when Imperial College London took the stage to talk about the efficacy of VR training. Johnson&Johnson has created a VR training experience for surgeons, and the Imperial College had decided to make an experiment using it: it made a part of the students using it and the other one not using it.

The results are very interesting: 83% of the students that used VR for training were able to perform a first surgery in the lab with just little guidance. Do you know what is the percentage of people achieving the same results among students using traditional methods? Well, ZERO.

Some people on my social media contested theses numbers, but in any case I think that they clearly show the huge potential of VR in training.

FacebookDEP

For sure you remember Facebook announcing a new innovative 6DOF camera in collaboration with RED: the very expensive Manifold camera. Well, at OC6, Facebook has deciding to go further, creating a full solution to help in creating 6DOF videos, that is videos in which the user can walk in.

These efforts has led to the creation of FacebookDEP, “an open-source software stack that supports this creative endeavor—one that reconstructs depth and supports real-time playback in VR headsets like Oculus Quest and Rift S”. Translated: FacebookDEP is an opensource solution (the code is available on GitHub) that helps you in creating 6 DOF videos from the recording of various cameras, and also help you in playing and streaming these videos in real-time on 6DOF headsets like Rift S and Quest.

And if you are curious: DEP stands for “Depth Estimation Pipeline”.

Oculus social platform, built by Facebook

Oculus is going to have some new social features added, and all of these will be powered by Facebook and will of course need you to login using your Facebook account.

These new features include “Chats”, that will let you chat with your VR friends while you are in VR or even outside. “Events”, that will let you organize meetings in VR, or multiplayer matches with other players. And you will be able to post into Facebook Groups from within VR. Of course, logging in into Facebook, you will also have Facebook recommending you people that you may know, the VR applications that may be the most suitable for you and the VR groups that you could like to join.

Facebook claims that it wants to integrate always more of Facebook’s infrastructure inside Oculus, so that to offer a better service in connecting people in VR. This is to fulfill Facebook’s purpose of “defying distance”.

This sounds nice, but hey, it is always facebook. Quoting its statement on the topic:

In order to provide these features, we’ll use information like your friend list, how often you jump into VR, and which apps you use to provide better VR-related recommendations on Facebook, including relevant Facebook Groups you may want to join, Oculus Events you might like to attend, or ads.

Basically: we will harvest more data about you, merge your Oculus data with the other data gathered while you use Facebook and we will use it to provide you relevant ads. This is exactly what I DO NOT WANT from Oculus. The dystopia is slowly becoming true.

Rich Presence and Destinations

This is one new social feature that I admit I have not understood completely, but I will try my best to explain it to you 😀

Facebook next month will provide developers of VR experiences the possibility to define special places in their experiences, called Destinations. When players enter these places, they can activate Rich Presence, and show everyone they want that they are in that place.

When you will open your Oculus app, you will see if some of your friends have triggered the Rich Presence in one of the various destinations, and you can decide joining them immediately. I envision this being particularly interesting for social VR environments. For instance, you see that a friend of yours is playing paintball in RecRoom, you press a button, and you reach him immediately. That’s pretty interesting.

In the future, developers will also be able to broadcast Destinations. But what is cool is that later on, it will be possible to share the Rich Presence not only to VR friends, but also on Facebook and Messenger. And people from tablets and PCs will be able to somehow join their VR friends and have fun with them. It is unclear to me how it does exactly work, though.

Oculus TV

Oculus TV app will be empowered so that to become the place to go to watch storytelling content in VR: it will include content from popular services like Amazon Prime Video and Fandango NOW (that has just signed a partnership to bring more than 90,000 movies and shows to Oculus TV), but also indie VR content like Travelling While Black. As far as I have understood, it will let you so watch all kind of media: photos, standard videos, 180 VR videos and 360 videos. And of course, this will be a social experience as well, so you will be able to meet with friends and have fun streaming your favorite TV series togheter in VR on a 180-inches screen.

Oculus Media Studio

Content creators will be able to use a new portal called Oculus Media Studio to distribute their VR content. Oculus Media Studio will provide a simple interface to upload content to the Oculus Store and Oculus TV, and to manage it later on. It will also help with spreading the word about your titles on Facebook with tools like easy 2D trailer creation and sharing on Facebook. It will also be possible to have a dedicated URL to a landing page with a special link that will let people open directly your experience on their Oculus devices and enjoy it in VR.

In Oculus Media Studio there are of course a long series of tools that will let content creators analyze the performances of their titles (how many people are watching them, for how much time, etc…).

It seems a good solution to let content creators manage their storytelling experiences. Baobab Studios is already using it, and it is very satisfied by it.

Gear VR is dead, long live to Quest

John Carmack took the stage and made an eulogy of the Gear VR, certifying its death. You don’t say, John.

He added that notwithstanding the many efforts on the platform, Gear VR was the device with the leatest retention of all: customers didn’t like it. One of the main reasons was the friction of turning it on, that required inserting your phone into a shell while inserting a USB plug in it: a very annoying process, that was necessary for various reasons, like the stabilization of the IMU of Samsung’s phone (Gear VR shell contained another IMU). Carmack admitted that Daydream had a better setup procedure, where you just put the phone into the shell. Anyway, Daydream has died as well.

Carmack told that at a certain moment there has been this idea of offering a lightweight headset connected to a phone, but then the all-in-one design of the Go has been the one that has won.

Carmack confirmed that Quest is the device with higher retention rate, and Rift S is performing better on this side than CV1.

According to Zuck, Facebook is selling Quests as soon as they manufacture it, and this shows why the company is betting so much on this device.

Facebook acquires Ctrl+Labs

Just before the beginning of OC6, Facebook has officially stated that it is concluding the acquisition of Ctrl+Labs, a startup that is working on electro-myography BCI. Basically, it creates a bracelet (similar to the Myo armband, from which it has acquired the patents) that aims at intercepting the electrical signals that the brain sends to the muscles of the hands to move them so that to know the pose of the hands without tracking the fingers. What is fascinating is that it is also possible to just send the signal to the hand without actually moving it. So, if you just think about moving your index finger, the band may detect this electrical pulse and make you move your virtual index finger. This can create in the future some kind of fantastic input system for AR/VR headsets, where you have full use of your hands, but without the need of becoming tired moving them.

Anyway, this is a long term dream, and won’t happen in the short term.

Abrash predictions

This year, Michael Abrash took the stage but he didn’t do predictions in his classical style. He just said that he was wrong, and that next-gen VR will come later than he thought, and he now doesn’t know when it will happen. From his words, it seemed that truly innovative next-gen VR will probably happen in 4-5 years.

Then he started talking about all the new cool stuff that Facebook is working on in its R&D center, saying that “the best way to predict the next-gen VR is creating it”.

Actually, this seems to me a political shift. First of all, saying this way, Facebook puts itself at the forefront of technological innovation, basically saying that the future of VR is what it is working on. Then, this way Abrash doesn’t give explicit timelines on possible next-gen devices, especially now that it is not clear if there will ever be a Rift 2, or if it will be just a Quest 2. And not talking about next-gen devices coming soon, people are more confident in buying the tools of this gen (like the Quest), as per the Osborne effect. I personally think that Abrash could have made predictions like in the last two years, but he didn’t want to.

We were all expecting a new demo of the Half Dome Rift prototype, but it was not on stage. Abrash has talked about it, but Facebook has not made anyone try anything. Then he said that while the future is bright and there is a lot of work happening in Facebook Reality Labs, the present is devices like the Quest. My suspect is that Oculus is waiting to decide if going all-in on mobile or not before making people try a new PC VR prototype.

Then Abrash told that these are what we will refer as “the good old days” of VR and he is proud of working on it now, as it was happy of working on PC games in the early days.

And his vision is that XR will be a great tool to revolutionize offices. It will be our new way of working, with a mixed reality office that can blend real objects and virtual ones, telepresence in all places of the world, and big virtual displays in front of our eyes.

Half Dome 2 and 3

Abrash showed some videos and photos of the varifocal PC VR prototype known as “Half Dome”. After one year and a half from its reveal at F8 2018, the headset has already two new versions: Half Dome 2 and 3.

Half Dome 2 is lighter than Half Dome 1 by 200 grams, and it is more comfortable. The varifocal system is better and more silent, because it “relies on voice coil actuators and flexure hinge arrays. These reduce friction, minimize noise, and maximize the lifetime of the headset’s moving mechanical parts”. But all of this required a compromise on the original FOV, that is now only 20% better than Quest (so around 130°).

Half Dome 3 is even better than Half Dome 2 and looks like cool sci-fi goggles. The varifocal system is now not mechanical anymore. Using mechanical parts is a real pain because moving a lot, mechanical parts always break. It is always better to go electronic for a hardware, and Half Dome 3 works only via electronic stuff.

There is no more a moving display to provide the change of focus, but a stack of liquid lenses, composed of two flat optical elements: polarization-dependent lenses (PDLs) and switchable half-wave plates. Let me quote Facebook to explain you how all of this works.

PDLs are special because their focal length changes based on their polarization state. By changing the voltage applied to the switchable plates, we can toggle between the two focal lengths. This could make for a great set of digital bifocals, but it doesn’t necessarily make for comfortable VR. By stacking a series of PDLs and switchable half-wave plates on top of each other, we’re able to achieve smooth varifocal that lets you comfortably and seamlessly adjust your focus in the headset.

Every added lens multiplies the number of available focal planes of the headsets by 2. In the video shown by Abrash, the headset features a stack of 6 lenses, so the total number of focal planes is 64. By changing the voltage on each lens in the stack, it is possible to make the user focus on a particular focal plane from the ones available, as you can see from this GIF:

On the right, there is depicted what lenses have been “activated” with electrical voltage (they are in yellow) and on the left the resulting focal plane for the user, in a scene where there is a dinosaur is in front of a plant.

Half Dome 3 looks impressive: it is compact, it has eye-tracking, an innovative varifocal display and a very good FOV. It is not clear when it will be on the market, though. And it is not clear if its technology will end on a PC VR headset or on a mobile one.

Full body codec avatars

Abrash showed again the impressive work that Facebook is making in creating ultra-realistic avatars that can mimic all our facial expressions.

But then it went further and also shown a new research that Facebook Reality Labs is carrying on: full body codec avatars, that is ultra-realistic full body avatars. These will be very important in a far future away when we want to meet with distant people in a realistic way.

Anyway, full body avatars at the moment require very big complicated equipment to be implemented, so they are a project for the far future.

Shared spaces

In the idea of defying distance, Facebook has also the true interest of making you meet with other people in real-world locations. Imagine if you wanted to meet your distant father, and so you both wear a VR headset, and you meet inside a photorealistic reconstruction of his house, and both of you see each other as a photo-realistic avatar. This would be fantastic and would help a lot in making people not feeling distant anymore… you could feel as really being in the same place, really being at home together. This would be perfect for the vision of Facebook of defying distance.

Facebook is working exactly on that: in creating realistic reconstruction of environments and in designing a platform to make people meeting there. Again, a long term research project, but the results are encorauging.

Facebook is working on its AR Cloud

I left one of the biggest news as the last of this article. Andrew Bozworth went on the stage and he clearly stated that Facebook is working on AR glasses. But he also said that it is a long term project and for now the company doesn’t want to show any kind of prototypes. According to some rumors, Facebook is building its AR glasses together with Luxottica, and the release date is after 2023… so it is a very long term plan.

But he also went on detailing that FB is working on the infrastructure onto which AR glasses will work, that is a full digital copy of the real world that is needed to create a full mixed reality ecosystem, where all the experiences are consistent in a specific place of the planet and may be shared by all people. That is, to have an AR/MR experience that all people may experience on the top of my desk, the AR glasses of all people of the world must know how it is made my desk, so that to recognize it when they see it. And this must hold true for every location of the planet that wants to be empowered by AR, and so a full digital copy, a full map of the planet is needed to create the true seamless MR experience that we all want. Every company gives this thing a name: Ori Inbar called it the “AR cloud”, some others call it “mirror world”, MagicVerse, CyberVerse, etc… Since people love confusion, also Facebook has given it its name and it is Live Maps.

To create a full mesh of the world, Facebook (like all other companies) will leverage all the cameras of all the devices running in this ecosystem. So, basically, while you use Facebook AR glasses, their cameras will reconstruct the mesh of the environment around you, and send it to facebook cloud together with your position. Facebook AI in the cloud will merge all these partial meshes together, until it will obtain a full mesh of the world, that will be constantly updated by all the devices running on it.

Once the full mesh of the world is ready, devices on it won’t have to perform a full scan every time, but just update it with new information, so they won’t consume much battery and computational power for creating the AR Cloud. And once all the meshes will be ready, it will be also easier creating the Shared Spaces experiences I described you before, because Facebook Servers will already contain the representation of all places of the world, so you will be able to teleport wherever you want, with everyone you want.

Since the devices will be scanning the world for the positional tracking and the AR Cloud stuff, they will also perform computer vision analysis of what they are seeing, so Facebook will be able to detect your surroundings, identify objects and people and give you pieces of advice based on what you see. It can analyze your actions and give you suggestions by using contextual pop-ups. Since all the world is mapped and cataloged, it will be also possible to search inside it: have you forgot where have you put your keys? No worries, launch a search function on the updated map that the system has created of your house and find them in a moment! There can also be a digital assistant that can help you and for instance, reminding you that you are forgetting something. This is an absolutely amazing mixed reality future.

But while it is fascinating, it is also very worrying. This means that Facebook will know exactly everything about our houses, our objects, the people we know, our interactions with them. It will know EVERYTHING about us and the whole world. Boz has said that they are investigating how they can guarantee that everyone’s privacy is protected in this kind of MR world that is a new concept for everyone, but I trust them ZERO when it comes to privacy. I am really afraid of a world where Facebook will know everything about my life, and will use popup notifications to give me perfectly tailored ads at the perfect moment. We need regulators to prevent a full dystopia to happen, or the future of Keiichi Matsuda may become a reality.

Full Keynotes

Here you are the keynotes of days 1 and 2:

---

And that’s it for this full roundup! So, what you think about it? Let me know in the comments and/or on my social media channels!

(PS If this roundup has been useful to you and if you want to make me a little happier after all these hours spent writing it, can you consider subscribing to my newsletter or donating on my Patreon? Thanks!)

(Header image by Oculus)