All you need to know on the Facebook Connect (OC7)

The Facebook Connect is over, and as we had imagined, it has been a very important conference with many announcements, with the most important of all being the release of the Quest 2. Wouldn’t it be cool if someone could summarize all the most important announcements in a single post? Well, your favorite ghost has just done it for you in this article! Read it to be sure not to miss anything important if you had not the time to watch the show. I will also add to it some info that has been shared after the event, to offer you a more complete picture.

Before starting, I remind you that if you find what I do useful, you can donate to my magazine on Patreon to express your appreciation. And now, let’s go 🙂

Oculus Quest 2

The biggest announcement of the event has been for sure the Oculus Quest 2, that Facebook has not only revealed but also launched. Let’s dig a bit into it.

Specifications

• Resolution (per-eye): 1,832 × 1,920 LCD
• Refresh Rate: 90Hz
• Processor: Snapdragon XR2
• RAM: 6GB
• Battery Life: 2–3 hours
• Controllers’ battery life: 4x the one of the Quest
• Field of View: Same of Quest
• IPD Adjust: 58mm, 63mm, 68mm
• Weight: 503g
• Storage: 64GB or 256GB
• Connectivity: Bluetooth/Wi-Fi 6

Description

Oculus Quest 2 is a great piece of technology. We were all waiting for a side-upgrade of the Quest, and instead, we got a huge upgrade, with dramatically improved computational power, display, controllers.

The Snapdragon XR2 is the newest chipset by Qualcomm, with 2x the graphical power of the 835 the Quest was based on, not to mention a huge boost of 11x on AI operations, that could be useful in the future for instance to perform object detection in the headset. Keep in mind that this won’t mean that we will have games that are the double more beautiful than the ones of the Quest, though. There are various reasons for that:

• In the beginning, many games will be porting from the Quest 1, so they won’t exploit completely the XR2 chipset;
• The headset runs at 90Hz and has a 4K display. This means that the GPU must render many more pixels than on the original Quest (90% more, that is almost the double per second) and so part of its new computational power gets lost there;
• The Snapdragon 835 on Quest was actually overclocked, and cooled by an internal fan. This dissipation mechanism is absent in the Quest 2 and this means that it will just use the plain XR2 chipset.

UPDATE (2020.09.27): reddit user geldonyetich commented on the last point clarifying some things

This isn’t quite right.

First, there is a dissipation mechanism. From what I heard during Carmack Unscripted, it’s not that the fan is lacking in the Quest 2. Rather they can’t run it fast enough to allow the XR2 to run higher without draining the batteries too quickly.

Second, we’re not using “the plain XR2 chipset.” The XR2 is benchmarked at being capable of about four times the performance of the 835 in the Quest, but according to Carmack the Quest 2 is shipping stock at only about 46% more CPU and 33% more GPU power than the original Quest. From this, I would assert that the XR2 is actually running a bit underclocked in the stock implementation.

Still, this translates to roughly twice the pixel-crunching power (%100 * 1.46 * 1.33 = %194.18) of the original Quest. The XR2 chip could be clocked twice as fast as that, but it would take roughly four times the power. Some of that untapped potential might come back from mods or future updates.

So in general the games will look much better, but don’t expect PC-like performances. I was pretty shocked to see the video of The Walking Dead on Upload VR and not being able to spot many differences between the two versions. I’m sure this thing will change as soon as developers will learn how to suck every last bit of computational power from the device. Qualcomm states that one of the advantages of the XR2 is that it has various internal units to which the main CPU can offload tasks like decoding video and tracking having so more power to perform the actual logic of the VR applications.

The display is where the new headset will truly shine: an almost 4K LCD display is night-and-day difference with the dual 1440×1600 pixel-per-eye displays of the Quest. Not only the resolution per eye is higher (there are 50% more pixels), but also the fill factor of LCD helps in almost removing the Screen Door Effect. When I tried the Pico Neo 2 Eye, that has a very similar display, I already remained astonished by the quality of the images, and I’m sure that with the great optical system that Oculus always provides (with reduced aberrations), here the visuals will be even better. You will have to focus on the pixels to see them. The framerate is increased to 90Hz for a more fluid experience. Keep in mind that in the beginning, just the initial menu and the browser will exploit it, and the rest will all go at 72Hz. Then Oculus will move the whole experience to 90Hz.

Of course, there are also some shortcomings: LCD means less deep blacks, the lenses have some god rays, and the single panel display makes IPD adjustment less efficient. Carmack has also stated that the FOV may appear a bit smaller than the one of the Quest 1 to some people, depending on their IPD. But all in all, the system is rock solid. Ah, and the lenses have not changed as some rumors said before the launch, but they are the same as in the Oculus Quest 1.

Regarding IPD, Facebook has studied a method to provide a sort of mechanical IPD adjustment with a single display: you have the lenses that can go farther or closer to each other by snapping into three IPD values: 58mm, 63mm, 68mm. Since the eyebox gives around 2.5mm of tolerance, most people from 56 to 70mm of IPD should find a good fit with this device. The fixed values may give problems to people with a value that is exactly in the middle between two of the allowed ones: for instance, the reviewer of Ars Technica found the Quest 2 barely usable until he moved the lenses in a position between two measures that was actually not allowed by the system but that was perfect for his eyes. What is weird of this IPD adjustment is that you don’t have to move a lever to change it, but you actually move the lenses farther and closer with your fingers until they snap. It is not a very safe mechanism, IMHO.

https://gfycat.com/sleepyfluidbats

RAM is improved to 6GB. The device also mounts a Wi-fi 6 modem (Qualcomm FastConnect 6800), meaning that your headset can download games faster and that local streaming of your games from PC to the Quest via Virtual Desktop / Riftcat can become much better (provided you have a compatible modem). This could also open the gates for an official local streaming solution for the future.

The battery life is more or less the same as in the Quest 1, with a battery that has been improved so that to be lighter and be able to support the beefier graphics of the device. The headset is 10% lighter than the Quest 1, but the ergonomics of the base version of the Quest 2 are just slightly better than the one of the Quest 1. Considering that comfort was the n.1 issue of the Quest, I found this a very weird thing, but it makes sense considering the after-market accessory that Facebook is already selling to improve it (more on this later on). For instance, the rear strap is very similar to the one of the Oculus Go, that was absolutely terrible and wasn’t able to sustain the weight of the headset. Expect to feel again some pain on your face after one hour of use of the device.

The controllers have become a mix of the one of the Oculus Quest and the one of the Rift CV 1 (still the best for what concerns ergonomics). They feel better than the ones of the Quest, but they are not perfect according to reviewers. The good news is that the sliding battery lid issue has been solved, and the overall battery duration has been increased 4x (yikes!) thanks to a better tracking algorithm that allows for less power consumption on the IR LEDs of the controllers (thanks TESTED for having pointed this out). The haptic feedback of these controllers has been improved, and they can provide much stronger vibrations than the one of the original Quest. And no, the controllers won’t track all your fingers. And no, you can’t use them with your Quest 1 or your Rift S.

The tracking technology is almost the same, with just the cameras being a bit better in low light conditions. The improvements on this sense will come later on, when thanks to the AI power of the Snapdragon XR2, the Guardian system will be able to detect objects, people, and situations around you (This has not been stated officially, it is a speculation of mine).

Price and availability

Preorders of the Oculus Quest are already open, and the shipping is starting on October, 13th. Price is $299 for the basic 64GB version, and $399 for the 256 GB version. I’ve preordered my basic $299 one (€349 in Italy) and it will arrive on October, 15th. The price is embarrassingly low for the features it offers: the XR2 chipset is around $150 alone, and so you can imagine how much at loss they are selling this headset. Facebook wants to establish a monopoly in the VR sector by ditching out all the competition with these super affordable prices no one else can sustain. It is the console business model: selling the hardware undercost so that subsidize it with exclusive software, and your data in this case. You’re part of the product being sold 🙂

Accessories

Facebook has been very smart in the sense that it is offering the basic headset at a very cheap price, and then it plans to make some more money with official aftermarket accessories:

• Quest 2 Elite Strap ($49 USD): it is something like the Deluxe Audio Strap of the original Vive. The Elite Strap can substitute the terrible new soft back strap with a rigid halo and a fitting knob. The Elite Strap adds also a back cushion and since it adds some weight on the back, it makes the headset more balanced. It DOES NOT include better speakers like the DAS;

• Quest 2 Carrying Case ($49 USD): An official soft box where you can store your Quest and its accessories. It seems elegant from the pictures, but we don’t have reviews about it.
• Quest 2 Elite Strap + Battery Pack + Carrying Case ($129 USD): With this complete pack, Oculus gives you the carrying case plus a special version of the Elite Strap that adds a battery on the back. The battery pack doubles the duration of the battery, while also making the whole headset even more balanced. The bad news is that if you use the battery pack, you can’t use the Link;

• Quest 2 Fit Pack ($39 UDD): It adds wide and narrow facial interfaces to accommodate people with heads of different sizes (my friend and ergonomic expert Rob Cole is surely happy because of this), plus some inserts to reduce the light leaks from the region close to your nose.

Apart from these official accessories there are also some third-party ones that Oculus is selling on its own website:

• Logitech G333 VR In-Ear Headphones ($50 USD): Don’t you want people to hear the VR porn that you are watching? With these tiny earbuds, you can have the privacy of your VR experience without having to wear big headphones;

• Logitech G PRO Gaming Headset ($100 USD): Do you want the top audio quality of over-the-ear big headphones? This gaming headset can give you top audio quality to your VR experience;
• VR Cover Facial Interface & Foam Replacement ($29 USD): VR Cover is well-known to do great covers for VR headsets (and I’ve also reviewed the previous model for Quest). These covers should add comfort to your headset while also helping you in guaranteeing its hygiene.

From a business perspective, the idea of the accessories is great:

• It lets Facebook selling the basic model of the Quest 2 at a ridiculous price of $299, while some of the features they may have added to it are relegated to accessories;
• If the accessories are not sold undercost (as I think), they are a good revenue stream for the company;
• Customers are happy because they are offered official customization solutions.

Kudos to whoever had this idea inside Facebook.

(To discover more about Oculus Quest 2 accessories, I advise you to read this post by Upload VR)

Facebook login

Yes, it has been confirmed: to use this headset you need a mandatory Facebook account. You’re paying only $299 because you are not paying only with your money, but also with the data that Facebook will harvest from you to propose to you better ads.

Of course the temptation is the one of creating a temporary fake Facebook account, but Facebook has clarified that it may suspend your account for 30 days if it finds you are cheating the system, and also cancel your account for bigger violations of the policies. If you lose your Facebook account, you also lose all your games and content you have there, of course. Facebook is aware that new rules have to be studied now that VR and social accounts are unified and there are still not exactly guidelines to follow for account violations (more on this in this article on Road To VR).

Oculus Link

Yes, the Oculus Link works also for the Quest 2, and the whole Link technology will finally come out from Beta this Fall. In the beginning, it will work like the one of the Quest 1 (same resolution and framerate) and then will evolve to take in count the Quest 2 improved specs. What is interesting is that Facebook is bullish that can actually exploit the XR2 chipset (that has some video compression algorithm implemented in hardware) to apply faster encoding/decoding of frames directly on the headset for a better visual quality of the experiences using the Link. If this is going to happen, the Quest 2 may become a fantastic PC VR headset and compete with the other PC VR devices.

Carmack once remarked the 835's decoder maxes out at around 150Mbps.

Can you share what kind of bitrates the XR2's can handle, or is the PC GPU's encoder the limit at this point?

— David Heaney ᯅ (@Heaney555) September 16, 2020

It is interesting that the whole interface of the Quest+Link will change: while now the Quest basically behaves like a Rift when it gets attached via the Link, in the future it will keep its own interface, and when connected via the Link, it will show in its main interfaces some new items. The list of PC games will be added to the list of Quest games, and when you click on them, the system will prompt you to attach your headset to your PC and then start the experience. This kind of seamless interface is great and it is exactly the same one that HTC already offers with the Vive Focus Plus and Viveport Streaming and that I’ve tried and appreciated.

This makes us ask some interesting questions, though:

• If the Quest + Link is not like a Rift anymore, will it be possible to use it to play Steam games? Facebook has not answered to this query, but I think the answer is yes because Facebook knows that removing Steam compatibility would mean reducing the appeal of its top device;
• What will happen to all the Oculus PC runtime? I mean Oculus Dash, Oculus Home 2.0, etc…. these things won’t be noticed by anyone, apart from Rift users, that will become always fewer. My idea is that this PC stuff will become abandonware and will stay there as legacy features until they will be killed when the Rift platform will be canceled (more on this later on).

Link will still be tethered. The god John Carmack expressed some frustrations because in his opinion an “Air Link” working with a dedicated dongle offering a high-performance network would already be good, and many people already enjoy Virtual Desktop streaming without having side effects. Anyway, he claims that the company has imposed very high-quality standards for this feature to be implemented, and it is hard reaching them, so the Air Link has not been approved yet.

Casting to PC and spectator modes

Oculus has announced that Casting, the technology that lets you mirror your Quest to your smartphone or TV will also let you stream the Quest to your browser on PC. This is a very welcome feature for demos.

Furthermore, some games will add Spectator mode, to let people watching it on an external display enjoy a better 3rd person perspective, a bit like what happens on Steam with LIV. This way following the action will be better than with a shaky 1st person perspective.

Reviews

I have not tried the Quest 2 yet because Facebook doesn’t love me enough to send a sample in advance (it’s weird, because I love Facebook so much that soon I will have not one but even two accounts for it!). But I have read many reviews, and the general consensus is that the Quest 2 is a great headset. The only review that completely disagrees with this seems the one from Ars Technica, but it seems a bit biased because of the Facebook login stuff.

The main points that all reviewers make are:

• It is a very good headset with great specs;
• Comfort is not that good, like in the original Quest. But it becomes much better with the Elite Strap, so many journalists consider this purchase mandatory and see the real price of the Quest as $350;
• The Facebook login is a big nuisance because no one wants to use it, and no one trusts Facebook;
• The headset will take some time to show its true potential. For now, many features are locked: in the first times, Oculus will let the Quest 2 operate at 72Hz like the Quest 1, with just an experimental mode at 90Hz. The Oculus Link will start being identical to the one of Quest 1, and then will gain resolution and framerate. The Guardian is starting like the one of the Quest 1, but then it will surely evolve by detecting objects and people around it exploiting the features of the XR2 chipset. Developers still have to learn how to get the most from this device, and Oculus has still to roll out features that exploit the new powerful features of the XR2 (like the AI acceleration). So, for now it is just a slightly better Quest 1, but it has the potentiality of becoming much better in the next 12 months;
• The price point is so low, the content is so great and the specifications are so high that is very hard to say no to this device. Launching before the Holidays means that this headset may sell a lot for Christmas.

Here you are some good reviews you can read about it:

• Road To VR (The best, in my opinion, also because it has been written by my review hero Ben Lang)
• Upload VR
• Ars Technica (The only negative one I’ve read)
• TESTED

New Oculus website

If you have a tour inside https://oculus.com you will see that the website is all different from before, and all the brand identity has changed. The Oculus logo has already been Facebookenized and if you try to look for “Oculus brand assets” you will get redirected to a non-existent Facebook page (meaning that there are no Oculus logo guidelines anymore, it is all Facebook now).

The absorption has been completed, and now Oculus is just the name of some Facebook VR headsets. RIP Oculus.

Oculus Quest For Business adopts Quest 2 at a discounted price

$299 is a ridiculous price for a Quest 2 headset, but since it requires a Facebook account, it is just usable by consumers. Businesses must buy the Oculus Quest 2 through the Oculus Quest For Business program. This program has just been upgraded so that to support the Oculus Quest 2, and the amazing news is that the enterprise price for the Quest 2 256GB edition is just $799. “The Oculus Quest 2 enterprise SKU comes with an extended two-year warranty, one year of the Oculus for Business software platform, and premium support”. After the first year, you have to pay $180/year to keep using the enterprise services and features (like kiosk mode).

While you may think that it is a high price, $799 it is actually $200 less than the enterprise price of the Quest 1 for business, and it is also in a price range where it gets competitive with the Pico Neo 2 ($700-$900) and the Vive Focus Plus ($800). HTC and Pico are still better in the services they offer to companies, but they must be careful because the Quest is a more famous brand.

Rift line gets discontinued

When Oculus launched the new Lenovo WMR headset, we were all pretty puzzled, and we were all even more confused when Oculus released the Oculus Link, making the Rift S de facto useless. During the Oculus Connect 7 aka Facebook Connect, the Rift has barely been mentioned, and Facebook when talking about PC VR content always talked about the Quest 2 + Link. Things sounded a bit weird, but all of this started to make sense when Facebook actually revealed that the Rift line is getting discontinued. Next year, the Rift S will be retired and Oculus will never produce new dedicated PC VR headsets.

Looking at the greater scheme of things, it all makes sense. Oculus had already decided this with the launch of the Quest, and the Rift S was just there to guarantee a transition and to work as a backup plan in case the Quest didn’t work as expected. But the Quest has sold incredibly well, so the Rift line has been killed. Honestly speaking, I think that the whole interest of Facebook on PC VR is decreasing: no exclusive PC VR title has been announced during this Connect, and the only title for which we had an update, that is Medal Of Honor, is weirdly arriving also on Steam. While I’m happy that MOH is not going to be an exclusive title, this is a clear sign that Oculus doesn’t see the PC VR platform as strategic anymore.

The Link is there to offer a useful feature for VR users (and developers), but considering that most Quest users aren’t using Steam (and so aren’t using PC VR much), I wouldn’t be surprised if Oculus would slowly abandon the PC ecosystem in the upcoming years. The news on the Link runtime showing directly the games you have on PC inside your Quest UI is a symptom that there may not be an Oculus PC experience anymore, and that the only runtime that will count will be the one of the Quest. I guess that, as always, Facebook will look at the usage statistics and decide what to do with PC VR.

What doesn’t make sense at all is that Oculus is still selling the Rift S at $399: why should someone spend $400 to buy a headset that is already dead and whose runtime and store are slowly being abandoned? Upload hypothesizes that maybe a 2-year partnership with Lenovo forces them to do so until May 2021. But if I were Oculus, I would slash its price to $299 and would sell all the devices I had in the warehouses.

Facebook is going to launch a pair of smartglasses in 2021

The most unexpected announcement has come at the beginning of the event. Mark Zuckerberg told everyone that Facebook’s goal is creating AR glasses that are light, useful, and fashionable and then he announced a partnership with the Italian glasses manufacturer EssilorLuxottica with the goal to build such a device. Facebook has the XR expertise, Luxottica knows how to build glasses, together they could create amazing AR glasses. This long partnership will start with the first product, that will be shipped in 2021: a pair of fashionable smart glasses under the Ray-Ban brand.

We don’t know much about them, and the only info we have are:

• They will be smartglasses like the North Focals, and not true AR glasses. AR will need more time;
• They will show notifications and probably used together with social networks;
• They will be untethered. In the presentation video, there is clearly the mention of the fact that the future will be untethered. No one wants to keep a cable hanging from his head: it may be annoying, and especially it is not cool to wear;
• They will be cool, like plain Ray-Ban glasses;
• Someone has interpreted the worlds of Zuck so that they mean that these smartglasses won’t have a display and work only with audio (see this post from Upload VR as a reference). I’m not so convinced about this theory, though.

These smart glasses will so be something like the North Focals or the Snap Spectacles depending on which rumor is correct. For now, we don’t have much to discuss on. For sure the announcement at FC and the release date in 2021 have been studied to put some pressure on Apple, which seems to be the only rival to Facebook for what concerns the AR future. But I don’t think Apple is particularly afraid: Apple is never the first (it was not the first for MP3s, for phones, for smartwatches, for wireless earbuds) but usually, when it arrives, it disrupts the market.

Project Aria

As I predicted, Facebook didn’t announce anything truly AR-related, but just showed us some work-in-progress on AR to keep our hype high. Basically, they threw some smoke in our eyes. And the name of this smoke is “Project Aria”.

Project Aria is Facebook’s attempt to start constructing the virtual map of the world (the AR Cloud, that Facebook calls Facebook Live Maps) and at understanding what means wearing glasses the whole day. This device looks like a big pair of glasses (it reminds me the Bose Rondo) and it has installed on a lot of sensors: 2 IR external cameras, 2 IR internal cameras (for eye tracking), 1 RGB camera, 7 microphones, 2 IMUs. It has also on a LED to tell people when it is recording, for privacy reasons. Thanks to all these hardware, Aria can see and hear the surroundings, but can also see and hear the user. It is so a good skeleton to build an AR glass later on, but Facebook underlines how this is just a “research device” and nothing more.

Employees and contractors from Facebook will start wearing this glass in September and live their lives with this device on. The device will almost always capture all the data it can, so that it can be studied later on by Facebook engineers. Aria will serve to:

• Experiment with creating the Facebook Live Maps;
• Experiment with user interactions in AR: Aria can track the eyes and the hands of its user, and with the microphones, it can also capture his voice. This means that the data collected can be used to improve the various tracking algorithms for the various parts of the body. It can also be used to analyze how the user is expected to behave with an AR glass on;
• Evaluate the comfort of wearing an AR glass for hours;
• Evaluate what are the social reactions towards AR glasses wearers. Pepperidge Farm remembers when people wearing Google Glasses were called “Glassholes” and kicked out of shops (a nice prize for having spent $1500 on a new device) and Facebook wants to see if in 2020 there could be similar problems (“Ariassholes” sounds good).

Basically Facebook wants to start experimenting with AR in real life and not only in the lab. A curious detail on Aria is that it is full of sensors but it has no display. It is meant just to collect useful data, not to provide output yet. That will come in the second step of the project. Imagine the people wearing Aria like the guys wearing Google Maps backpacks inside museums of campuses: they go around and collect data to build some maps. The real difference is that they wear the sensors on their face.

This pilot program will be run by Facebook together with Carnegie Mellon University’s Cognitive Assistance Laboratory with the purpose of building 3D maps of museums and airports that will have multiple applications, including helping people with visual impairments better navigate their surroundings. All the data of this experiment will be managed by the University.

I think the name ARIA is a mix of AR and AI, because the glass goes around collecting data for the AI, and since ARIA means also “air” in Italy, it can be a suggestion about the fact that it is used outside the lab.

Also Aria is untethered but it works together with an external box. This attitude makes me understand that Facebook won’t release an all-in-two headset with a cable like the Magic Leap One, and will always go for an untethered approach.

For more news on Project Aria you can refer to this official blog post by Facebook.

Privacy in AR

Of course, Facebook knew that a device built by them just to collect “data” around the streets could have raised some concerns, so Boz has immediately clarified that they are building this device together with maximum privacy experts and with new privacy principles. Some of these guidelines they are following are:

• In the data collected walking around, the faces of the people are blurred, and so are the faceplates;
• The data collected is encrypted and stored in a secure way;
• The headset doesn’t record in private spaces like the restrooms;
• Every recording employee has 3 days to erase part of his recording (maybe because he was doing something private in that part) before the data gets actually analyzed and used;
• The employees using Aria will have a T-shirt clearly identifying them;
• And so on…

These are all great rules and I personally appreciate that Facebook understands that AR poses enormous problems concerning privacy. But personally I don’t trust them for four reasons and I think that whoever trusts them is naive.

The first reason is that “a man is worth his word” and Facebook in XR has already a history of breaking its promises. Do you remember when it promised Palmer Luckey that Oculus devices would have never needed a Facebook login? Or when they told that Oculus would have remained an independent company, and now it is so independent that is called “Facebook Reality Labs”? Trust is important in life and they have never proven to be worth my trust.

The second reason is that Facebook has a long history of shady practices, like when a report got that the Facebook app installed on your phone harvested all possible data it could about your phone, like the statistics on your usage of your other apps (data it shouldn’t have been able to get). Not to mention the ongoing debate on what kind of content should be allowed to stay on Facebook (hate speech, etc…)

The third reason is that the line between what can be an can not be recorded is very blurred. They say they won’t record people going to the bathroom… but how this will be detected? And what if I’m crying in the bathroom and want to have a shared videocall with my friends to cheer me up? Doesn’t this mean that bathroom can be tracked to get AR as well? Then what activities exactly won’t be recorded? Who decides what is legit and what not? And what about the ongoing training? I always refused to have an Alexa in my house because I know that what I say may randomly be sent to some Amazon offices in India where a person listens to my worlds and checks if the AI has detected well what I’ve said, with a clear violation of my privacy. Will something like that be done by Facebook as well? There are too many open questions.

The fourth reason is actually the nail in the coffin for my trust in Facebook in XR. While the first two can be proven wrong in the future if Zuck enters the Nirvana and he understands that his company must be good, and the third one may be will be solved by a super-AI that understands what data of mine are private and what not, the fourth one is just what it is impossible to change. And it is: Facebook is an ad company, probably the best one around. I don’t know if you are in business, but if you are and you have already done some marketing, you know that Facebook is the best marketing tool. Since it harvests lots of data, it is amazingly good at profiling, and so it is amazingly good at making you target exactly the kind of people that you want to reach for your product. It offers the best ad performances for your buck and when you want to start a commercial campaign, it is the best platform to use.

Facebook’s business is harvesting data and providing a great ad service to marketers. It is in the DNA of the company. So, even if in AR they stop recording when you pee or when you have sex, even if they obscure some personal data, even if they do their best to not enter your intimate space, they MUST gather data about you, because it is their business. So they’ll record in which shops you’ll go, how do you buy stuff in supermarkets, what are your reactions when you see certain products, how your eyes react to their ads, what products you use the most, etc… In a way or in another, they must do that, because you must become the target of their ads. It is just impossible they won’t record anything about your personal life, because for them it would mean not being anymore the best at offering a marketing service. So, privacy can’t be guaranteed at all with a Facebook product, because the more data they get about you, the better for them.

Every company collects data, but I trust more companies like Apple, Microsoft, HTC, Valve who have other business models (like selling games, cloud services, phones, games) and so can guarantee better privacy to the user. I don’t trust Facebook, Google, and all the other marketers where “You are the product” being sold.

VR Content

“Content is the King” and Facebook knows it well. If you remember, some Oculus Connects ago, they stated they were investing $250M a year in good content. Oculus Quest is the best product not because of its hardware (that is more or less the same of the Vive Focus Plus), but for its software: the good runtime and the amazing exclusive games. That’s why with the launch of the Quest 2, they have announced some new fantastic titles (that will be compatible with both Quest and Quest 2):

• Sniper Elite VR. This sniping gaming is coming to VR, Quest included;
• The Walking Dead: Saints & Sinners. The game is a launch title of the Quest 2 and will be shipped on October, 13th;
• Warhammer 40,000: Battle Sister. An experience in the Warhammer universe, with the main character being a woman. It is launching later this year for Quest;
• Star Wars: Tales from the Galaxy’s Edge. ILMxLAB has showcased a little teaser of this game and announced that it is coming this holiday season on Quest;

• The Climb 2. Crytek is releasing the sequel of the popular climbing game, with more features and richer environments. Coming this holiday season on Quest;
• Jurassic World Aftermath. A game in the Jurassic World environment, this time with comic-like graphics. Coming soon by Universal and Coatsink;
• Population: One. The Battle Royale game is coming to VR later this year;
• Pistol Whip: 2089 Update. A new upgrade of Pistol Whip with five story-driven sci-fi-themed chapters in Cinematic Campaign Mode. It’s coming this holiday season;
• Myst. The puzzle game from 1993 is coming to VR again in 2020, and with VR I don’t mean only Quest but also Steam;

• Puzzle Bobble VR. Survios and TAITO bring this game back from the past and take it to VR, to please the Japanese market. Coming soon;
• Rez Infinite. The trippy action-shooter is coming as a launch title for the Quest 2 on October, 13th;
• Space Channel 5 VR. SEGA’s classic game is coming on Quest 2 to please Japanese people as well on October, 13th;
• Little Witch Academia VR. This VR broom racing game is coming at launch day on Quest 2;
• Kizuna AI: Touch the Beat. Dance and Cheer with Kizuna AI on Quest, from October 13th;
• ALTDEUS: Beyond Chronos. Coming to Quest and Rift on December, 3rd, this game is a sci-fi adventure full of robot battles, pop music, and emotion.

You may think that the list is over, but actually I left the juiciest news out of it. First of all, Lone Echo is still in the work and will still target PC VR, but Facebook hasn’t shared any release date.

Medal Of Honor: Above and Beyond is going to launch on December, 11th, just in time for the holidays. It will be a PC VR game, targeting the Quest 2 + Link. BUT surprise surprise, it will be available also on Steam. This piece of news shocked us all a bit and makes everyone understand how Oculus doesn’t care much about the domination on the PC VR platform anymore.

The most popular VR game, Beat Saber, is going to add for the launch of the Quest 2 a new multiplayer mode, with up to 5 people dancing and playing together (no Oculus and Steam crossplay in the beginning, though). And in November, a new DLC themed after the songs of the BTS is coming. I have no idea who the BTS are, but Facebook people seemed so excited of having signed a deal with them.

Dulcis in fundo, Facebook has officially confirmed it is working with Ubisoft to take both Splinter Cell and Assassin’s Creed to VR. And they will both be high-quality AAA games. This is an enormous piece of news because this means that in the next years, we’ll have two amazing big games using very famous IPs in VR and this will attract many gamers to our ecosystem. Ah, of course, they’ll be Oculus exclusive titles.

For more info on the content announced, I advise you to read this roundup post by Road To VR.

Japan

During the conference, Facebook has announced that with the launch of the Quest 2 it wants to give more attention to the Japanese market, and the Quest 2 will be easier to be found in Japanese stores. This attention to the Japanese market is also noticeable in some games announced above: some of them are arcade-style games or games inspired by anime.

I don’t know why there is this explicit interest for Japan: maybe they noticed that in Japan they sold very little, and it is actually a very profitable market for electronic innovations. Or maybe Zuck is a fan of hentai movies. I don’t know, but my Japanese friends will be happy about this news.

New Content Distribution platform

No one likes the fact that the Oculus Store has very strict approval guidelines and the success of SideQuest is proof of the fact that people also enjoy VR experiments and indie games. Oculus has listened to the community and so it is going to create a new content distribution platform for content not approved on the Oculus Store.

This platform is going to be launched in “early 2021” and will allow programs to be basically published on the Oculus Store, without being visible on the Oculus Store. Imagine it as the Dark Web of the Oculus Store: those pages are there, but they are not indexed, so you don’t see them while you browse the store. But if you already have the link to the store page of these hidden programs, you can access them via the Oculus app and install them on your Quest.

It is actually a very smart way of solving the problem. This way Oculus doesn’t make its curated store dirty (because these games don’t appear in its public page), but at the same time, creators can distribute easily their apps with the same services that official apps have (e.g. a dedicated page, automatic updates, etc…). And SideQuest won’t disappear, but it will just become the storefront through which you can search all these hidden pages (like the search engines of the Dark Web, but without weapons and drugs :D). Everyone wins, both the developers and the consumers of this conten.

This is all fantastic, if not for two problems.

The first is the Facebook Login. As Chris Pruett confirmed to me, Facebook consider this as an official way of downloading content for the Quest, so you must have a Facebook account to enjoy it, even on the Quest 1.

The second one is that it is not clear how much control Facebook wants to apply over this content. If these games are hosted on Facebook servers, they will have to adhere to Facebook TOS, I guess, and this means that Facebook would like to have some control over them. Not to mention that probably it will want a 30% share on paid titles. So, my question, that I ask FOR A FRIEND is: Sex Like Real, that has a popular porn app on the Quest, will be able to use this distribution method considering that porn is not allowed on Facebook products? And what about all the customs songs for Audica, Beat Saber, etc… that are technically illegal? We need an answer, but my fear is that Facebook actually will use this new distribution platform to close the sideloading hole that was opened by SideQuest, applying finally more control over the off-store content. For 95% of cases, this is ok, and even welcome if it stops dangerous or offending content, but for the remaining 5% this may be a problem.

(More info on this on Upload VR)

Oculus Move

With great surprise of Carmack and the other people of the Facebook Reality Labs team (that is, Oculus), people love playing fitness game in VR. This was unexpected because they thought no one wanted to sweat inside a headset, but actually it has happened and games like Box VR have become incredibly popular.

Facebook has so decided to surf this wave, and will release on Quest Oculus Move, a full fitness tracker in VR dedicated to the Oculus Quest. Move will calculate how much calories you will burn playing each title, and then you can watch statistics about your daily and monthly consumption, also divided by game.

It can also motivate you by letting you set some goals and see if you reach them (to add gamification to the mix). Move is the perfect fitness companion for the Quest, and in the future will probably integrate also with other fitness trackers. I feel sorry for YUR that was working on a similar system… they had better focus on the PC platform now.

Oculus Move is coming later this year.

Oculus Quest platform profitability

To demonstrate the profitability of hosting a game on the Oculus Quest platform, Facebook has revealed that now more than 35 titles have grossed more than $1M on the Quest store, with 3 titles grossing more than $5M. Considering that back in May Quest users had spent $100 million on games and apps, this means that $50 million in revenue have been generated in less than four months. It’s a lot of money for VR, and it proves how the Quest platform is profitable and successful. And especially, as investor Tipatat Chennavasin points out, how the success on it is repeatable, because games earning a lot are not only outliers. This is a piece of very good news for developers: we need platforms that can give us a sustainable businesses.

Mike Verdu has underlined how Onward has grossed $1M in just four days, and now you get while they rushed the game out for Quest, ruining completely the experience for PC users. The economic gain of this operation has been so great that they could afford to lose some old-school angry gamers. Not that I approve this way of doing, but now I understand better why they haven’t waited for the release.

These numbers are great, but remember that numbers without a context mean nothing. On Steam, Half-Life: Alyx has grossed probably around $30M in just a few weeks, and it is a game alone on the whole Steam. Animal Crossing on Switch (a game console Quest wants to compare with) has sold 10M copies in Q2 this year, grossing $600M ALONE. What I want to mean is that the number $50M of content sold in 4 months may be good or bad depending what you compare it to. But what is interesting is that there are many games earning more than $1M and so the platform is healthy, even if still niche if compared to traditional consoles.

Infinite Office

Facebook plans to transform your VR headset in your future personal computer. And with “VR headset”, I mean the Quest 2. This is one of the reasons why they are not interested in PC VR anymore: they see the PC disappearing and being substituted by an XR headset in the next 5-10 years.

One of the goals is so evolving the Quest 2 from the gaming console that the Quest 1 was to a more general-purpose device. This means also transforming your Quest in a productivity tool that you use every day for your work like today you use your laptop. This is something that will need some years, of course. For now, they are going this Winter to release “Infinite Office”, which is the first step in that sense. Look at the announcement video to have a glimpse of it.

With “Infinite Office”:

• Your Quest OS can let you multitask between different 2D apps (like Browser or the just launched Messenger for Quest);
• You have multiple virtual displays that you can resize and put where you want;
• These big 2D virtual screens can be attached to real surfaces that the Quest will detect: tables, chairs, sofas, etc… and you will find them again every time you enter in the room where you put them
• You can operate with the various screens by just using hands tracking
• Everything can work in passthrough AR mode, but you can also use it in a VR environment
• You can connect a special keyboard made by Logitech to your Quest so that you can finally type on a real keyboard to write easily in VR! And since this keyboard has also a touchpad, you can use it also to have a mouse in VR.

https://gfycat.com/distinctnextdunlin

The video looks a bit too promotional, and in my opinion it sets unrealistic expectations, but it makes you understand that with Infinite Office you can have your productivity screens installed everywhere, in every room. You just take with you your Quest 2 and you can have as many big virtual screens as you want, and this is fantastic for creatives for instance. Imagine working on the plane by just having to wear a glass on your face, wouldn’t be cool? I remember someone years ago posting a video of he using Excel on HoloLens on the plane, it was cool.

Anyway, I think it is just a start, because there are some practical problems:

• Quest 2 resolution is good, but text isn’t always readable if these virtual screens are not big enough;
• With Quest 1 the system 30% of times forgot about the places where you had been before and you had to re-set the Guardian, so the risk here is that every time you have to put the screens where you want them to be;
• Seeing through the black-white-distorted passthrough can be tiresome for the eyes on the long run;
• Not everyone wants to buy a Logitech keyboard just for this purpose. UPDATE: Zach Wendt told me on Twitter that also other keyboards may work with this system;

Excellent overview!

One tweak: Carmack confirmed the Logitech keyboard is not special, just guaranteed to track. Anything that looks similar will also work.

Reddit will figure them out the first week. 🙂

— Zach Wendt (@zach_wendt) September 19, 2020

• There are no tools I want to use this way. The video looks dull because she uses Messenger and some online tools on the web. We need Facebook to provide us the Excel, Word, E-mail readers, etc… all the software that we would like to use in this mode. Otherwise, it just becomes a gimmick that you use for 1 hour. I’m a developer, where is Unity, where is Visual Studio?

I don’t want to sound too much critical: “Infinite Office” is very cool, I just want you to understand that is just the first step towards having your office in VR, but the road is still very very long.

I know it because I did a prototype of something similar with the Vive Cosmos XR, that has slightly less screen resolution than the Quest 2 but much better passthrough than the Quest 2, and I saw how difficult it is building something like that:

Infinite Office SDK

While talking about Infinite Office, Facebook dropped the bomb news: later on, they’ll give developers the SDK to create their applications with the same technology of Infinite Office. This SDK is already being used by selected partners like Spatial.

Yes, you’ve read it well: we will be able to use the Oculus SDK to create mixed reality applications. I guess that in the beginning, they’ll be simple ones with just an AR overlay and then they’ll become more complex and more context-aware.

Well, this means that we’ll be able to port our game HitMotion: Reloaded (the first mixed-reality fitness game) to the Quest 2! :O

Messenger in VR

Messenger is coming to the Oculus Quest 2. You will be able to connect to your friends while you are in virtual reality and chat with them without removing your headset. I personally don’t use Messenger much, but I recognize that it is a good idea to connect people in VR together, especially if this gets connected to VR functionalities. Imagine being able from a Messenger group to start a multiplayer game together… that would be very nice!

Horizon is not opening its gates

One of the most common predictions for this Facebook Connect was the launch of Facebook Horizon. This is not happening though, and Horizon remains in a closed beta stage.

Horizon avatars become the official Oculus avatars

I called this in my predictions post and in fact, this has happened: the new avatars that Facebook has built for Venues and Facebook Horizon will become the official avatars for all the Faceboculus ecosystem. In all games that will employ Oculus avatars, you will be represented by these new cartoonish and expressive characters. And the news is that one day you will also be able to use them for your profile picture in Messenger.

This is a very good idea because it gives you a digital identity across all the Facebook ecosystem. The theme of digital identity is very important in VR because as you’re always yourself in real life, you should always be your digital self in your virtual life. The broader use your avatar has, the better.

Challenges

Facebook is introducing new social features also to games. With “Challenges”, people can challenge themselves, their friends or the whole VR community in performing a better score in a particular game. It is like if you could create mini-tournaments for certain games.

Challenges must be explicitly supported by the developers, and will be available in selected games like Synth Riders at start.

AR in Messenger and Portal

Spark AR has proven to be a great platform for creators, that can so easily create AR filters that get then used by millions of people. Facebook reports that “Today, Spark AR is the largest platform for mobile AR, with more than 400,000 creators from 190 countries, who have published over 1.2 million AR effects on Facebook and Instagram. And in just the last three months, more than 150 accounts have seen their effects generate more than 1 billion views”.

Strong of this success, Facebook is expanding the creation of custom filters from Instagram to also Messenger and Portal. This gives creators new channels to express themselves.

Michael Abrash

https://gfycat.com/warmboringkoalabear

I think that Abrash has been less brilliant than in other years, but his speech was very interesting nonetheless. He talked about the development of XR technology at Facebook to arrive at the final goal of having lightweight AR glasses we wear all day. And this work requires people from many sectors: neuroscience, optics, computer science, electronics, etc… Facebook is working on all the technologies need to build this kind of glass, and has a clear internal roadmap on how to get there. He anyway thinks that we will need at least 10 years of R&D to reach to the lightweight-AR-glasses point.

Abrash has highlighted the work that Facebook is doing in some of these various sectors.

They’re working with Ctrl+Labs about using electromyography (EMG) to let you interact with your hands in your virtual environment without having cameras tracking your fingers, but just an EMG bracelet that is very accurate (he says up to 1mm for fingers positions).

https://gfycat.com/faithfulsomeavians

The cool thing about EMG is that once it has been trained, it understands your intentions even if you don’t actually make the full movement of your finger. So you can type in VR without having to actually move completely your fingers. And then with EMG, we can study if the brain, thanks to its neuroplasticity, can adapt and also control the VR system with something that goes beyond the physical 5-fingers hands. What if the brain could control a hand with 6 fingers? That could enhance our interactions. They made a disabled person with only 3 fingers interact in VR with this kind of devices, and in just 5 minutes he learned how to control a virtual 5-fingers hand. The possibilities are mind-blowing.

https://gfycat.com/negligiblepassionatedamselfly

Some weeks ago, Facebook had teased its research on audio, and Abrash has explained in deeper detail this work. He explained that thanks to Facebook technologies, in the future you will be able to speak with a group of your friends in a crowded environment, and listen only to their voices and almost not hear the background noise. This will be possible because FRL is studying both how to have audio from your AR glasses that sounds like real (by calculating your HRTF function) and how to filter out all the audio that doesn’t come from the audio sources you’re interested in (basically making you hear only your friends talking and filtering out all the rest). Of course, this means that you will only hear all the time through Facebook earbuds, that will filter out everything you don’t want to hear and will make you hear everything that you want to hear in a crisp and very realistic way. “What you want to hear” is detected by an AI, that understands the context of your situation and matches it with what you’re eyes are looking at (who are you looking at) and your past history (it knows who are your friends). This is fascinating, but also creepy (imagine if the system filters out something that it wants to hide without you knowing…). At the moment, though, it must be you activating this kind of feature.

What is even more fascinating, that among these friends in the crowded environment, some of them may not be physically there, but you would see and hear you being there with you, in a very realistic way, thanks to AR. And the AR system could position them perfectly on a seat around the table with you, finding the perfect spot to put his/her avatar.

All these fascinations are built by an AI assistant. Facebook is not promising a humanoid AI assistant like Magic Leap’s Mica, but an underlying AI service that understands the context of the situation and helps you with advice, or by activating/deactivating automatically certain features. Context is of paramount importance according to Abrash: the system must provide you the right info exactly when you need them, not to clutter your visuals every time with too much info. For instance, it could detect that you are going out of your house without taking the keys and it could warn you. This could be very useful.

But to build this kind of assistant, and also to build a truly shared AR experience, we need the AR Cloud, that is Facebook Reality Maps. Abrash has told that Facebook needs to build a complete digital map of the world. This is fundamental because AR glasses would need too much computational power and battery consumption to perform every time the tracking from scratch: it is much better if they can pre-load from the cloud a version of the surroundings already calculated, and then just update the map given what they’re seeing now (and send the update to the cloud of course). This way the tracking procedure is more efficient. Once there is a shared map of the world, creating shared AR experiences on a massive scale is easier. Yes, this creates enormous privacy concerns, but I have already touched this theme above.

Notice that as Abrash says, it is not only the map of the world that matters, it is also very important understanding what is happening: what are the objects that are in the scene, what are they doing, and what relations they have among them. This kind of information is what gives the AR glasses the context, that makes them understand what is happening, and let the AI assistant actually help you in your everyday life.

These are all very interesting research projects FRL Research is working on and that will lead to the AR glasses of our dreams.

John Carmack

This year the god Carmack has not only performed his usual unscripted talk live on Facebook, but he has also gone to Horizon for a Q&A session. Here you are the videos of both talks, courtesy of Upload and InsomniaDoodle. As always he said lots of cool things worth listening to: it is impossible to summarize them all, but Upload has made a good job in segmenting them.

Other

Some other tiny bits of news:

• Zuck defined AR/VR the most social technology ever. He said that it is also very useful to work and he finds incredible doing business meetings in VR (in something like Horizon), because it is so immersive, and the shared space gives more realism (I call this bullshit: having a business meeting in VR is weird at the moment, and doing it in a cartoonish world like Horizon is cringy). Facebook plans to move 50% of its workforce to remote work, so they are building these social tools in VR because they are useful to them in the first instance;
• Apart from games, Oculus is also announcing In Protest, and interesting documentary about the current protests of the Afroamerican communities in the States. “IN PROTEST: Grassroots Stories From The Front Lines” is a series of 360 videos made by black storytellers that spotlights “the unsung heroes responding to the recent race injustices in America”;
• Oculus has also announced a partnership with some museums, like the Smithsonian, to bring heritage content to your home through AR;
• Portal will be enhanced to let people also tell stories via the device, so that parents can tell remotely stories to their children. These stories will be powered by AR, so the children can also see AR effects surrounding their parents during this story time. The new book allowing for this kind of feature will be “Thank You, Omu” by Oge Mura.

Official Videos

Here you are the official video of the keynote

And here you are the playlist with all the videos of all the sessions. Enjoy!

---

And that’s it! I hope you liked this huge roundup and it has been useful for you! Don’t forget to express your comments on these pieces of news in the comments below!

And if you appreciate my work, please subscribe to my newsletter and support me on Patreon. Have a nice VR Sunday!

(Header image by Facebook)