The most important AR and VR news from F8 Refresh

The Keynote of Facebook F8 Refresh, the digital version of the usual Facebook F8 event, has finished some hours ago. Since I’ve watched it all, I think it’s my duty to write an exhaustive summary of the most important AR and VR news announced at this event for you all members of the XR community, so that you are sure not to have missed anything important. Ready? Go!

The most important AR and VR announcements at F8

These have been the most relevant XR announcements from F8 that you must know about:

And that’s it for this post! See you at the next one!

Ok, before leaving, let me actually give you a bit more of context…

F8 Refresh was a minor event

The rumors had already announced it: this F8 has not been the usual big event. For this “Refresh” all-digital version, Facebook has chosen to keep things very lightweight, and focus a lot on the developers’ ecosystem. There has been no big news from this event at all, and all the announcements that have been made were all focused around developer tools (new APIs, etc…). And even among these news for developers, the majority of news were about Facebook, Whatsapp, Instagram, and we had just an update on SparkAR for what concerns AR, and not even a mention to the Oculus Quest for what concerns VR. XR was very marginal.

We already knew that the keynote would have not been carried on by Mark Zuckerberg, but when the speaker announced that Zuckerberg was going to welcome us, I hoped that he was going to make a surprise announcement. Actually… no. He just arrived with a pair of red eyes and a scared face (I’m really afraid of asking what he was doing before) and gave an overview of the upcoming event, saying once the word “metaverse”, and then he left us, returning to do… whatever he was doing.

Then the event went on, and it was all about marketing, social media, and all this boring stuff. The speakers repeated a bazillion times the world “business” (I think they got close to beat the record set during the Vivecon) and a gazillion time “diversity” (that is the quality they don’t have when choosing the words to say during an event).

SparkAR updates

The only moment when someone spoke about AR was during the session about the new updates to SparkAR. A Facebook executive talked about the vision of an MR future, and the fact that Facebook is working on AR glasses… but the present is AR filters like the ones created with SparkAR.

Then he talked about the impressive numbers of SparkAR, with more than 2M of effects created by the community on Facebook and Instagram.

He then reminded us the new features just added to SparkAR, like Multi-class Segmentation, that lets you combine multiple segmentation textures, including hair, skin and person, together in the same effect,

or the possibility to track multiple moving images at the same time to create a joint effect.

Today they have also announced two new features. The first one is full-body tracking. Facebook arrives after Snapchat in this, but better late than never, and this functionality will be amazing to create new categories of filters.

The second one, that I have appreciated the most, is the possibility of creating “multiplayer filters” via the Multipeer API. You can apply a filter during a video call on Messenger, Instagram, and Portal, and this filter can work on all the people of the call together. The filter may be a simple cosmetic filter (like a funny hat that a person can trigger for all the participants of the call to celebrate a birthday remotely) or a little game, with interactions happening between the people participating in the call, that can so play little AR games together.

I think this is great, because it opens up a lot of new possibilities for creators on one side, and makes video calls funnier for the users on the other side. I loved this idea because if you add AR, interactivity, and multiplayer, it is possible to create very fun mini-applications. I can’t wait to see what the AR communities will come up with!

But I wanted AR glasses…

The F8 Refresh was pretty useless for me: I want 80 minutes of my life back!

But I’m glad to have watched it for you, so you haven’t lost your precious time too 🙂 If you want to be thankful for that, why don’t you subscribe to my newsletter?

(Header image from the Facebook livestream)