The XR Week Peek (2021.11.08): Microsoft Teams gets immersive, Meta may open its stores, and more!

Today I’ll travel to the US to be a speaker at AWE, and I hope to meet you all there. I should be packing my suitcase, but instead, I’m here writing this newsletter so that you can enjoy the roundup of best XR news even this week. Since I’m a bit in a hurry, I won’t be as detailed as usual in my commentary, but in any case, there isn’t that much to comment from this week. After the crazy October, we had finally a pause from the important announcements, and we just had a series of interesting pieces of news, but nothing that important.
 
 By the way, stay tuned on this blog to read about my experience at AWE! I’m still not sure what will be the format for that series of articles, but for sure I’ll tell you everything about it! I’ll let you travel with me…

Top news of the week

Microsoft Teams to integrate Mesh in 2022

With a surprise move, Microsoft has announced that in 2022 it will integrate Microsoft Mesh, its framework to create shared XR experiences, inside Microsoft Teams. This will have two main outcomes.
 
 The first one is that people will be able to use a virtual avatar instead of the webcam feed. This won’t require you to use a VR headset, because the avatar will emulate your facial expression by analyzing your voice tone and the images of your face captured by your webcam. Your Microsoft Mesh avatar, something that Microsoft hopes you can use also in other applications, will so be able to substitute the real you, letting you be freer in answering a call in front of a cluttered background or without make-up on. Personally, I would find talking with a digital avatar in a video call a bit weird, but according to Microsoft’s tests, for 70% of people, this is ok.
 
 The second one is that it will be possible to meet virtually inside Teams. That is, there will be an option that will let all the avatars abandon che classical videocall format and meet inside a virtual space (something like Altspace VR) for a more immersive experience. This experience will of course be compatible with XR headsets and will let people meet a bit more like in real life.
 
 Of course, Microsoft has talked about the m**averse in its presentation, and in this case, about an enterprise “m**averse”. It is clear that Microsoft has accepted Facebook’s challenge in the enterprise space, and has answered to Facebook Horizon Workrooms and its recent integration with Zoom with an equivalent product. Even more, Microsoft has currently a big edge over Meta: Microsoft Teams is a solution already employed by many big and small companies. This means that when the update will roll out, all these companies will automatically receive these new XR-oriented features. And if they have these features from Microsoft, they won’t need to get them from Meta.
 
 Of course, we have still to evaluate if this new product will be good, but this smart move gives Microsoft an edge in the competition in the enterprise XR space, and it is a bad hit for Facebook/Meta and its Horizon Workrooms. Meta has to prove that its product is highly superior to the one offered by the competitor to convince potential customers to switch, or they must integrate it directly into Zoom (not just as a compatible software, like it has been announced) if they want to exploit another big existing market.
 
 In any case, it’s good that the competition is ramping up in the space…

More info (Microsoft Mesh to be integrated into Teams)
More info (Detailed article on the announcement)

Other relevant news

Facebook is working on Meta since March, and now plans to open its own stores

CNBC has discovered that Facebook has been working on the concept of the m**averse for at least 3 years. The magazine has had access to a letter written by Jason Rubin in 2018 in which he lays out his idea of the m**averse. In this letter, he says that the m**averse should be the end goal of the company because it can be the true killer application for VR consumers. Some of his ideas actually have been integrated into the actual roadmap of Meta (like for instance the idea of moving before all the other competitors to outrun them), while others have changed: for instance, Rubin was all in for a m**averse completely built by Facebook without the help of any other companies, but now Meta has realized that this journey is too big to be done alone. In fact, Zuckerberg always talks about an open m**averse into which Meta takes part.
 
 In the letter, Rubin is quite bullish about the concept of the m**averse:
 “If delivering the Metaverse we set out to build doesn’t scare the living hell out of us, then it is not the Metaverse we should be building, it is not what customers want, and it is, therefore, meaningless,” he wrote. “Anything else is a Mini-verse.” It’s also funny that he described the real world as the MEATVERSE. I love this name, I think I’ll start adopting it.
 
 Rubin has confirmed that the letter is true, but he also stated that in that period there were many internal documents about the same topic, and his one was just one of the many. This proves that Facebook’s new plan is nothing new and has actually been in the works for many years.
 
 And also the idea of changing Facebook’s name is not new. While some people think that the name change was decided to answer to the recent bad press about Facebook, Meta CMO Alex Schultz has said that the decision was taken in March. This makes a ton of sense considering the insane amount of bureaucracy and internal adjustments that the decision of creating Meta needed. It can’t be something decided the day before.
 
 Talking about the future, a report from the New York Times, says that Meta is exploring the idea of opening its own stores. All major companies have some stores of theirs (think about Amazon’s recent futuristic stores), but of course, Meta wants to specifically copy Apple in this. Apple has the huge advantage that it will be able to showcase its AR/VR products with amazing demos in its dedicated stores, and we know how it is important to make people try AR/VR to understand it. That’s why Meta may be working on creating its own stores. According to the report, at the moment the idea is in the works, but it is not sure that it will be finally approved. I think that either Facebook opens its stores, or it has to partner with already existing stores (like it is doing with Luxottica) to make people try its glasses, otherwise, it will have a hard time selling them.

More info (Jason Rubin’s letter)
More info (Facebook’s name change decision timing)
More info (Meta wants to open its own stores)

Oculus Quest Application SpaceWarp can give apps a +70% boost of performances

At Facebook Connect, Meta has detailed Application Spacewarp, a new warping technique that should give an enormous boost of performances to experiences running on Oculus Quest 2.
 
 Application Spacewarp lets applications render only one frame every two, with the composer reconstructing the missing frames using the previous frame, the rototranslational data of the head, and some values provided by the application (motion vectors and depth map). This means that the rendered frames can take almost double the time to render: since ASW of course needs some resources to run, the total gain can not be 100%, but up to 70%. An enormous result, because this means that the applications can provide much richer graphics as if the Quest had a chipset upgrade. This can help in solving one of the biggest issues of the device, that is the too flat graphics of its games.
 
 ASW can be activated by the developer of the VR experience at his/her wish. It brings a performance boost, but it may also take with it some visual artifact and the fact that the input is sampled at half of the frequency as before. As always, it’s a matter of making a compromise, there’s no free lunch.
 
 This fantastic feature is coming very soon: according to Meta, it will be available to devs in the next 1–2 weeks. I can’t wait to see if it will be able to bring games with richer graphics on the Quest Store.

More info (ASW on Road To VR)
More info (ASW on Upload VR)

Samsung heavily invests in DigiLens to build its AR glasses

Samsung has to lead a new round of investment into DigiLens, which is now evaluated at more than $500M.
 
 DigiLens is one of the most important manufacturers of waveguide optics, the kind of optical system currently employed in AR glasses. Many major companies are heavily investing in waveguides in their run to build AR glasses, and some have also performed strategic acquisitions, like when Snap bought WaveOptics for $500 million earlier this year.
 
 This move seems a clear sign of the fact that Samsung wants to build its AR glasses too and is securing the support of DigiLens as the manufacturer of the related optical systems. Given also the recent leak of some videos about Samsung AR glasses, we should consider also Samsung as a participant in the AR race, which is not only something that concerns Apple and Meta.

More info

News worth a mention

Meta releases v34 update for Quest

Meta has released the Update v34 for Oculus Quest, which brings Android phones notification, Space Sense Guardian, improved voice commands, and more. I have personally tried the new Guardian with Space Sense, which should make you see people and animals that enter your play area, and I’ve found it an interesting idea, but with a too rough implementation.

More info (v34 Update)
More info (My review of Space Sense)

Facebook suspends its facial recognition service

Meta has stopped the face recognition service on Facebook and will delete the facial templates of more than a billion people. There were too many concerns and upcoming regulations about it, so Meta has decided to kill it. I think that this decision deserves big praise from all of us, because it something good for our privacy and for our personal data protection.
 
 On the other end, Meta is not abandoning completely the technology. It will keep the DeepFace face-detection AI engine, that was trained with millions of faces; and is investigating how to use face-detection on the device, without data being transmitted to the cloud, for possible future XR experiences of the m**averse.

More info (Meta stopping facial recognition)
More info (Meta still planning to use facial recognition in the future)

Meta is incentivizing subscriptions inside VR apps

During Facebook Connect, Zuckerberg stated that Meta was interested in games with continuous updates and continuous engagement of the community. He also expressed appreciation for the experiences offering subscriptions.
 
 This week, he has announced some facilities for developers selling their Quest apps through subscriptions:

• Promotional link for creators, that give developers 100% of what they get from the subscription payment
• Possibilities of having the e-mail addresses of the subscribers;
• A bonus program that gives money to the developers for every new subscriber.

These are all very good things, but I find it pretty unfair that they are offered only to devs that sell through subscriptions, considering that subscriptions are more expensive for the users than a single payment.

Also, Zuck underlines how the 30% fee that Apple takes is unfair, but he should remind himself that also taking a cut of 30% on all App Lab experiences is unfair, considering that they are not even discoverable in the store.

More info

Enjoy two interesting hands-on

This week we had two interesting reviews on new VR headsets:

• Ben Lang from Road To VR has reviewed the new HP Reverb G2 and he found it very good
• Sebastian Ang (MRTV) had a look at the new Arpara VR headset and found it incredibly lightweight. He was also surprised by the crisp visuals.

(Thanks Rob for the tip on the second one!)

More info (Hands-on HP Reverb G2)
More info (Hands-on Arpara headset)

Niantic kills Harry Potter and gives birth to Lightship

Niantic has decided to shut down Harry Potter: Wizards Unite, the Pokemon Go clone set in the Harry Potter universe. I hope that at this point the company has understood that Pokemon Go was one of a kind, and no clone is going to work (in fact, they have all shut down).
 
 But Niantic is not surrendering and on the 8th it should launch Lightship, its SDK to create shared multiplayer AR experiences in the AR-Cloud. This should empower developers to create multiplayer games in mobile AR, but considering that mobile AR is currently a flop for what concerns games, I have no idea who is going to use it. I see it more as a product that will become relevant when Niantic will launch its AR glasses, than something that has a valid use now.

More info (Harry Potter: Wizards Unite is winding down)
More info (Lightship SDK launching on 8th)

Tiltfive AR glasses are shipping to their backers

The very interesting AR glasses TiltFive (ideated by Jeri Ellsworth) have started being shipped to their Kickstarter backers. I can’t wait to see what creators will be able to come up with them!

More info

The chip shortage is not going to end soon

Some weeks ago, I reported to you that the chip shortage could get better next year, but according to some news reports, the situation is worse than it seemed and it may take years to fully recover. This is going to impact all future electronic devices, including XR headsets and graphics cards, and will slow down the growth of technology.

More info

Sony patents a new kind of foveated rendering

Upload VR has spotted a patent from Sony that details how it may use eye tracking to implement foveated rendering to boost the performances of its device. What I’ve found incredibly fascinating about the patent is that the foveation algorithm doesn’t regard the pixels to show in the screen, like in all algorithms we know, but also the meshes of the rendered objects. According to the images of the patent, it may be that the system decimates the parts of the meshes that must be rendered in the peripheral vision of the user for an even bigger boost of performance than when you operate only at rendered pixel level. Of course, it is just a patent, so take all of this with a grain of salt.

More info

Participate in these contests!

There are two interesting contests you may participate in to win some money:

• Meta is organizing a big online hackathon from November, 9th to 20th with a total of $700,000 USD in cash prizes, with each team winning a maximum of $55,000 USD. Every team will have to develop an experience either with the new VR Presence Platform or Spark AR. It seems a fun experience
• XR Research Awards is an initiative that offers €40,000 in equipment and support for researchers or content creators to study the impact of virtual reality on behavior. If you are in the XR psychology or neuroscience field, give it a look.

More info (Facebook Hackathon)
More info (Official website of Facebook Hackathon)
More info (Official website of XR Research Awards)

Some news on content

• Blade and Sorcery: Nomad has been reviewed by the major VR magazines. It seems to offer the same experience it has on PC, so it is cool, but still feels too much like a sandbox
• The Twilight Zone VR is coming to Oculus Quest and PSVR as three self-contained episodes
• Gravitational, arrives on PSVR and PC VR on November, 11th
• Wanderer is officially launching on PSVR and PC VR on January 27th, 2022
• Resist, a new action RPG, is releasing for Oculus Quest on November 11
• Arcade Legend, a game that lets you manage an arcade and also play all of its minigames, is launching for Quest and PC VR in 2022
• Resident Evil 2 played in VR with a new mod looks amazing!

More info (Blade and Sorcery review by Road To VR)
More info (Blade and Sorcery review by Upload VR)
More info (Twilight Zone VR)
More info (Gravitational)
More info (Wanderer)
More info (Resist)
More info (Arcade Legend)
More info (Resident Evil 2 VR Mod Gameplay)

News from partners (and friends)

The Secret of Retropolis has been promoted to the official Oculus Store! A well deserved recognition for a very nice indie game.
Learn more (Peanut Button announcing the news)
Learn more (Secret of Retropolis promotion reported by Upload)
Learn more (My review of the game)

VR Blast is a nice game on App Lab mixing Puzzle Bubble, Tetris, and a shooter. It has pretty good reviews.
Learn more

We of New Technology Walkers have been featured for our game HitMotion: Reloaded on VR World Tech Magazine!
Learn more

Some XR fun

Behind the scenes of the recent headsets announcements
Funny link

A random guy has posted a picture of him holding his newly bought Quest and he has immediately become a meme. I love the internet
Funny link / 1
Funny link / 2

Wake up Neo
Funny link

The true purpose of Space Sense
Funny link

The Patreonverse

There is a special place called the Patreonverse, where nothing changes from your actual reality, but you give me some money every month. I know, it’s a pretty bad deal, but it’s still better than most of the metaverse fluff you hear out there every day.
 
 These are the people and companies already immersed in the Patreonverse:

• DeoVR
• Raghu Bathina
• Jonn Fredericks
• Jean-Marc Duyckaerts
• Reynaldo T Zabala
• Richard Penny
• Ilias Kapouranis
• Michael Bruce
• Paolo Leoncini
• Immersive.international
• Bob Fine
• Nikk Mitchell and the great FXG team
• Jake Rubin
• Alexis Huille
• Jennifer Granger
• Jason Moore
• Steve Biggs
• Niels Bogerd
• Julio Cesar Bolivar
• Jan Schroeder
• Kai Curtis
• Francesco Strada
• Sikaar Keita
• Ramin Assadollahi
• Jeff Dawson
• Juan Sotelo
• Andrew Sheldon
• Chris Madsen
• Tracey Wong
• Matthew Allen Fisher
• Horacio Torrendell
• Andrew Deutsch
• Fabien Benetou
• Tatiana Kartashova
• Marco “BeyondTheCastle” Arena
• Eloi Gerard
• Adam Boyd
• Jeremy Dalton
• Siciliana Trevino
• Joel Ward
• Alex P
• Marguerite Espin de la Vega
• Lynn Eades
• Sb
• Vooiage Technologies
• Caroline
• Liam James O’Malley
• Paul Reynolds
• Hillary Charnas
• Donald P
• Wil Stevens
• Brian Peiris
• Matias Nassi

Join this magic world by clicking the link here below!

Enter my Patreonverse

(Header image by Microsoft)