The XR Week Peek (2024.06.11): Apple announces Vision OS 2, Meta fixes passthrough on Quest 3, and more!

June 11, 2024June 11, 2024

Today I’m traveling to the US to attend AWE, so I was hoping that the past week would be a bit boring, so I did not have to write a long post while traveling, but actually, the XR ecosystem decided not to help me because there are many things to talk about. There’s been no massive news, but still, many things are worth mentioning. So, enjoy this rich roundup of news written on the go!

(By the way, last week I promised you I would publish two cool posts, so here you are: my hands-on impressions on the Apple Vision Pro, and a plugin I personally created to let you interact with physical objects in mixed reality!)

Top news of the week

Apple announces VisionOS 2 at WWDC

At WWDC, Apple revealed important pieces of news about the Vision Pro.

First of all, the device is going to be available in more countries: mainland China, Hong Kong, Japan, Singapore, Australia, Canada, France, Germany, and the UK. The launch is already happening this month and every country will have its specialized version with the right language, keyboard, and dedicated apps installed. For instance, in mainland China, there will be the Taobao app, while in Japan there will be the Nikkei one.

Then, VisionOS 2 was announced, and it is going to be released this fall, even if developers can already access a preview of it now. VisionOS 2 is going to bring some important updates, like:

The ability of the headset to transform every standard 2D photo into a spatial 3D photo that you can enjoy on your headset
Wide and ultra-wide high-resolution format for the virtual screen of your Mac seen from the AVP. No, multiple virtual displays won’t be available yet
New environments available
Improvements to the main menu, like the customizability of the icons and a gesture to open it
Enhancements to Personas
Enhancements to the Guest mode, to make it easier for people to share the headset with some friends sometimes
Breathing detection to be used inside meditation apps
Live Captions: the system can transcribe everything the people around you are saying, which is good for accessibility now, but I think will become great for everyone in the future when it is used as a live translation service.

Apart from the updates for the users, there are also interesting updates for developers, like new APIs through which developers can create more powerful apps. The dev updates have been important especially for enterprise developers, that is big companies that are developing applications to use the Vision Pro in their production processes. These special developers may finally get access to the camera frames to run computer vision or AI algorithms, provided that they are authorized by Apple on a case-by-case basis. They will also get barcode and QR code scanning capabilities, which will be very useful inside warehouses.
It seems that all developers will get access to some sort of object tracking from the headset, but enterprise developers will have better control of this feature.

I am so happy about the last part: it’s months that I have advocated for camera access in MR headsets and for object tracking capabilities (you have seen that I’ve just created a plugin to make it possible on Quest 3), and seeing that Apple has finally started providing these features, even if just to enterprise customers, makes me feel to have won a little personal battle for the benefit of the whole community. Of course, it’s not been just me promoting these things, but we all together enticed Apple to evaluate these features and I think that we all did a great job with this.

Last but not least, Apple released Apple Intelligence, which is basically the integration of LLM everywhere inside all Apple devices. Even Siri has been rewritten to interact with generative AI, so you can speak more naturally with it.

At the end of the day, no announcement has been mindblowing, but a few of them, like the 2D photo conversion or the camera access to enterprise, have been quite important.

More info (Apple Vision Pro launches in more countries)
More info (Wide virtual screen for Mac)
More info (2D to 3D photo conversions)
More info (VisionOS 2 is available in preview to developers)
More info (All VisionOS 2 features announced — Road To VR)
More info (All VisionOS 2 features announced — Upload VR)
More info (AVP Enterprise Developers features — Road To VR)
More info (AVP Enterprise Developers features — Upload VR)
More info (Apple Intelligence)

Other relevant news

Meta Connect dates have been announced

Meta has announced that this year’s Meta Connect will take place on September 25–26. We have no further info about this event, if not that it will be possible to watch it on the Facebook pages of Meta and Meta For Developers. Further details will be shared during the summer.

We can anyway speculate that the Meta Connect 2024 will be very important for XR. There are a few topics Meta will for sure talk about:

The launch of the Meta Quest 3S, a mixed reality headset with the ridiculous price of $199
More news about Horizon OS and the headsets that Meta partners are building on top of the Meta platform
A demo of Meta’s AR glasses prototype, something that according to all Meta executives, should blow our minds
Maybe we can also hope for Mark Z to tell us why he’s obsessed with BBQ Sauce

I can’t wait for September, I’m sure Meta Connect will be a great event for the whole XR ecosystem.

More info (Meta Connect — Road To VR)
More info (Meta Connect — Upload VR)

Palmer Luckey is working on a new headset

Palmer Luckey has launched this week a new device intended for gaming: called ModRetro Chromatic, it is his “ultimate tribute to the most important handheld console of all time” (the Game Boy). While talking about it with the community, he also out of the blue announced that he’s going to announce at AWE that he’s working on a new HMD!

Palmer Luckey, the guy who rebooted the whole VR ecosystem, is working on a new headset. This is overly exciting. The details are scarce, so Road To VR tried to ask for more details from him and came out with this additional info: the head-mounted display (HMD) in question will indeed be a VR headset “and more,” however it won’t be shown off “for a while,” as the announcement is intended to precede possible leaks since “it won’t stay secret.”. Whatever this means, the community started dreaming about an Oculus Rift 2.

I will be at AWE cheering for Palmer like a groupie at his speech and I will be very curious to see what he’s working on. But the mission I have now is understanding how we can define the “announcement of an announcement” that Palmer just did. Is the announcement of an announcement an announcement itself? Is it a pre-announcement? If he already announced what he’s going to announce, what’s the point of announcing it again? We need answers…

More info (Palmer Luckey working on a new headset — Road To VR)
More info (Palmer Luckey working on a new headset — Upload VR)

Meta improved the distortions of the passthrough on Quest

The just-released v66 runtime of the Quest is bringing a few interesting updates, like the menu buttons on the wrists and the possibility of hiding applications in the Library list (finally). But the most important of them all is the removal of the distortions of the passthrough.

Quest 3 is a headset with a good passthrough, but it suffers from the problem that the Quest always had with mixed reality: there is evident warping, especially if you start moving the hands in front of your face. Finally, Meta managed to drastically improve on this front: in a recently shared video, the company compares the visuals of the v65 versus the visuals of v66 in some challenging scenarios. The difference is staggering: while v65 shows enormous distortions, v66 has only slight deformations. The improvement is very noticeable: this update is going to improve a lot the experience of the users of mixed reality applications.

I bet that the high-quality distortions-free passthrough of the Vision Pro pushed Meta to improve a lot on this side. Competition is good…

More info (Meta Quest 3 Passthrough Fix — Road To VR)
More info (Meta Quest 3 Passthrough Fix — Upload VR)

HTC is adding body tracking capabilities to Vive Ultimate Trackers

HTC has just showcased in a video that it is adding full body tracking capabilities to its Vive Ultimate Trackers, the trackers that do not work with SteamVR tracking, but that operate independently thanks to onboard cameras.

You may wonder what is so strange about this announcement: Vive Trackers have been used for body tracking for ages. Well, yes, but not this way. In this video released a few days ago, the company is showing that Vive Trackers can be used for Rehabilitation also without wearing them. You just put the tracker on a mount in front of you, and you let the device track your full body pose using its onboard cameras from a distance. Basically, the Vive Ultimate Tracker can become a sort of Kinect.

That’s pretty impressive and would make the Vive Ultimate Tracker useful for many use cases. Unluckily we do not know many details about it, if not that the full body tracking is powered by AI (which is kinda obvious). Since I have practical experience in the field, I can tell you that full-body tracking is incredibly complicated, so I would like to test the quality of the results before expressing any opinion about this functionality. But if it worked, that would be amazing and would make the Vive Ultimate Trackers the most versatile type of tracker around.

(Thanks Ivan Aguilar for the tip)

More info

News worth a mention

Pico 4S to feature the same chipset as Quest 3

New details start to surface about the upcoming Pico 4S, the rumored new consumer-oriented headset by Pico. According to a recent leak, the device should feature the same Snapdragon XR2 Gen2 chipset as the Quest 3, but have more RAM, to be exact 12MB. It seems that this new device will try to compete with the Quest 3, then. I wonder if the competition will be only about VR, or about MR, too…

More info

Meta Summer Fest 2024

Meta has just shared a long blog post in which it talks about its updates on the gaming side. There is a mention of the new mixed reality mode of Supernatural, the new OST7 for Beat Saber, Netflix now working inside the Meta Browser, and more. One of the most interesting announcements is the “Summer Fest”, which is a collection of initiatives related to gaming, including the “annual Summer Sale, exclusive events in Meta Horizon Worlds, and more”. Meta is also going to host many weekends during this Summer when you can play some cool VR games for free. All of this sounds very cool.

More info

Hellsweeper VR devlog

The developers of Hellsweeper VR have written a very interesting blog post in which they talk about how they managed to port the game to PSVR 2 and implement eye-tracked foveated rendering. Many practical pieces of advice are worth a read for all VR developers our there.

More info

Thrillseeker’s video about VR UX

The most shared video of the week has been for sure Thrillseeker’s video about the unsolved problem of UX in immersive realities. He gives his personal insight on what currently XR lacks in terms of user experience and gives practical insights on how we can improve it. I don’t think he says anything that we did not already know, but he explains the concepts so well that he clarified the ideas to many XR folks out there.

More info

An original training for policemen

Axon is providing a training program for Canadian police that teaches them how to operate when someone is having a crisis and becomes a danger to the people around him/her. We’ve all already heard similar news a thousand times, but this time I thought it was worth reporting because this training program has a special part in it. Before doing his duty in the simulation, the policeman also has to embody the person who is “having a bad day” and overreacting. By embodying the “target”, the policeman may understand better what are the reasons that are driving this person to behave this way, so that he is able to better negotiate with him and use force only if it is strictly needed.

More info

Some important news about content

IO Interactive and UK-based studio XR Games have announced that they are going to release this summer “Hitman 3: Reloaded”, a version of Hitman 3 completely rewritten for Quest 3. Preorders are already open with a discount available
Skydance Games, the studio behind The Walking Dead: Saints and Sinners, has started revealing more details about Behemoth. The studio shared 9 minutes of gameplay of the game, plus various interviews for the VR magazines. Upload VR went hands-on with a demo of the game and came out so impressed that they already defined it as “a contender for the game of the year”
Camouflaj has started speaking of its upcoming game Batman: Arkham Shadow. It seems the studio is working hard to exploit all the possible computational power of the Quest so as to be able to implement crisp real-time shadows. Almost no one uses real-time shadows on Quest because they are very resource-consuming, but Camouflaj thinks they are the ideal way to make you truly feel like Batman inside the dark Gotham City. I’m very curious to see what they have been able to achieve
Beat Saber is releasing its OST 7, the last one that will also be available on Quest 1.

More info (Hitman 3: Reloaded)
More info (Behemoth — Gameplay video)
More info (Behemoth — Hands-on)
More info (Batman: Arkham Shadow — Road To VR)
More info (Batman: Arkham Shadow — Upload VR)
More info (Beat Saber)

Some other news about content

There is a discount code to buy the cool VR platform game Max Mustard for only $3
Riven is now available for preorder on Quest 2/3, discounted to $31 from its launch day price of $35
Skatrix Pro, the Vision Pro skateboarding app from legendary skater Rodney Mullen, has just got a massive 2.0 update
Homeworld: Vast Reaches is currently available for Quest 2/3/Pro
MiRacle Pool, a mixed reality billiard game, arrives on June 20 on the Meta Quest platform for $15
The Exit 8 VR brings the Japanese walking horror sim to Quest next month
On Point is a new ‘90s-inspired VR lightgun shooter for Quest and PC VR, and it’s getting a Steam Next Fest demo. Its launch is planned for Q3 2024
Upload has written its usual ICYMI post with some minor XR news
Another great prototype stemming out from a Meta-sponsored hackathon is a cooking assistant simulator

More info (Max Mustard)
More info (Riven)
More info (Skatrix)
More info (Homeworld)
More info (MiRacle Pool)
More info (The Exit 8 VR)
More info (On Point)
More info (ICYMI)
More info (MR Cooking Assistant)

Some reviews about content

VRIDER is a great simulation game for motorbike racing, with a great balance between realism and fun
Sushi Ben tells a manga-themed heartwarming tale filled with great humor and a charming cast. Its minigames are not very deep, though
Blackforge looks like a relaxing game where you just sit around and chill doing some woodworking
Journey To Foundation is a game with a very strong focus on storytelling, but its interaction feels a bit outdated for the current status of VR
Silent Slayer: Vault of the Vampire is a game with a well-crafted game design that is able to constantly keep you in tension

More info (VRIDER)
More info (Sushi Ben)
More info (Blackforge)
More info (Journey To Foundation)
More info (Silent Slayer: Vault of the Vampire)

Other news

Good news: Glue has not been completely lost because it has been acquired by Hyperspace

Learn more

Web3 metaverse The Sandbox has just become a unicorn

Learn more

Somnium partners with TG0 to ship a bundle of a Somnium VR1 headset with Etee controllers

Learn more

HTC launches a bundle of the Vive XR Elite with 4 free accessories

Learn more

A wild Quest 3S has been spotted in the background of a video shared by Meta CTO

Learn more

Microsoft lays off other people from its Mixed Reality division but keeps supporting HoloLens 2

Learn more

New Meta Quest commercials tease its upcoming UI updates

Learn more

News from partners (and friends)

Participate in the AR/VR Survey!

Every year SlashData organizes an AR/VR survey to better understand the XR ecosystem. If you spend some minutes participating in it, you can have the chance to win cool prizes like Bose Ultra Open Earbuds, YubiKeys, Amazon gift cards, Udemy credits, and more!
Learn more and participate

Submit your work to the Beyond The Frame festival

The call for XR projects of this year’s Beyond the Frame festival is now officially open! If you want to participate in the festival XR Competition, you just have to click on the link here below.
Learn more and participate

VRpatients just got a big update

Medical training application VRpatient just got its Release 9 update. With it, you can learn how to operate on a patient in a hospital setting.
Learn more

Discover Rebuff Reality updates

Rebuff Reality just had a few announcements, including an exclusive preview of BodyLink, which brings FREE full-body tracking to Dance Dash on Mobile, PC, and VR
Learn more in the press area

Some XR fun

Let’s say that it looks like half of a PSVR 2…
Funny link

XR headsets are very comfortable… maybe
Funny link

The most accurate prediction about WWDC 24
Funny link

Donate for good

Like last week, also this week in this final paragraph I won’t ask you to donate to my blog, but to the poor people who are facing the consequences of the war. Please donate to the Red Cross to handle the current humanitarian situation in Ukraine. I will leave you the link to do that below.

Let me take a moment before to thank anyway all my Patreon donors for the support they give to me:

Alex Gonzalez VR
DeoVR
GenVR
Eduardo Siman
Jonn Fredericks
Jean-Marc Duyckaerts
Reynaldo T Zabala
Richard Penny
Terry xR. Schussler
Ilias Kapouranis
Paolo Leoncini
Immersive.international
Nikk Mitchell and the great FXG team
Jake Rubin
Alexis Huille
Raghu Bathina
Chris Koomen
Cognitive3D
Wisear (Yacine Achiakh)
Masterpiece X
Dimo Pepelyashev
Jennifer Granger
Jason Moore
Steve Biggs
Julio Cesar Bolivar
Jan Schroeder
Kai Curtis
Francesco Strada
Sikaar Keita
Ramin Assadollahi
Juan Sotelo
Andrew Sheldon
Chris Madsen
Horacio Torrendell
Andrew Deutsch
Fabien Benetou
Tatiana Kartashova
Marco “BeyondTheCastle” Arena
Eloi Gerard
Adam Boyd
Jeremy Dalton
Joel Ward
Alex P
Lynn Eades
Donald P
Casie Lane
Catherine Henry
Qcreator
Ristband (Anne McKinnon & Roman Rappak)
Stephen Robnett
KaihatsuJai
Christopher Boyd
Carol Dalrymple
Sb
Pieter Siekerman
Enrico Poli
Vooiage Technologies
Caroline
Liam James O’Malley
Hillary Charnas
Wil Stevens
Brian Peiris
Francesco Salizzoni
Alan Smithson
Steve R
Brentwahn
Matt Cool
Simplex
Gregory F Gorsuch
Matias Nassi

And now here you are the link to donate:

Support The Red Cross in Ukraine

(Header image by Apple)

Disclaimer: this blog contains advertisement and affiliate links to sustain itself. If you click on an affiliate link, I'll be very happy because I'll earn a small commission on your purchase. You can find my boring full disclosure here.