I have a bit of nausea when writing this roundup because I was playing around with passthrough and created an app that makes you feel like you’re drunk, high, or something like that. I think it’s cool, but my stomach doesn’t share my same enthusiasm… Anyway, I’ll tell you more about it in the upcoming days… for now let’s talk about the disappointment about Google I/O and all the other XR-related news!
Top news of the week
Google did not introduce Android XR at I/O 2024
Google had its most important annual event this week, dubbed “I/O”. But more than speaking about what it has announced there, we of the XR world should talk about what has NOT been announced there, that is the Android XR operating system. Various rumors said that Android XR or the Samsung Headset could have been announced during the event, but actually, XR was rarely mentioned and the event was almost all about artificial intelligence.
Of course, AI is a priority for Google, because for the first time in decades, there is a startup, OpenAI, which could disrupt Google’s business. So we were all aware that AI would have been the main topic. But still, I expected 15–20 minutes dedicated to announcing Android XR. I wonder why it has not been announced: maybe Google management wanted to focus only on the main topic that investors were interested in, that is how to avoid Google being disrupted by OpenAI. Maybe the recent negative sentiment around the Apple Vision Pro made Google take the decision to wait a bit before talking about XR. Maybe the announcement of opening up its operating system by Meta scrambled the cards on the table and Google is still re-assessing its strategy. We do not know. The only thing I know is that I wasted more than one hour of my life watching an event with no XR announcements. The official explanation by Google has been that there were so many things to talk about at I/O and a similar important announcement like the one of an operating system for XR deserved a dedicated event to be explained better. At least, the good news is that Google has not killed the project (yet).
There was a tiny bit of XR at I/O 2024, though. For instance, Google has announced a partnership with HP to manufacture Starline, its immersive headset-less videoconference device. The first units should come out in 2025. Honestly, I’m not excited by this because I personally think this project will be shut down in a few years. While cool, it’s bulky, expensive, and with no clear value.
Then Google announced the introduction of AR elements inside Google Maps. Some selected partners have been able to put augmented reality elements in specific positions in Paris and Singapore, and people using Google Maps are able to click on them and see them in AR through the screen of their device in the selected location. This can be interesting, for instance, to see how the Eiffel Tower was during the Expo in Paris. That’s a good thing because Google is finally “painting the world with data” (as Charlie Fink loves to say), but at the same time, it looks pretty useless to me today because almost no one uses Google Maps in AR on the phone. But it’s a start for having augmentations all around us in the future.
Last but not least, Google showcased the power of its latest multimodal AR engine on a pair of smartglasses. The glasses were similar to the TCL Ray-Neo X2 but were an internal prototype by Google. The video showed the AI seeing what the user had in front of her and answering some of her questions both in written form and reading them loud via the glasses speaker. Let’s say that it looked like an advanced version of what is possible today on the Ray-Ban Meta.
As you can see, XR has not been central at all at Google I/O, but still, we have some interesting news about it to share anyway…
More info (Google explains why it didn’t speak about Android XR at I/O)
More info (The AR announcements at Google I/O)
More info (Google and HP partner for Starline)
More info (Google adds AR elements on Maps but they do not seem useful without glasses)
More info (Multimodal AI demo with smartglasses)
More info (Bonus: Project gameface for accessibility)
Other relevant news
Meta has given visibility to App Lab apps
Meta kept his promise. A few weeks after the announcement, it finally made the indie VR applications on App Lab searchable and discoverable. App Lab apps now appear on search results even when you don’t look for their exact name, they have their own dedicated section where they can be discovered, and when you select them, they don’t show a popup that tells you that your headset is going to explode if you use them. This is very good: we indies needed something like this for a long time.
In my opinion, this is a testament to the importance of indie VR apps: Meta has realized that most of the successful titles in VR have come from indie studios (e.g. Beat Saber and Gorilla Tag) and so it has given more rights to indie titles, hoping to have more successful games surfacing from the community. My friend and indie developer advocate Julien Dorra, instead thinks that this has been made as an answer to Apple: Meta needs to show it has many apps on its headset, and so it is making more apps visible on its store to make it shine against the million iPad apps on the Apple Vision Pro. For sure I agree with him that the Vision Pro made Meta speed up the process.
While I applaud Meta for what it has done, I have to say that unluckily this didn’t have many positive effects on the titles I personally have on App Lab. The download stats are the same as one month ago. Maybe this news is going to be beneficial mostly for new titles and not for the games already present in the store.
More info (Meta created an App Lab category on the Quest Store)
More info (Julien Dorra comments the news)
More info (Existing titles on App Lab had no download spike)
Meta introduced Travel Mode for Quest
After the announcement of the Apple Vision Pro, Meta has started to rush in copying many of its features to show that the Quest is able to do almost the same things for a fraction of its price. The latest of these new features inspired by the vision pro is Travel Mode.
You can now use your Quest 2 and 3 on a plane (train and other means are coming) and watch a movie on a giant virtual screen in front of you, or play a relaxing game like Cubism. The implementation of such a feature was much more complex than I originally thought, so it is pretty remarkable that the people involved in this managed to deliver this feature. One of the difficulties is providing stable tracking without relying on the IMU data: IMUs on the headset will sense that you are moving (because the plane you are on is actually moving) so tracking must rely mostly on optical features. But even in this case, things are not easy: what if you look out of the plane window, where the landscape is actually moving? Meta engineers made the tracking work also in this case, so big kudos to them.
Meta also managed to bring onboard (pun intended) Lufthansa, which will now provide headsets to people in the Business Class Suite. People will be able to enjoy movies, TV shows, 360 videos, meditation activities, and simple games. No, no one will start jumping swinging his arms like a monke while playing Gorilla Tag on your next flight, don’t worry. I’m very curious to see how this pilot program will go, because actually as my friend Chris always warned me, headsets on a commercial flight bring with them a bunch of problems related to hygiene, safety, and other stuff.
More info (Meta announces travel mode for Quest)
More info (Meta Quest travel mode — Upload VR)
News worth a mention
Apple Vision Pro may expand to new countries soon
The rumors about the Apple Vision Pro being sold soon outside of the US are intensifying. If I had to bet, I would say this is going to be announced at WWDC. In the meanwhile, Apple added accessibility features to its Vision Pro and among them, there are system-wide live captions, which may also pave the way to real-time translations for everyone.
More info (Vision Pro sold outside the US)
More info (Vision Pro live captions)
Quest is going to add new UI features
Meta is keeping updating its Quest 3 as well. Two upcoming features are the introduction of menu buttons on the wrists when using hand tracking, something that worked incredibly well already with HoloLens; and the addition of the possibility of pinning 2D windows everywhere in your space, another feature that will make the Quest closer to the Vision Pro.
More info (Wrist buttons)
More info (Window pinning)
Andreessen Horowitz is always more bullish on VR
Andreessen Horowitz, one of the biggest tech VC funds in the US, keeps being very bullish about VR and it is very vocal about it. It has recently published an article with 5 trends of VR gaming which ends with “VR is here to stay”, which is a normal sentence for us XR enthusiasts but it is a pretty bold statement made by investors like them, especially in this moment of crisis of the whole gaming sector. Considering how their VC fund is famous, I’m sure this is going to inspire other investors to invest in VR, which is good for all of us.
1 out of 4 teenagers in the US play with VR
According to a report by Pew Research Center, around 32% of boys and 15% of girls aged 13–17 and resident in the US play with VR. This is a very good percentage and shows that XR is exiting its niche stage. It is also true that the US is the country with the highest distribution of headsets, so I expect these numbers abroad to be much lower.
Haptic stimulation on the brain
One of the most intriguing research projects I’ve got to know this week is about full-body haptics. This research work didn’t use any haptic suit, though: the user was just wearing a VR headset and then there was a device stimulating specific areas of his brain with magnetic impulses (TMS). With this system, the user was able to be touched on his virtual feet and feel haptic sensations on his real feet even if he was not wearing anything on the feet. It’s early-stage experimentation, but it is already mindblowing.
More info (Linkedin post showing a short video about this system)
More info (Research paper about it)
A new way of doing social mixed reality
Greg Madison, who is one of the brightest minds in our space, has created a very fascinating concept for making people meet in mixed reality. It is called Slices and it is based on the concept that you could be in your real room in MR/VR and then merge your room with the room of another person distant from you, taking a slice of your room and a slice of his/her room. The result is that you can create a common space where you can be and cooperate with other people that are distant from you but feel like being in the same space. Watch the video below to be fascinated by Greg’s genius, or jump directly to 27:30 to discover Slices.
Get image frames on Quest 3
A very smart engineer found a hacky way to access the image frames of the cameras of the Meta Quest 3. It is a cool trick: from the app he’s running, he shoots a picture of what the headset sees (you can take pictures and videos from Quest 3), then he from Unity gets the latest image on the gallery and can use that image for other tasks like computer vision analysis. This is hacky and has a million problems, so it can’t be used in a production app, but it truly surprised me with the genius of this solution.
More info (Tweet about Meta camera images / 1)
More info (Tweet about Meta camera images / 2)
More info (Tweet about Meta camera images / 3)
The debate of the week: should people working in VR also play VR?
A Reddit post on /r/virtualreality sparked a big debate in the VR communities: the OP complained that in his VR company, almost no one wanted to play with VR after work. He also complained that people tended to develop games using flatscreen and a VR emulator and not by wearing a VR headset. Most of the people answering the post were shocked about it and the post became pretty viral. Let me give you my 2 cents about it.
I have worked in VR for 10 years, and what he has written does not surprise me at all. If you work with VR, when you stop working maybe you want something different… it’s like saying that you love pastries, but if you go working in a pastry shop, then at dinner you prefer not eating sweeties but pizzas. It’s understandable. I find it absurd anyway that he stated that his colleagues NEVER play VR, The fact is that playing VR content is not only fun, but it is also good to copy smart ideas from other content in the market. So one thing is not willing to play much VR after work, another thing is never playing VR at all.
Regarding the development with the emulator, what he wrote is the RIGHT approach instead. Testing in the headset is to be done mostly during design and at a late development stage, because putting on and off the headset continuously is a huge waste of time and is not good for the productivity of the team. Furthermore, most of the features do not need to be tested specifically in VR (e.g. if you need to test Health-Damage mechanics you don’t need a headset), and doing that on a flatscreen is much more efficient. Let me tell you a weird story about this: when I was in VRROOM, the lead developer and I spent like 2 months implementing complex features without even pressing Play in Unity: we just wrote code and ran unit tests. So working in a VR company may have different nuances, it’s not like in the movies.
LG presents a display for the “next generation headsets”
At SID Display Week 2024 LG Display unveiled a micro-OLED display with “4K” resolution and 10,000 nit brightness. This can be great to offer next-generation headsets with high resolution and bright displays. LG is a partner of Meta, so I wonder if this is what we’ll see in the rumored Quest Pro 2.
Varjo explains how its passthrough with auto-focus works
In quite an interesting technical post on its blog, Varjo explained how it was able to offer in the XR-4 a passthrough vision where you can focus on elements at different depths as you normally focus on them with bare eyes.
(Thanks Chris for the tip!)
Innovega presents its combination of glasses and contact lenses
You know that I’m a big fan of AR contact lenses and these days I’ve been pretty excited to get in touch with another company working on them, called Innovega, which actually combines contact lenses with glasses. Let me paste here how they presented their solution to me, so you can discover more about them, too:
Innovega believes that smart (display) glasses will be an essential element of technologies required for the growth of markets that will leverage new paradigms in media, including 3D, immersion, spatial computing, and AI. Since 80% of humans own a pair of glasses, it seemed that “normal glasses”, and not headsets or goggles would be essential if wearers were to adopt this new interface.
Since humans struggle to focus on items that sit less than 6 inches from one’s eye, designers must place a focusing lens inside display glasses. Consumers also want to experience panoramic-size media, so TVs, computer monitors and smartphone screens have grown in size and have reached their natural limitations.
In the case of display eyewear, the size of the focusing lens and therefore the goggles or headset increases geometrically as the display size and experience expands. Rather than try to break laws of physics, we removed the focusing lens from the eyewear, allowing them to retain the weight, style, and comfort of normal glasses.
Eliminating the large lenses from the eyewear did not eliminate the struggle wearers would have to focus on the near-eye screen that is placed inside the smart glasses. Innovega engineers realized that a 1 mm lens that floats on the surface of the eye, could deliver the same focus of a 300 mm lens placed in a conventional headset. This realization resulted in the design of a soft, disposable, bi-focal contact lens that allowed the wearer to focus on a tiny screen inside the glasses, and also offered conventional vision correction when glasses were not worn. More than half and in certain geographies, up to three-quarters of humans require real-world vision correction.
Innovega added to these benefits of normal glasses and of gaining real-world vision correction, the elimination of eye strain that is experienced by wearers of most every conventional goggles and headset design.
The Innovega solution enables a unique combination of high-performance yet normal glasses form factor. Its team decided to launch a product for a user group that needs and therefore benefits from this capability — vision impaired, including legally blind patients. You will find a reference to this market and to game-changing results from independent testing of our contact lens-glasses combination.
Read about XR at the Cannes Film Festival
The one and only Charlie Fink has written a detailed report about the most intriguing XR experiences presented at the Cannes Film Festival. I think if you are into storytelling you should give it a read.
Some news about content
- RIVEN should come to Quest 2 and 3 on Summer 2024
- Rocking Legend VR is a rhythm game that includes some very Rock Band-inspired gameplay that doesn’t require dedicated guitar controllers (and the studio behind it is Italian… yeah!)
- Schell Games’s Silent Slayer is slated to arrive on Quest 2/3/Pro on June 6th for $20. You get a 10% discount if you preorder now
- Blacktop Hoops, probably the most popular VR basket game is exiting from early access
- Ultimate Swing Golf is a new golf game for Quest, priced at $30
- Weld VR is the most expensive app on App Lab because it costs $999. Of course, it is an enterprise application
- Retronika is an FPS shooter with real-life cycling controls. It should be released on August 2024
- CleanSheet Soccer, a VR game that trains you to be a goalkeeper, is available now on PSVR 2 and the Meta Quest platform for $19.99
- Mecha Party, a game where you compete to become the league’s strongest mecha pilot in a robot battle league, is now available on Steam and PSVR 2
- Upload has written its usual In Case You Missed It post with minor VR news
- FLUX is an app developed during a Meta hackathon where you can build and program virtual reality robots as if they were done physically using Arduino
More info (Riven)
More info (Rocking Legend)
More info (Silent Slayer)
More info (Blacktop Hoops)
More info (Ultimate Swing Golf)
More info (Weld VR)
More info (Retronika)
More info (CleanSheet)
More info (Mecha Party)
More info (ICYMI)
More info (FLUX)
Some reviews about content
- Upload VR has gone hands-on with Masters Of Light and expressed mixed feedback about it
- Upload has also described in detail how Retropolis 2 works and why it is a great evolution of the point-and-click genre.
More info (Masters Of Light)
More info (Retropolis 2)
Other news
Virtual Desktop added a multimonitor feature and the Reddit community has been super happy about it
Pimax introduced a new payment system where you can try the headset before fully buying it
Meta’s research found a way to make holograms with a wider field of view
Ben Lang analyzes the UX of Half-Life: Alyx
CNET reviews very positively the XREAL Air 2 Pro
Exercising in VR is beneficial also for your memory according to a research study
News from partners (and friends)
Discover events about the Apple Vision Pro
XR Bootcamp summarized in a thread on X some important events, courses, and parties for the AVP app creators. If you are an Apple developer or aim at becoming one, check it out.
Learn more
Discover the XR Spotlight newsletter
Luminous XR has started a newsletter where they collect relevant news about immersive realities. Something like this summary I write, but with a different cut. Check it out and see if you enjoy the news they talk about!
Learn more
The Quest: Everest VR Experience comes today!
Starting today, you should be able to watch “The Quest: Everest VR” experience on your Quest (pun intended) and other VR devices. This seems an interesting documentary to watch that is described by the creator as follows:
THE QUEST: Everest VR’ is a one-of-a-kind “real-life” Virtual Reality documentary to climb and reveal the most legendary mountain on earth, Mt. Everest, in stunning 360 degree video! It is the only real-life 1st-person Virtual Reality experience that puts you directly in the boots with filmmaker and explorer, Alex Harz, on an epic 52 day quest to the Top of the World (29,032 feet / 8849 meters). It is the closest thing to climbing Mt. Everest without all the rigorous training, planning and high risk needed to physically step foot on the mountain
Learn more and watch
Sponsored Area
Collaborate with me
If you are in need of someone who can help you with the development of an XR experience or who can suggest a technical strategy for implementing virtual reality in your business, let me know. I have many years of experience on my shoulders and a lot of passion for immersive realities so I’m pretty sure I can give you a helping hand!
Contact me
Some XR fun
The Vision Pro is dead, VR is dead
Funny link
Some irony of mine about Google I/O
Funny link / 1
Funny link / 2
Charlie seeing the glass half full about Google I/O
Funny link
Why Microsoft, why
Funny link
We Italians created an LLM by just using Italian texts and the result is an AI that talks about sex, drugs, fascism, and nonsense. I think it is kinda accurate, after all
Funny link
Donate for good
Like last week, also this week in this final paragraph I won’t ask you to donate to my blog, but to the poor people who are facing the consequences of the war. Please donate to the Red Cross to handle the current humanitarian situation in Ukraine. I will leave you the link to do that below.
Let me take a moment before to thank anyway all my Patreon donors for the support they give to me:
- Alex Gonzalez VR
- DeoVR
- GenVR
- Eduardo Siman
- Jonn Fredericks
- Jean-Marc Duyckaerts
- Reynaldo T Zabala
- Richard Penny
- Terry xR. Schussler
- Ilias Kapouranis
- Paolo Leoncini
- Immersive.international
- Nikk Mitchell and the great FXG team
- Jake Rubin
- Alexis Huille
- Raghu Bathina
- Chris Koomen
- Cognitive3D
- Wisear (Yacine Achiakh)
- Masterpiece X
- Dimo Pepelyashev
- Jennifer Granger
- Jason Moore
- Steve Biggs
- Julio Cesar Bolivar
- Jan Schroeder
- Kai Curtis
- Francesco Strada
- Sikaar Keita
- Ramin Assadollahi
- Juan Sotelo
- Andrew Sheldon
- Chris Madsen
- Horacio Torrendell
- Andrew Deutsch
- Fabien Benetou
- Tatiana Kartashova
- Marco “BeyondTheCastle” Arena
- Eloi Gerard
- Adam Boyd
- Jeremy Dalton
- Joel Ward
- Alex P
- Lynn Eades
- Donald P
- Casie Lane
- Catherine Henry
- Qcreator
- Ristband (Anne McKinnon & Roman Rappak)
- Stephen Robnett
- KaihatsuJai
- Christopher Boyd
- Carol Dalrymple
- Sb
- Pieter Siekerman
- Enrico Poli
- Vooiage Technologies
- Caroline
- Liam James O’Malley
- Hillary Charnas
- Wil Stevens
- Brian Peiris
- Francesco Salizzoni
- Alan Smithson
- Steve R
- Brentwahn
- Matt Cool
- Simplex
- Gregory F Gorsuch
- Matias Nassi
And now here you are the link to donate:
Support The Red Cross in Ukraine
(Header image by Google)