Trends and predictions for XR in 2024

We are at the beginning of March and I just realized that I haven’t written yet a post with my predictions for augmented and virtual reality for this year. So (a bit late) here you are a list of some of the most relevant XR trends to follow in 2024, with some predictions of what to expect next. (Some of them have been brainstormed with Lorenzo Cappannari with whom I wrote an article a few months ago and who I thank for the collaboration 🙂 ). I don’t know how many of them will prove to be true, but for sure they all can be something that can inspire you to define your XR strategy for this year.

A new cycle of XR

As I said in a popular post a few months ago, I think we have entered a new cycle for immersive realities. The last cycle before this one was the Quest one, which started to make virtual reality known to the mainstream audience for what concerns games and fitness. Anyway, with the end of the life of Quest 2, this cycle finished and a new one started with Quest 3, Apple Vision Pro, Google and Samsung announcing their own headset, and other companies chasing them (e.g. Huawei and Pico are rumored to be working on an Apple Vision Pro competitor). These new headsets have hybrid virtual and mixed reality features, natural interactions and come at a high price: Vision Pro costs $3500 and the Samsung headset is rumored to be around $2000.

We can say the new cycle started with the announcement of the Apple Vision Pro, because that has been a disruptive moment for our market: Road To VR’s Ben Lang says that it has been the most important moment for virtual reality after Facebook’s acquisition of Oculus, and I agree. When Mark Zuckerberg acquired Oculus, all the eyes of the tech industry started looking at VR as the “next big thing”. And now that Apple entered the field, all people in the technology world are talking about the Vision Pro, and especially of the future evolutions of the device, which could become mainstream. Investors, which were all focused on AI started to wonder if they should focus on XR again.

Something new has started, and it is not clear where it will lead us, but I have the sensation that this new cycle will be an important step for immersive realities.

The big players enter the field

One of the most encouraging signs of this new cycle is that all the big players are now in the immersive realities field. Especially Apple and Google, which are the dominant players in the smartphone industry, are both launching headsets this year. Apple has unveiled the Vision Pro, while Google has announced a mixed-reality headset that is being built together with Samsung and Qualcomm. Both Apple and Google already have very popular stores, they have full suits of tools (think about the Google Docs suite) and entertainment channels (think about Apple TV), and they have enormous expertise in building hardware and operating systems. They are entering this market now because they believe this is the beginning of the road towards mainstream adoption. And they are entering it to dominate it.

It is interesting that for what concerns Google, the “domination” may not just be about a headset, but for the first time about an open operating system. According to rumors, Google is already working on an Android XR operating system, with the idea of distributing it to all the interested headsets, probably together with its own store Google Play XR, trying to replicate on XR what it is already doing on mobile phones. This would be pretty disruptive, because up to now every standalone headset has been a walled garden with its own store and hardware, with the only exception being HTC trying to distribute its Viveport store to some other Chinese devices. Basically, Google could try to be for standalone devices what Steam has been for the PC VR ones. We are not sure it will work, because today most headsets earn their revenues through content, so for a hardware manufacturer using the store of another company may be an issue, but it is also true that we’ve seen over the years how it is difficult to build a new store without having very deep pockets (like Google has). Google may use its existing partnerships it has on the smartphone sector (e.g. with Xiaomi, Oppo, Samsung, etc…) to convince hardware manufacturers to create headsets powered by Android XR.

This is the moment of the truth for Meta. Mark Zuckerberg has been incredibly smart in entering this field before all the others, so now he already has a strong headset with a great content catalog around gaming. He knew that the only possibility he had of escaping the rules of Google and Apple was being a first mover, and he did a great job with the hardware it is building: in many comparison reviews, the Quest 3 comes out pretty well in a head-to-head against the Vision Pro. But now, Google and Apple are coming to the field with their enormous ecosystems and now we will truly see if Meta can be a relevant player in this market or if it will be crushed by its competitors. It will be a long battle and we won’t see the results this year for sure, because this competition is still indirect. Meta’s Quest 3 is an affordable device for gamers, while Apple Vision Pro is a very expensive device for professionals and technologists, so they operate on different markets for now. But this is the year where the battle starts. We have no idea how it will end, but my gut feeling is that Meta won’t disappear anytime soon, because it has spent too much money building its ecosystem and its expertise, and the products it is shipping are fairly good.

XR bounces back, but slowly

There’s a new optimism around XR: if all the major players are entering the field again, it means that something big is cooking. Probably they have calculated that this is the right moment to enter, the very beginning of the road to mainstream adoption. With Meta being a believer of “metaverse” for ages, Apple, Google, Samsung, Huawei, and Bytedance actively working on headsets, Microsoft still into XR thanks to Mesh and its partnership with Meta, NVIDIA’s CEO saying that “Omniverse” is the next big thing for his company after AI, the Rayban Stories smartglasses getting positive reviews and good sales… it seems that XR is more alive than ever. I feel there is more optimism in the field, and even investors have started to be interested in immersive realities again.

This is all good, but to avoid the usual emotional rollercoaster, I warn you that this new rise of XR will be quite slow. It is true that all the big names are in, but I doubt that throwing a bunch of standalone headsets worth $2000 or more will make XR mainstream overnight. We need to manage expectations realistically: this is the beginning of a new cycle that will last a few years. I have the gut feeling this cycle will be an important step forward for XR, and it will end with average people knowing more about XR and wondering if they need an XR headset, but the situation is evolving slowly. Consider that Google and Samsung are rumored to manufacture only 30,000 headsets this year, and Meta will respond with a premium headset probably only next year. So, if you have an XR company, enjoy the growing optimism, but keep playing safe.

Meta doubles down on the consumer market

There is a strong rumor about Meta launching a new affordable headset in the upcoming weeks. The rumor comes from multiple sources, and usually, when this is the case, the rumor represents reality. The new headset would be a lightweight version of Quest 3 (probably dubbed “Quest 3 Lite”) costing only $199. If it happens, it would strengthen even more the positioning of Meta in the consumer market. We just saw how well the Quest 2 performed during the holidays thanks to its $199 price… a more powerful headset able to run more compelling games for the same price can increase these numbers even more.

Again, I do not expect something like this to make VR skyrocket, but it may convince some people who were in doubt about buying a headset to buy it. For sure it will increase the number of people with a headset at home, so it can be another step forward for our ecosystem. And this is also a good moment for Meta to launch it, so it can consolidate its positions before Apple and Google arrive at targeting the average consumers, too.

Mixed Reality as a milestone for AR glasses

We are all seeing how every VR headset now has also mixed reality features. Apple doesn’t even focus much on VR, but it just talks about mixed reality (or “spatial computing”, as it loves to call it). The reason for this love for passthrough mixed reality is not the technology per se, but because it is a stepping stone toward AR glasses, the technology that will substitute smartphones. Consumer AR glasses are not here, and won’t be for many years because the technology is simply not ready. Meta may have a prototype to show this year, but the first release of a product is rumored for 2027. Apple has postponed indefinitely its plans. Companies can not defy the laws of physics, so we will have to wait a bit before having Futurama’s EyePhone in our eyes.

Passthrough mixed reality is instead already here, so companies are all launching passthrough headsets to prepare the field for future mixed reality glasses. During these years, they can earn market shares, gather content, and also get knowledge about mixed reality, so that later, when they will switch to real glasses, they have already a full ecosystem, and have just to make a smooth transition. I strongly believe that one of the reasons why Apple has chosen only eyes, voice, and hands as the interactions means for its glasses is because with future AR glasses people won’t take controllers with them all the time.

So I expect this year all companies will focus on creating passthrough mixed reality ecosystems that can in the future be translated easily to AR glasses. I think the more we go on, the more VR-only headsets will disappear unless they have a strong focus on gaming.

Experiments around MR and natural interactions

We are in the middle of a paradigm shift: before the focus was on VR, while now is on MR. Before the focus was on controllers-driven interactions, while now natural interactions (like hand tracking) are getting always more important. Even here, the launch of the Apple Vision Pro, with its strong focus on MR and hand tracking, is changing a bit the rules of the market. Many companies always follow the trends set by Apple, and the XR field is no exception to this rule.

Like during all paradigm shifts, there are no standards yet, and I have to say no killer app yet. While I hate the concept of the “Killer app”, because I think there are more “Killer ecosystems with many useful apps”, it is true that up to now I have no memory of a very successful mixed reality application. There are some nice MR games on Quest, and the Vision Pro has now more than 1000 native apps, but no one of these pieces of content seems very compelling to me. On Quest, there are some nice demos and games with MR features, but nothing more. On the Vision Pro, many of these apps are basically 2D screens. I think no one has cracked the formula of MR yet, but this is normal: it’s a new paradigm and people need time to find what is the “language” to be spoken in this new technology.

Natural interactions like hand tracking are being debated: some people still think that controllers are superior, while others are all-in with hand tracking. Apple has released this innovative interface with eye and hand tracking, but it is not always working as intended. Big companies need to make experiments with these new paradigms, too…

Different devices, different use cases

One important trend that I’m seeing is that now consumer-oriented headsets are getting different use cases. In the past, all headsets of the same category had similar form factors and use cases: gaming, fitness, and lightweight productivity. Now we have the Vision Pro which is strongly focused on productivity and media consumption, and the Quest which is all about gaming. They are two very different devices, and also the aforementioned interaction schemes are coherent with them. Quest is about gaming, and gaming in VR can not exist without controllers (at least for certain categories of games), while Vision Pro is all about media consumption, where controllers are just a nuisance, so it uses hand interactions. I think this rift (pun intended) is going to expand always more this year. We will see different devices, with different purposes, different prices, and different interaction schemas. This has always happened, but it was mostly an enterprise vs consumer division, while now I expect a bit more of fragmentation also on the consumer side.

We are seeing this also on the smartglasses side: there are smartglasses for media consumption (e.g. XREAL Air 2) and smartglasses for media creations (e.g. Rayban Stories) and they have different form factors. On the VR side, Cix Liv is working on an MR/VR headset super-optimized for fitness, while BigScreen released the Beyond headset specialized in media consumption, and Immersed should release the Visor with a strong focus on productivity. Niantic teased it is working on a reference design for AR glasses with Qualcomm, and they are specialized for outdoor usage (so people can play Pokemon Go XR on the streets), while most of the current AR glasses are dedicated to being used indoors.

I think there will be more and more options coming: for instance, the rumored Google headset could be priced in the middle between a Quest 3 and a Vision Pro, offering even more options to its users.

Tethered VR will stay as a niche

Tethered VR has been superseded by standalone VR: there are many more standalone VR users than PC VR ones. The PC community is very strong and dedicated, while the standalone one is more casual. But since Quest has outsold PCVR headsets by a wide factor, many developers are focusing on releasing their games for the Quest store, and the Steam porting of these games usually has potato quality for PCVR standards. For this reason, PCVR people are complaining about the lack of quality content for their devices. Luckily Praydog’s recent universal mod has brought a lot of fresh content to PCVR users, but these are not native VR games, so your experience with them may vary.

PSVR 2 could have been the salvation for the tethered sector: Sony released a cabled headset because it wanted to focus on the top quality for the visuals of its games, and this could have reinvigorated the whole sector of people focusing on content with top-notch graphics. The release of Horizon: Call Of The Mountain was very positive in this sense, but then the headset did not have the sales we all hoped for. Road To VR claimed that Quest outperformed PSVR 2 during the last holidays by a 30 to 1 factor, which is impressive, and shows that not even the most famous console can stop the triumphant march of standalone headsets.

I’m not going to say that PCVR is dying because I do not think it is true. It even just got the release of new headsets like Bigscreen Beyond. But it is going to stay a niche. While standalone devices will be for the masses, people wanting a VR experience of the best quality will focus on buying VR-compatible computers and high-end VR headsets. The new headsets for PCVR, like Bigscreen Beyond or the upcoming Somnium VR-1 are all going to be very expensive and this is another proof of the fact that PCVR has become a premium niche. So I think we will keep seeing some dedicated content, headsets, and accessories for PCVR, but they will all be limited in number and expensive. Luckily, the PCVR community will still stay strong.

XR and smartphones will still have a multifaceted relationship

AR glasses will replace smartphones, but this is not going to happen anytime soon. For now, the relation between XR and smartphones will still be complicated to define and very multifaceted:

• AR in the sense of “filters” or “virtual try-ons” is now a commodity on smartphones. It is taken for granted, and people do not even see it as something innovative. Platforms like Spark AR will continue releasing new features to give creators the power to do more compelling filters and filters are becoming pervasive everywhere. I even use filters in all my professional Teams calls…
• 6 DOF AR content on smartphones is sparse and not very successful. Yes, Pokemon Go has been a huge hit, but that was not an AR game, AR was just a feature of it. There is no “Killer App” for 6DOF AR on smartphones in my opinion and I don’t expect it to happen this year, either
• Smartphones are instead working very well to support smart glasses and even some 6 DOF glasses (like XREAL ones). Since all people have a powerful smartphone in their pockets and they will not separate from it, it is smarter for smart glasses manufacturers to make the glasses connect with it instead of trying to substitute it… something that won’t happen soon. Qualcomm’s Dual Render Fusion, which lets smartphone apps run as usual but also show additional AR content only if AR glasses are connected to the phone, is a good example in this sense
• Last but not least, it deserves a mention the fact that 3D tablets (like the one from Leia) and smartphones are also becoming a thing much more than before, and this is another good example of hybridization between the two technologies

I think that exactly as it is happening with the line between VR and MR, also the line between XR devices and phones is becoming always more blurred.

The growth of digital economies

The “metaverse” is dead for the press, but actually, it is more alive than ever. Virtual worlds like Fortnite, Roblox, or also RecRoom, and VRChat are becoming every day more successful. There are hundreds of millions of users in the “metaverse” every day, as Tim Sweeney reminds us. The metaverse platforms are growing and their internal economies are growing, too.

The metaverse is dead! Let's organize an online wake so that we 600,000,000 monthly active users in Fortnite, Minecraft, Roblox, PUBG Mobile, Sandbox, and VRChat can mourn its passing together in real-time 3D.https://t.co/tRpFHsdZLw

— Tim Sweeney (@TimSweeneyEpic) May 9, 2023

Back in the day, if you wanted to offer an experience to people, you had to create a full-fledged application from scratch, but now there is another possibility and it is the one of building an experience inside an existing platform. I know it very well since I helped craft many award-winning concerts inside VRChat myself.

Some creators make a very good living by selling their creations on these platforms: Dave Baszucki made the bold prediction that “in 5 years, a Roblox developer will be valued at $1 billion”. 1 billion is an enormous amount of money, and until a few years ago we didn’t even think was possible that a developer leveraging an existing platform could think about reaching that amount. But now that prediction at least makes some sense, and this shows how the virtual worlds market is growing, both in number of users and in the size of its economy, and this trend is continuing this year for sure. In the VR space, for instance, VRChat going mobile is increasing a lot of its user count.

Some governments support the metaverse

Some governments are already putting the “metaverse” among their strategic priorities, even if now the hype has deflated. In this blog, I told you about the Finnish Metaverse Strategy, but I’ve heard that also The Netherlands and Korea are bullish on this new technology. If a state supports a technology, it means that there will be money invested in the sector, plus support for the companies of the space, and increased communication towards the citizens. That’s a very good thing. Probably this year we will see other countries taking inspiration from the ones mentioned above, and announcing their own metaverse strategies: I’m pretty sure that the release of the Apple Vision Pro is going to move some governments toward this decision.

Contextual AI is on the rise

Couldn’t I write a tech article in 2024 without mentioning AI? Of course not. Artificial Intelligence and immersive realities are a match made in heaven, and already many companies are mixing the two: Meta’s communication is all about either artificial intelligence or the metaverse, or a combination of the two; NVIDIA is another powerhouse of both technologies and with Omniverse is making good use of both. AI is actually used everywhere in VR, from full body tracking to frame reprojection, not to mention depth correction in passthrough, so there’s no way that these technologies go on separately. In 2024 there will be even more co-use of AI and XR.

A trend that I imagine rising in 2024 is the one of “contextual AI” that Meta has already announced for its Rayban Meta glasses: headsets and glasses will be able to see what you see and react to your queries accordingly. I’m sure this will be a big trend because this is what in the end we want from AR glasses. We want to walk on the streets in a foreign city and just say “Tell me where to eat tonight”, and let our virtual assistant mix the info about our past opinions on restaurants, the location we are in, our usual budget, etc… using some AI magic and give us one to three optimal options to choose from. We do not want to say “Hey glasses, I’m in Paris, in this street, I have this budget, usually I like this food blablabla” or enter the info in a webpage form. Even better, when we have a problem, for instance, because the car is broken, the AI should help us tell how to fix it by asking us to give it the minimum info that it needs. RayBan Meta started implementing this, and I’m pretty sure that over the years all glasses and headsets will implement something like that. I don’t expect all glasses to implement contextual AI in 2024, but for sure I expect this trend to grow.

Of course, while I am super-enthusiastic about the potentialities of this technology, I am also very scared by the problems with privacy and manipulation. An AI that is always on would send all my profilation data to the company managing it, and I’m 100% sure it would be sold to the highest bidder. Kavya Pearlman and the whole XRSI are working hard to prevent this from happening, but it’s an uphill battle. Plus no one guarantees that the AI is unbiased: like the first results on Google are the sponsored ones, the first suggestions from the AI assistant may be of a company that has paid to be suggested, or even worse, they may be the result of some evil manipulation. Louis Rosenberg is constantly warning us about these risks we should be aware of.

AI as a support for content creation

These months I have heard everywhere that AI will substitute my job and the job of everyone else. And that a team of a few people will be able to make an AAA game just thanks to Generative AI and make billions. But my practical experience with it showed something else. While I don’t underestimate the disruptive importance of generative AI, I think the technology is still unripe to create content independently. When I tried to use ChatGPT to create Unity scripts, it managed to write something good, but often I had to correct it. When the artists created 3D assets with AI, the results were absolutely not optimized for XR usage. Plus it’s hard for AI to keep memory of all the work that a team is doing and create work that is coherent with the work of everyone, so it has limited use for now.

What AI is amazing to do today is SUPPORT content creation. The artists I work with use AI as a source of inspiration for some projects they have to do. I use GitHub Copilot to have AI-powered suggestions while I write the code, and I noticed a speed-up in my work of around 15%. AI is very good for increasing the productivity of a team.

Because of this, Generative AI is also being integrated directly into the production tools: Unity has in beta the integration of AI in its engine, so it is possible to use it to facilitate the creation of content without having to switch to another tool. Photoshop is doing the same. Copilot works inside Rider and Visual Studio. All the tools are integrating AI in one way or another.

Many see Generative AI as the way people will create content for the “metaverse”. While I think that long-term this is true, I also believe that short-term, AI will be a support for people creating content for the metaverse… but we will still need professional figures in 2024 to create virtual worlds.

Cross Platform Tools

The number of headsets is increasing, but it’s refreshing to see that there are tools and libraries that let you develop your content once and deploy it on all the devices. I’m a Unity developer, and for instance, I use a lot the Unity XR Interaction Toolkit and the features of the new Unity Input System to develop content for all the headsets using just one Unity project. Also, OpenXR is a standard that is helping a lot in standardizing our industry.

I was very surprised to see that also Apple somewhat adhered to these standards: when you develop VR content for the Vision Pro (sorry if I called it “VR” and not “fully immersive spatial computing compliant with Steve Jobs’ will”), you use exactly the Unity XR Interaction Toolkit. This means that theoretically, you can make a VR application working for the Quest and the Vision Pro at the same time.

Cross Platform Tools are an amazing thing for us developers and I foresee they will be expanded this year to work also with the new upcoming headsets powered by Google.

WebXR and its slow growth

I’m a big believer in the potentialities of WebXR, but it’s a technology that has been an “eternal promise” for a decade. Probably the reason is that big companies like Meta and Apple are more interested in growing their profitable stores than a technology whose content they do not control. It’s a pity because WebXR lets you create content compatible with all the headsets out of the box, including the Vision Pro.

The death of Mozilla Hubs has not been good news for the field, but luckily there are some new cool virtual world projects running on the web, like Frame VR. And there are cool people supporting the technology: in a few days there are the WebXR Awards, for instance, and I suggest you all attend them.

Overall I see WebXR keeping its slow growth, but unluckily it will still be a niche.

XR in the enterprise is now consolidated

The XR use in the enterprise sector is now consolidated. XR is already used everywhere for training, remote assistance, and prototyping, and there is not much to say about it. Microsoft and NVIDIA are increasing their investments in the “industrial metaverse” which mixes immersive realities, AI, and digital twins to support companies in making better planning and more efficient execution. This trend will continue this year for sure, and it will see even more integration of technologies like AI, XR, 5G, Cloud, digital twins, etc…

What about the metaverse associations?

More than a prediction, this is a question: what are all the metaverse associations doing? Are they producing some results? A lot of entities have been created to facilitate the creation of standards for the metaverse, but I saw nothing coming out of it if not a lot of meetings. The Khronos Group was very practical in the creation of OpenXR, and even if it took some time to be delivered, it brought great advantage to the whole ecosystem. I wonder if these Metaverse Associations and Metaverse Forums have a similar practical goal in mind, or are only there to foster collaborations between people…

You will support this blog

For sure in 2024, you will read this blog, which provides useful information to the XR community, and share its content on social media. You will register to its newsletter and donate something to its creator on Patreon. And I can reliably predict that I will be very thankful to you 🙂

Enjoy XR this year and also tell me… what are YOUR predictions for 2024?

(Header image by Qualcomm)