The XR Week Peek (2022.09.26): Pico 4 announced, RTX40 cards revealed, and more!

This is the first newsletter of the Autumn! It is not my favorite season of the year, but it is going to bring with it many cool pieces of VR news, and many cups of hot chocolate, so I guess I will be very happy in the upcoming weeks…

Top news of the week

Bytedance launches Pico 4, directly challenging Meta Quest 2

The announcement of the Pico 4, the first consumer-oriented headset by Bytedance, has been much more disruptive than I expected. With it, Bytedance directly challenges Meta and the Quest 2 with a product that is more powerful, cheaper, and offers a quite good content catalog.
 
 Let’s see some of the specifications:

• Chipset: Qualcomm Snapdragon XR2, 8 Cores 64 bit, 2.84GHz
• RAM: 8GB LPDDR4X, 2133MHz
• Storage: 128GB/256GB, UFS 3.0
• Display
• Type: 2.56″ x2 LCD display
• Resolution: 4,320 x 2,160 (2,160 x 2,160 per eye)
• PPI: 1200
• Refresh rate: 72Hz / 90Hz
• Field of view (FOV): 105° Diagonal
• IPD: Support stepless adjustment with a range of 62～72mm
• Cameras: Fisheye Cameras x 4, 16MP RGB Camera x 1

As you can see, the chipset is the same as the Quest 2, but it is not underclocked, so it is more powerful. The RAM is bigger than on the Quest, and the display is very crisp, with a 1200 PPI screen that makes sure that the screen door effect is absolutely not visible. The device also features an RGB camera, so that to provide color passthrough AR. The headset is also very small and lightweight thanks to the use of pancake lenses. Some hardware accessories have been announced for this headset, like a fitness 3DOF tracker for the feet, and a dedicated router for wireless streaming.
 
 On the content side, Pico is going to bring in many famous titles already released on Quest, like Demeo and Walkabout Mini Golf, not to mention Rec Room and Virtual Desktop. It is also planning to have important future releases like The Walking Dead: Saint and Sinners — Chapter 2. Pico is also going to have its first exclusive title: Just Dance VR, a popular game good for all ages. And it’s not over on the content side: they are going to launch the Pico Video app with 600+ videos on day 1, and are partnering to have immersive concerts with Wave. Pico Worlds will be the official social VR application. The company is also starting various initiatives to support content creators, including a $12M fund for developers.
 
 All this package is going to cost only $429 for the 128GB version, and $499 for the 256GB one. This device is more advanced than the Quest 2, and it costs less. Meta has still a better ecosystem, but Pico is running fast in trying to improve its one with all the initiatives described above. I think that for the first time we have really a device that can be more interesting than a Quest for certain categories of people. It is impressive, and I think that Meta has to answer fast if it doesn’t want to lose its market leadership. Pico 4 is going to launch initially only in selected countries in Europe and Asia, so Meta can keep its leadership in the US, but the situation may change fast, so it’s better for Meta to tease a Quest 3 pretty soon.
 
 Regarding the Pico 4 Pro, the company has confirmed it is going to come in the future, and it will feature eye and face tracking. More info will be shared at AWE EU in Lisbon, on October, 20th.
 
 Finally, some competition in the consumer VR space. I am so happy about it.
 
 (Thanks to /u/suozhe for sending me the link to Pico 4 manual)

More info (All you need to know about Pico 4 and the announcements of the keynote)
More info (Pico 4 hands on by Upload VR)
More info (Pico 4 hands on by MRTV)
More info (Pico 4 hands on by one of the devs having early access to the device)
More info (Pico 4 vs Quest 2 features)
More info (Upload VR’s editorial about Pico 4 being what we were expecting from a Quest 3)
More info (Virtual Desktop coming to Pico 4)
More info (Pico 4 may not have 3.5mm jack for audio)
More info (Manual of Pico 4 Pro where you can see the face tracking sensors)

Other relevant news

At GTC, NVIDIA introduces new RTX40 graphics cards and much more

The annual GTC event for NVIDIA has been a blast. Doing its keynote on video, CEO Jensen Huang introduced the new RTX4080 and RTX4090 graphics cards, which are based on the new Ada Lovelace architecture. As usual, they are “the best cards ever released”.
 
 This is a small chart that summarizes the characteristics of the new cards, courtesy of Upload VR:
 Card CUDA Cores VRAM TFLOPS MSRP RTX 4080 (12GB) 7680 12GB GDDR6X 40 $899 RTX 4080 (16GB) 9728 16GB GDDR6X 49 $1199 RTX 4090 16384 24GB GDDR6X 83 $1599 
 The new generation of graphics cards features at least 2x the performance of the previous ones. They also bring with them updated ray tracing (with around 2x the performances), increased AI capabilities, and DLSS 3, which is able to generate synthetic frames to double the output framerate, and better encoding and decoding of videos.
 
 They are more expensive than the RTX30 counterparts, but considering the current high inflation levels, this was to be expected.
 
 But the new amazing RTX40 cards (which we are anyway too poor to afford) were not the only news of GTC, which was full of interesting announcements, like for instance:

• A new generation of OVX Server powered by the Ada Lovelace architecture, which is meant to create the backend of the metaverse, by delivering groundbreaking real-time graphics, AI, and digital twin simulation capabilities.
• The new DRIVE Thor superchip aimed at autonomous vehicles, which companies will be able to use from 2025
• Omniverse Cloud, which lets companies use Omniverse completely via cloud rendering even on non-powerful machines
• A connector to let you use Omniverse with Unity. So now on the same Omniverse project, you can have people using both Unity and Unreal Engine
• Some interesting enterprise use cases for Omniverse, including one made in partnership with Lowe and Magic Leap
• A lot of new tools for AI and ML, including a new system to generate diverse 3D objects to populate virtual worlds
• Jetson Orin Nano, a tiny robotics computer that is 80x faster than the previous super-popular Jetson Nano
• Omniverse Create XR, which lets you navigate Omniverse scenes in AR and VR, and even modify them and interact with them
• NVIDIA RTX Remix, which lets you automatically implement RTX and AI supersampling into old games, so they make a sudden jump in visual quality. I invite you all to see the video I’m linking below to see how this can be cool to give a new life to many old games. During the keynote has been also shown Valve’s Portal transformed into an RTX experience (and published for the community to enjoy).

It’s been a very interesting GTC. It has shown once more how NVIDIA is shifting from a company producing graphics cards for personal computers to a powerhouse of cloud services, for everything that concerns 3D graphics and AI.
 
 (Disclaimer: I had a partnership with NVIDIA to promote the GTC, but now that the GTC is over, this partnership has finished, so I included this news in the roundup just because I think it deserved that)

More info (NVIDIA GTC Keynote)
More info (Official summary of GTC Keynote on NVIDIA blogs)
More info (NVIDIA RTX40 graphics cards)
More info (Unity connector for Omniverse)
More info (Omniverse Create XR)
More info (NVIDIA RTX Remix video)
More info (Portal RTX)
More info (NVIDIA Omniverse use case with Lowe’s and Magic Leap)
More info (3D generative AI research)
More info (Drive Thor superchip)
More info (Omniverse Cloud)
More info (OVX Server)
More info (NVIDIA Blogs link, containing all news about GTC)

Bonelab is coming on Thursday

With a surprise “date reveal” trailer, Stress Level Zero has announced that one of the most awaited games of the last years, Bonelab, is coming on Thursday, September, 29th for $39.99. The game, which is the successor of Boneworks thought with the Quest 2 in mind, promises an action game with physical interactions working very well on standalone devices. The trailer shows also the best insight we had until now of the gameplay of the game, and one thing got all of us interested: there is a mechanic that lets you change your avatar, and this will let you solve some puzzles of the game. In the trailer, it is visible that the player changes his avatar so that to become taller and able to grasp an object.
 
 Given the attention that Blade and Sorcery: Nomad is getting on Quest, it is very possible that also Bonelab can become a hit for the Quest Store. The game will also be released on Steam so that to please the current PCVR fanbase of Stress Level Zero.
 
 I’m pretty excited about this. And you?

More info (Release date announcement)
More info (Gameplay on Quest and price reveal)

The 8th Wall brings VPS to WebAR

The 8th Wall is revolutionizing WebAR, and after the acquisition by Niantic, it is growing even faster than before. The great news from this week is that the company has brought Niantic Lightship VPS into its engine, so now it is possible to use VPS from a web AR application.
 
 This means that you can select one of the VPS locations made available from Niantic (there will be 100,000 worldwide at launch), and your AR application will be able to work exactly in that location, showing the AR elements in that place with great accuracy. So if you decided to put a virtual smiling face on a stone close to the Eiffel Tower, everyone going to the Eiffel Tower can see the augmentations you put, exactly in the point you put it. All of this, in WebAR.
 
 This opens up lots of possibilities because you can augment a particular place (e.g. to make a game to incentivize tourism) with an AR application that doesn’t need to be installed but works directly on the browser, causing a very small friction to the user. And even more, the system is able to also import the mesh of the place, so the augmentations have also full occlusion by real elements. I think it’s simply incredible, and I can’t wait to see what content creators will be able to do with it. I envision lots of possible applications in marketing, tourism, and gaming.
 
 The only problem with The 8th Wall is that it is quite expensive, but apart from it, it is crazy good, and I’m amazed at what incredible features they are going to release every few months.

More info

News worth a mention

Meta and Google announce layoffs

The bad economic conditions are not changing, and so big tech companies are keeping doing layoffs. This week we had the announcement of both Meta and Google laying off people. Both companies are anyway offering the fired people the opportunity to find a new position inside the company. I guess this is going to slow down the metaverse efforts of these companies.

More info

Rec Room announces Rec Room Studio

Rec Room is one of the most used social VR platforms. It has just announced that it will release a new tool to create virtual worlds for the platform based on Unity. It will be called Rec Room Studio and “will bring top quality graphics and animations to the fingertips of the creator community,” according to the startup.

More info

Meta to bring multiplayer to homes in Horizon Worlds

Meta has announced that soon Horizon Worlds players will be able to invite their friends to their home in Horizon Worlds. It is fun because this means that you can have a multiplayer home in Horizon Worlds and also a multiplayer Horizon Home on your same device. If you are confused, well, we all are. It seems that Meta is having a very confused social strategy for what concerns the Horizon ecosystem and is trying different things to see what sticks (probably no one of the above).
 
 On a separate note, the Quest 2 has just been updated to Runtime v44, which brings additional parental control tools, plus finally many customization options to record videos on the device. This is great news for us content creators.

More info (Multiplayer Horizon Worlds home)
More info (Meta Quest runtime v44)

Some interesting XR research project

Let me tell you about some interesting experimental projects happening in the XR space:

• Google has published a blog post about its research with a NeRF algorithm able to reconstruct the images of an object or animal from different points of view starting from a single picture
• Meta has shown its internal prototype that does full body reconstruction of the pose of the user using only the tracking cameras of the Quest 2. The reconstructed pose is never perfect and the system has quite a high latency, but it is very cool nonetheless
• Another research by Meta shows how the company is able to reconstruct the video flow of a person from different points of view. So, for instance, you record a video of a person from a low position and then see how it looks from a higher position
• Apple has just published a patent about showing invisible things on mixed reality glasses. This is actually quite a cool concept: if you for instance install a soundwaves detector on your glasses, you can transform the information it gets to visual shapes, so through your MR glasses you can actually see the soundwaves propagating from a guitar you are playing. The same concept could work with a Wi-Fi sensor, so you could see the strength of the Wi-Fi waves around you

More info (Google NeRF algorithm)
More info (Quest 2 full body tracking)
More info (Meta research on video points of view)
More info (Apple patent for mixed reality)

Some news on the metaverse

Let’s talk about the M-word:

• JingDaily has published a very long and detailed article about the ecosystem of virtual idols in China
• Decentraland has just created a marketplace for the emotes of avatars. I had never thought that also emote animations could be sold
• Epic has invested $30M in Hadean, which works in creating the infrastructure for large-scale multiplayer experiences
• I found an interesting article about what could be some future frauds in the metaverse, like having someone impersonating your digital avatar and doing something nasty. The article also reports an interesting finding: according to a study, people find more trustworthy the faces of AI created people, instead of the ones of real people
• Bloomberg has dedicated a long article to the figure of the Chief Metaverse Officer. It reports that CMO may earn up to $1.5M/year to help big corporates in metaverse efforts. I have chosen the wrong job, I think
• Lamina1, the blockchain company cofounded by Neal Stephenson, has just released its whitepaper in which highlights the values of openness of the metaverse the company wants to build. It will also work on a blockchain that is as green as possible.

More info (Asian virtual idols)
More info (Decentraland emotes marketplace)
More info (Epic invests in Hadean Labs)
More info (Frauds in the metaverse)
More info (What is a Chief Metaverse Officer)
More info (Lamina1 whitepaper)

Some news on content

• Ben Lang from Road To VR went hands-on with Resident Evil Village on PSVR 2 and loved its high-quality environments full of details
• Guardians Frontline, an upcoming strategic VR shooter based on the Guardians experience on App Lab is releasing next year for Quest and Steam VR
• Fast Travel Games revealed a new VR roguelite set in the universe of Stellaris. It is called Ghost Signal: A Stellaris Game, and is slated to arrive on Meta Quest 2 in early 2023
• Upload VR went hands-on with Dyschronia: Chronos Alternate — Episode 1 and appreciated its immersive gameplay and intriguing plot
• An unexpected VR version of the Bafta-nominated puzzle adventure Call of the Sea is coming next year for Quest

More info (Resident Evil Village)
More info (Guardians Frontline)
More info (Ghost Signal: A Stellaris Game)
More info (Dyschronia)
More info (Call Of The Sea)

Other news

Meta has published the full list of talks for Meta Connect. There will be also a way to enjoy the conference in Horizon Worlds

Learn more (Meta Connect program)
Learn more (Connect on Meta Horizon)

If you are a VR storyteller, this 10,000€ grant may be interesting for you

Learn more

I just discovered that there is someone shooting amazing photographs in VRChat

Learn more

A very cool video that shows the power of AR in the enterprise sector

Learn more

News from partners (and friends)

Support an indie

I’ve no news for this section this week, so let me just invite you all to head to SideQuest and support some VR indie creator! It’s very hard to be an indie developer, so please find what looks like a good indie game, and download it, post a comment, give it a high review, share about it online, buy it… you decide what to do, but do something, because every engagement action is a lot for us indie creators, trust me. Thank you for supporting our community!
Go to Sidequest and support an indie creator!

Some XR fun

#truestory about the metaverse
Funny link

When virtual reality has the wrong kind of haptics…
Funny link

If you crop the video of the full body tracking on Quest, well, the results are hilarious!
Funny link

Pico Neo 3 Link guardian in a nutshell
Funny link

Donate for good

Like last week, also this week in this final paragraph I won’t ask you to donate for my blog, but to the poor people that are facing the consequences of the war. Please donate to the Red Cross to handle the current humanitarian situation in Ukraine. I will leave you the link to do that below.
 
Let me take a moment before to thank anyway all my Patreon donors for the support they give to me:

• Alex Gonzalez VR
• DeoVR
• GenVR
• Eduardo Siman
• Jonn Fredericks
• Jean-Marc Duyckaerts
• Reynaldo T Zabala
• Richard Penny
• Terry xR. Schussler
• Ilias Kapouranis
• Paolo Leoncini
• Immersive.international
• Nikk Mitchell and the great FXG team
• Jake Rubin
• Alexis Huille
• Raghu Bathina
• Chris Koomen
• Cognitive3D
• Jennifer Granger
• Jason Moore
• Steve Biggs
• Julio Cesar Bolivar
• Jan Schroeder
• Kai Curtis
• Francesco Strada
• Sikaar Keita
• Ramin Assadollahi
• Juan Sotelo
• Andrew Sheldon
• Chris Madsen
• Horacio Torrendell
• Andrew Deutsch
• Fabien Benetou
• Tatiana Kartashova
• Marco “BeyondTheCastle” Arena
• Eloi Gerard
• Adam Boyd
• Jeremy Dalton
• Joel Ward
• Alex P
• Lynn Eades
• Donald P
• Casie Lane
• Catherine Henry
• Qcreator
• Ristband (Anne McKinnon & Roman Rappak)
• Stephen Robnett
• Sb
• Enrico Poli
• Vooiage Technologies
• Caroline
• Liam James O’Malley
• Hillary Charnas
• Wil Stevens
• Brian Peiris
• Francesco Salizzoni
• Alan Smithson
• Steve R
• Brentwahn
• Pieter Siekerman
• Simplex
• Matias Nassi

And now here you are the link to donate:

Support The Red Cross in Ukraine

(Header image by Pico)