The XR Week Peek (2021.11.15): Vive Focus 3 aims at LBEVR market, Niantic and Qualcomm launch AR SDKs, and more!
Hello everyone from Palo Alto, California! I’m still here in the US for some days after having attended AWE, planning to meet some interesting people before heading back to Italy. Until now my trip has been amazing, and I have met so many cool people of the community and have tried so many XR devices! You can read some of my hands-on articles on my blog (e.g. this one about Holoride and this about Lynx-R1), but many will come in the following days!
After I will be back in Italy, on the 20th, I will also be a speaker at Virtual Reality Day, the 24h-conference about VR held by Bob Fine. Be sure to tune in if you want to participate in this interesting event!
Top news of the week
The new updates to Vive Focus 3 may revolution LBEVR
In a side event happening after AWE, HTC has announced a bunch of updates of Vive Focus 3 that have the potential of revolutionizing the Location Based Entertainment in VR (LBEVR o LBVR) sector.
With the upcoming update of the runtime, Vive Focus 3 will be able to track its position in a space up to 33m x 33m, that is 1000 square meters. This is the biggest tracking area among the standalone headsets on the market, and while it is totally useless at home, having such a big area is important in entertainment locations because it lets players move around a big space to play their games. You may argue that with the Quest 2 you can remove the Guardian and track an infinite space, but if you remove the Guardian, the tracking is prone to drift, that is if you set a mapping between the real and the virtual space, after various minutes of play this relationship gets broken because the tracking system doesn’t have anymore an area that can use as a reference system. With Focus 3, inside this 33m x 33m area the tracking is stable, that is the tracking region doesn’t move over time, and this guarantees the safety of the user.
What is even more important is that with the new Focus 3 update, different headsets can share the same area: that is multiple players can play in the same space using the same reference system. This means that multiple players can enjoy co-location, playing in the same space, with every player that can see exactly what is the real and virtual position of the other players. Together with the area update, this means for instance that two teams of four people can play inside a warehouse a laser tag game in VR.
Even better, the Focus 3 now supports Wi-fi 6E, the new Wi-fi protocol that offers a bigger bandwidth and fewer interferences. It also supports integration with NVIDIA CloudXR, for remote rendering via an on-premise or cloud servers. Thanks to this, it is possible to show on the standalone device the same graphical quality of a VR-ready PC. And there is even more: the headset can be calibrated inside a SteamVR-tracked area so that it can also integrate with props tracked with Vive/Tundra Trackers, which are used a lot inside VR arcades.
All of these features together mean that LBEVR venues won’t have any more to use PC backpacks and tethered headsets, and they won’t have to use expensive external tracking solutions like Vicon, but they have just to buy Vive Focus 3 devices. The onboarding of the players will be much easier, the maintenance costs lower, and the fun of the players guaranteed. I have worked on a similar solution last year, and I can assure you that colocation with standalone headsets is a pure blast. Until now, it required the developers to create ad-hoc solutions, while now HTC is offering this functionality out of the box, so many more activities will be able to offer it.
And if the users are not satisfied with the graphical quality provided by the headset, the venue may implement remote rendering to solve the issue. This could offer multiplayer colocated games with the same quality of PCVR games.
We are in a moment where LBEVR is growing again after the COVID, and this new update of the Focus 3 may make sure that many venues cut the costs by using just a standalone headset to offer their services. This can help these activities in being more profitable and offering a better service to their customers. This can be a little revolution for VR arcades, that probably will all abandon PCVR headsets and switch to Focus 3 to offer multiplayer virtual reality experiences that will be similar to laser-tag in VR.
More info (New features of Vive Focus 3)
More info (Vive Focus 3 being compatible with NVIDIA CloudXR)
Other relevant news
Niantic launches Lightship SDK
Niantic has just launched its Lightship SDK to offer every developer the same toolset Niantic is using to build its geolocalized AR experiences like Pokemon Go.
The SDK offers many interesting features to developers, like:
- Tracking and mapping for AR
- Real/virtual objects occlusion
- Shared anchors
- Local multiplayer (up to 5 people for now)
- Environment understanding (the system can detect if you are pointing at the sky, grass, buildings, etc…)
and it already has a roadmap for further updates for the next months. It is basically an SDK to build outdoor shared experiences that exploit the AR Cloud. And what is amazing is that it is free to start with, and it becomes paid only if you go beyond 50K users. So if you are not successful, you don’t have to pay.
I love the features of this SDK, I love that it is free, and I love that Niantic doesn’t want to build a closed ecosystem out of it. What puzzles me is who is going to use it TODAY. Because the potential is huge for the future, but up to now, mobile AR games never skyrocketed (not to say they sucked completely), and in fact, Pokemon Go remains the only successful title of this kind. Very few developers are still betting on it. Personally, I see the current status of the SDK as something interesting for developers and research centers to play with and make experiments, but I think that it will become truly useful only when AR glasses will be more widespread.
If you want to use it for your project, you have a good incentive, though: Niantic has opened up a $20M fund for startups that want to build products using the Lightship SDK. I think this fund is a smart idea: with it, Niantic is pushing companies in designing projects that could exploit mobile AR in a new way, and maybe across these ones there is something that can become successful as Pokemon Go
More info (Niantic Lightship SDK)
More info (Niantic and its new SDK)
Qualcomm announces Spaces SDK
Qualcomm has announced Spaces. It is not a new kind of chipset or reference design for XR headsets, but it is a software solution to offer facilities to developers of AR applications, especially the ones built for AR headsets tethered to the smartphone.
Spaces offers functionalities in:
- Image tracking
- Object tracking
- Hand tracking
- Spatial mapping
- Environment understanding and meshing
- Plane detection
- Real/virtual world occlusion
- …and more
Long story short, it is an SDK to help developers in building applications for tethered AR headsets. To build it, Qualcomm is also exploiting the expertise gathered by acquiring AR framework Wikitude and hand tracking system Clay Air (whose acquisition has just been announced).
The announcement also comes in a timely manner: Lightship by Niantic can work together with Spaces because Lightship is more focused on outdoor games and Spaces on indoor applications.
As a developer, I am happy that there will be an SDK that will let me develop once and deploy to all (or at least many) tethered AR glasses, which up to now had all their own dedicated SDK. It also has a good set of features, that will let me create rich experiences for these devices. And I’m not the only one being happy: Resolution Games has announced that it is already building AR games using Qualcomm Spaces.
8th Wall releases Reality Engine
8th Wall is probably the best paid engine for building WebAR solutions, and it has just become even better with the just-announced Reality Engine.
Reality Engine is a total rewrite of the 8th Wall framework so that you can now create a mobile web AR experience and have it working out of the box on all possible devices. Before, when you created a mobile AR experience for the web and you tried to open it on a PC or your VR headset, it was not working at all. Now, with Reality Engine, it is instead possible to open it on all devices, with the runtime taking care of translating all the visualization modes and all the interactions so that they become the ideal ones for the platform you are running it on. So for instance, if you have developed an AR experience where you tap on the screen to make something happen, on PC you click on the screen to make that thing happen, and on VR headsets you point and press the trigger with your controllers. All of this with the developer only taking care of the mobile AR platform, and not writing a single line of code for the other ones.
These are the five pillars or Reality Engine (copy-pasted from its website):
- Reality Application Runtime — 8th Wall’s award-winning augmented reality application framework has been rebuilt to respond to every device type.
- Spatialized UI — Automatic translation of 2D web UI elements onto a spatial control panel when an experience is accessed on a VR or AR headset.
- Interaction Mapping — Universal mapping of gestures to handle input across devices including controllers and hand tracking.
- Environment Mapping — Instant application of an environment to ground 3D content in space when the experience is accessed on computers and VR devices.
- Responsive Scale — Accommodating user position by making 3D content comfortable and accessible while ensuring the developer’s vision is consistent across devices.
Long story short: you create an AR experience for mobile, and magically it works with the best UX possible on all devices that have a browser. This is mind-blowing and can accelerate the adoption of WebXR.
More info (8th Wall announced Reality Engine)
More info (My description and hands-on with Reality Engine)
NVIDIA adds new features to Omniverse
At its GTC event, NVIDIA has announced a set of new features for its Omniverse engine, which is becoming always more powerful.
The most intriguing of these new features is the introduction of Omniverse Avatars. Omniverse Avatars are interactive AI avatars that combine technologies in speech AI, computer vision, natural language understanding, recommendation engines, and simulation. That is, you can create an avatar that can talk a bit like a real person.
NVIDIA CEO Jensen Huang has demoed the technology by showing a mini version of him that could answer the questions of the journalists. It is interesting how the company was able to create an avatar that resembled a cartoonish version of the real Jensen. Another interesting demo showed a virtual avatar that was able to talk with two customers of a restaurant and suggest to them what to eat with a natural conversation, all in an automated way. This can be a fantastic feature for concierge services for hotels, restaurants, or similar structures (it reminds a bit of what Amazon Sumerian tried to offer some months ago, and that my friend Eloi exploited well). But it can also be very interesting to create virtual idols and other kinds of conversational characters for entertainment purposes.
More info (All announced updates to Omniverse)
More info (Omniverse Avatar technology)
Decagear headset is going to pivot
Megadodo simulation has announced a hard pivot for its Decagear headset. The company promised a headset including many dream features for a dream price (around $450), and now it is waking up and realizing that reality is a bit different from dreams.
The company blames the chip shortage and says in a blog post that the headset now can’t cost less than $700, plus it must be delayed until the end of 2022. Since this was not the original goal, it now aims at making partners build the device, and the company will work on creating a sort of reference design, plus a runtime called “DecaBrick OS” that is not clear what should it be for.
I’ve always warned you against believing the claims of this startup because what they wanted to build was not possible for such a small company. I want to believe in their good faith, so probably it’s just naiveness on their side. Regarding this new path, I invite you all to wait for the details that should be revealed next week before judging if it can be interesting or reliable or not.
News worth a mention
Quest 2 now features ASW and passthrough MR apps
Meta/Facebook/Oculus/Whatever has just released the new version of the SDK for Oculus Quest 2 and in it there are two important features:
- Application SpaceWarp is now available. This means that an application built with the Quest may have +70% of computational power per rendered frame. This can improve the graphical quality of Quest apps
- Passthrough AR and MR apps are finally accepted on the Oculus Store and in App Lab. My friends at Crazy Kung Fu have already updated the application, and also some devs are experimenting already with passthrough, with one of them creating a cool horror app that makes you see ghosts inside your house. I mean, real ghosts, not me.
More info (ASW now available)
More info (Documentation about passthrough on Quest)
More info (Crazy Kung Fu in passthrough AR)
More info (Little horror game in passthrough AR)
Unity has bought Weta Digital for 1.6B
Unity has just bought Weta Digital for 1.6B. This is a strategic acquisition so that Unity can integrate in its offering all the tools that Weta has developed over time that regard the creation and management of 3D assets and special FX.
The short-term goal is to provide better tools to professional video creators, so Unity can compete better with Unreal Engine, which is already used in Hollywood productions. The long-term vision is that these tools may be useful to help everyone create and modify animated 3D elements to use in our shared mixed reality future. In the future, when we all will live in MR, all the users will need an easy way to create and manage 3D content (like today in our 2D tech world we have tools to easily shoot pictures), and Weta tools may be useful in this sense.
A Chinese startup has unveiled the prototype of a very lightweight headset
The Chinese start-up EM3 has just unveiled Ether, the prototype of a VR headset that is small and lightweight almost like standard sunglasses. The glasses are 6.8mm thick, weigh less than 35 grams and are fashionable. Of course, to have that dimension, they need to be tethered to an external unit (e.g. the smartphone) to have electrical and computational power.
Ether features a micro-OLED display with a resolution of 2,500×2,500 pixels per eye and a field of view of 90 degrees. All of this sounds very interesting, and I hope that the company can bring this from the prototype to the product stage because I would love to test it.
Google to put all its experimental projects inside “Labs” division
Facebook is not the only company going under a restructuring operation. Alphabet is reportedly creating a new unity, dubbed “Labs” to include all its experimental projects, including AR and VR solutions, and its incubator. The new unit, led by Clay Bavor, with report directly to Google CEO Sundar Pichai.
I wonder if this means that AR and VR are becoming relevant again at Google, with the CEO caring directly about the growth of the new Labs unit. Someone spotted that Google has recently updated something in Google Earth VR, so it may be…
More info (Google Labs)
More info (Google Earth VR updated)
More info (Someone jokes about this new unit)
Sandbox gets $37M investment round
Sandbox is getting a new $37M investment round. This is the proof that with the COVID situation improving, the LBEVR sector may thrive again. Sandbox CEO has also shared an interesting post on Medium detailing these two complicated years for the company and how it managed to survive.
More info (New investment round for Sandbox VR)
More info (Steve Zhao’s Medium Post)
Senso is a new glove for SteamVR
Senso has just released its latest device: called Senso Gloves, it is a pair of haptic gloves for virtual reality. The peculiarity is that they are compatible with SteamVR and have been built so that the user can use them instead of the controllers in every SteamVR game. This means that all VR content already available on Steam is compatible with them (but being compatible doesn’t mean that it is not uncomfortable using the gloves as the input device). The price for this device is $1000.
Half Dive headset to launch its Kickstarter campaign
Some weeks ago I told you about the Half-Dive headset, a weird HMD coming from Japan that is meant to play virtual reality while you are lying on the bed. We now know that this device is launching its Kickstarter campaign on December 6th, 2021. If you want to play VR in the bed (for porn, of course), get ready to support this project!
Have a look to VR Cover’s deals
To celebrate Black Friday, VR Cover has announced a long list of deals on its products. If you need a cover for your headset, this is the time to buy it!
Someone found a way to apply filters to videos
A developer has created a solution that lets you apply Snap Lenses not to the camera stream but to an offline video. The solution is a bit hacky but very interesting nonetheless.
Upload VR’s showcase is back
Upload VR is organizing a new VR showcase for December, 2nd. That day we will receive a lot of news and updates about VR games! It will be a good day for the gaming ecosystem, and I can’t wait to see what will be announced there.
Some news on content
- Medal of Honor: Above and Beyond will release on Quest 2 on November 15th for $40. Be prepared to clean some space, because this game needs 45GB of storage on your device
- Vail, a multiplayer shooter game, is implementing lobbies where players can hang out and play peaceful games like chess. This is pretty weird
- Upload has gone hands-on with After The Fall by Vertigo Games, and it found the UX to start a game with other players very interesting. As someone noted, it is not completely a good sign if the most notable feature they have reported is the UI…
- Tvori has launched ShapesXR on Quest, a tool to let you prototype in VR together with other people. It’s an interesting concept.
- Felix & Paul Studios have released the trailer for the AR book they are developing: Jim Henson’s The StoryTeller. Given their pedigree, I’m sure it is beautiful
- Someone is building a VR mod for Resident Evil 8. It’s still a prototype, but I guess it will be ready in the upcoming weeks.
More info (Medal Of Honor)
More info (Vail VR)
More info (After The Fall)
More info (Shape XR)
More info (The StoryTeller)
More info (Resident Evil 8 VR Mod)
News from partners (and friends)
Japanese game RUINS MAGUS is starting its Kickstarter campaign on November, 17th. Support indie VR developers!
Learn more (Game trailer)
Learn more (Kickstarter date announcement)
The VR Awards ceremony will take place in AltspaceVR on Nov 18th, 6.30 pm UTC. I’m one of the media partners, so I invite you to join it! It’s a nice fest for VR.
Learn more
Some XR fun
The Icelandverse is much better than Zuck’s Metaverse
Funny link
A dad joke.
Funny link
VR > physical displays
Funny link
As the developer of a Quest game, I can say that also the vice-versa is true.
Funny link
This Meta commercial is a total what-the-f***. It looks like the m**averse that you see without using glasses but using LSD.
Funny link
People keep mocking Zuck and his announcement of the m**averse.
Funny link
Donate!
Donate to my Patreon!
People that already donate:
- DeoVR
- Raghu Bathina
- Jonn Fredericks
- Jean-Marc Duyckaerts
- Reynaldo T Zabala
- Richard Penny
- Ilias Kapouranis
- Michael Bruce
- Paolo Leoncini
- Immersive.international
- Bob Fine
- Nikk Mitchell and the great FXG team
- Jake Rubin
- Alexis Huille
- Jennifer Granger
- Jason Moore
- Steve Biggs
- Niels Bogerd
- Julio Cesar Bolivar
- Jan Schroeder
- Kai Curtis
- Francesco Strada
- Sikaar Keita
- Ramin Assadollahi
- Jeff Dawson
- Juan Sotelo
- Andrew Sheldon
- Chris Madsen
- Tracey Wong
- Matthew Allen Fisher
- Horacio Torrendell
- Andrew Deutsch
- Fabien Benetou
- Tatiana Kartashova
- Marco “BeyondTheCastle” Arena
- Eloi Gerard
- Adam Boyd
- Jeremy Dalton
- Siciliana Trevino
- Joel Ward
- Alex P
- Marguerite Espin de la Vega
- Lynn Eades
- Sb
- Vooiage Technologies
- Caroline
- Liam James O’Malley
- Paul Reynolds
- Hillary Charnas
- Donald P
- Wil Stevens
- Brian Peiris
- Matias Nassi
Donate to my Patreon!
Disclaimer: this blog contains advertisement and affiliate links to sustain itself. If you click on an affiliate link, I'll be very happy because I'll earn a small commission on your purchase. You can find my boring full disclosure here.
