How Mozilla is working on the (privacy-safe) immersive web of the future

Hello Lars, introduce yourself to my readers!

I’m Lars Bergstrom, Senior Director of Engineering for the Mixed Reality team at Mozilla. We’re focused on making sure that the entire ecosystem for VR and AR to be successful on the web is in place – from browsers to collaboration tools to content creation, discovery, and monetization tools.

At Mozilla, you’re doing a lot for the AR/VR space: A-Frame, Firefox Reality, Mozilla Hubs, etc… can you make a list of all the projects you’re doing?

At a high level, I think of our efforts in three areas:

1. Browsers. Firefox Reality is a brand for bringing the web to any VR or AR headset. Today, we’re focused on android-based all-in-ones, Windows PC-based, and the Hololens 2. We also have an older build for Magic Leap, but currently there are technical limitations preventing third party browsers on that platform.
2. Mozilla Hubs. We believe that collaboration is going to be the thing that takes VR/AR from a really useful tool to a key way that everyone goes about their work.
3. Content Creation. For creators to be successful, they need to be able to build great things, get them discovered, and make money off of them. So whether it’s frameworks, tool vendors, or promotion of experience in Firefox Reality, we pull together everything creators need.

Why do you believe so much in immersive technologies?

The dream of moving games, video, training and social experiences from flat screens to the 3D world around us is finally becoming possible on a wide scale. XR devices are on track to get into the hands of tens of millions of users in the coming years, and developers are eagerly prototyping applications and platforms to meet this demand. At the same time, the web has emerged as the premiere vehicle for delivering experiences that can span desktop, mobile and tablet platforms.

Our goal is to create a better ecosystem than the app-based one we are living in right now, one where anyone can publish content and anyone can discover it — all in the same place, and with the great safety and privacy properties that users already expect on the web.

I think that WebXR is still very rough. Graphical quality is simple, the applications are still a bit buggy. Why isn’t it accelerating yet? And when will it become truly usable?

Developers have not had great examples yet of how to make visually polished, reliable applications. The devices are now powerful enough and the browsers are mature enough. And so we’ve worked on things like Hello, WebXR! (https://blog.mozvr.com/hello-webxr/ ) and Mozilla Hubs to show what’s possible and meet or exceed native experiences – even according to the people who run the Quest platform (https://twitter.com/JustRogDigiTec/status/1049059049277337601 ). We’ll continue to provide examples, documentation, and tutorials that will help developers to deliver native-quality applications in WebXR.

How do you envision the future of the immersive web? How will webpages of the future be? I mean, text has no sense in 3D, so I don’t believe in webpages fully in 3D. So, what will be the true format of the webpages of the future?

To start, we know that users love videos. Short-form video, livestream video, and movies are all even more popular (as a percentage of time) in VR than they are on mobile — where they are already hugely popular. Being able to have an immersive, theater-like experience really delights users, and the web continues to be a great way to find and deliver that content.

In the future, we think of broader web content allowing people to experience educational experiences, training, and a variety of social and collaboration experiences. And within those, people will want to do many of the same things they do on the web, but in 3D: view images, read text, look at visualizations of data, and listen to audio.

Janus VR has probably been the first VR browser and it has just shut down. In your opinion what have been the reasons for this?

With the combination of a small consumer base, difficulties in making money, and a high cost of production for making and marketing VR apps, the web is still a challenging place. And it’s made harder when you don’t have a default installation and good visibility on the platform. There have been many VR browsers and “open metaverses” and there will be many, many more. But until there is content that users want to experience and a clear place where users get value that they can’t get anywhere else, it’s hard to ship a platform for content. 

Talking about UX, have you already understood what is the right way to interact with a browser in XR? Typing the web pages addresses is so boring…

From our research with Firefox Reality, we know that users enjoy our voice search as a way to easily discover content. Additionally, people can enjoy greater benefits with a Firefox account which lets you sync your history and bookmarks as well as send tabs easily from their desktop Firefox to your headset. We constantly ask people how they’re using VR, what are their pain points to develop features that will delight users and help VR integrate seamlessly into everyday life.

A-frame is the de-facto standard for the development of WebVR applications, and it is now past version 1.0… you’re doing great in this sense…

Certainly! Even though most of Mozilla’s investments at this point are in providing examples and components on top of core A-Frame, we’re delighted that between the website and hosted versions of the framework, A-Frame has over 10 million monthly active users. And that doesn’t even count all of the sites that are hosting the A-Frame library on their own CDN or forked versions from 8th Wall, etc.! For the things that it does well, A-Frame is really enabling creators to easily deliver 3D experiences on the web today.

Mozilla Hubs instead is not usually much considered by people, that still seem to prefer VRChat, Altspace or similar software. What is the reason in your opinion?

Hubs is focused on getting groups of people who have a common interest and network together into a space to collaborate or interact socially. And we’ve focused our early adoption in scenarios that are not just passive consumption of content coming from a big stage: more on conferences, workshops, discord community events, etc. We continue to see fantastic growth in usage, especially as people look to reduce their travel for events.

I’ve seen some forks of Hubs like Mespper or UXR getting interest from the B2B market… may this be the possible application of the future?

We are currently working on a version of Hubs that users will be able to access in the cloud for business collaboration users. We hope to have more to share soon. Being able to both directly enable business and to allow third parties to build businesses on top of our Hubs platform are key business models we want to enable.

How do you envision the differences in the future for the Web VR and the Web AR?

The new WebXR standard not only unifies access to VR and AR hardware, but provides a way to improve the privacy and performance of web applications. On privacy, we will be able to more clearly prevent fingerprinting of web users by delaying when information about a headset is available plus we can add features that today require giving a web page or some third party closed source platform full access to your camera in order to have an experience in the real world. Those additional features can also improve performance by relying on underlying hardware, such as dedicated image detectors, rather than having to do computationally intensive image processing on the device.

Privacy is a big concern for XR devices, especially now that eye tracking is coming. What is Mozilla’s stance on this? And how can we preserve privacy in XR?

At Mozilla, we want an immersive future that respects individual creators and users. That means rethinking privacy — making sure companies consider the personal data they are taking and explicitly share what they will do with it.

For example, eye-tracking has the potential to expose individual’s intrinsic characteristics, such as race, age, gender, and sexual preferences, as well as revealing sensitive health characteristics such as whether they have autism or disorders like anxiety and depression. We have very little conscious control over our eyes, so the idea of providing unfettered access to data that can reveal all of this information (as well as provide unique user fingerprinting) is antithetical to our values of treating privacy as a first-class citizen. Instead of providing access to raw gaze data, the web browser can act as an opinionated user agent — an intermediary between the application and the device sensors and resources. In this example, perhaps we would only reveal that a user had looked at an element on a page if they’ve dwelled on it for a certain period of time. 

And how can we avoid a web of the future full of banners and annoying ads like the current WWW?

We’re working to enable other means of monetization that reduce the friction between users and content authors. This should not only enable us to get rid of annoying ads, but also the kind of “clickbait content” that is just trying to drive as much traffic as possible, no matter how small the engagement is, in order to maximize the ad revenue.

Firefox Reality has become the official browser for HTC Devices. What’s your vision of the Wave platform? How do you see this partnership in the present and in the future?

HTC’s VIVE products have always pushed the boundaries of VR, delivering the highest-end experience to consumers and enterprises not only in graphics, but also in room scale usability.

Working with HTC enables Mozilla to deliver web content to the top platform, where users and companies will push the boundaries the furthest. Being able to work with HTC directly and the Wave platform allows us to more deeply optimize the browser and integrate it more fully with the platform than we can on other devices. We look forward to being able to have deep integrations with both the platform itself and the system services on the platform to blur the lines between web and native content.

How do you think 5G can really change the future of immersive technologies in the short and mid-term?

One thing we’ve seen is that as VR and AR devices get lighter and more compelling, battery life is often one of the key limiters to more immersive content and longer session times. So we’ve been working to make it possible to just run our browsers on edge compute, streamed over 5G to the headset (https://blog.mozvr.com/browsing-from-the-edge/ ), taking advantage of the lower latency to still provide responsive experiences.

What are the future plans for Mozilla in XR?

Coming in to 2020, we knew that there would be a reduction in travel and a need for more virtual meeting solutions. Over the past few weeks, we’ve seen current events impact a huge number of people. We’re pushing extra hard right now to support all of the creators and speakers whose events could not happen this year so that they can still reach a broad audience with their work. We’ve just announced today that we will be supporting the top academic conference for virtual reality, IEEE VR (https://ieeevr.org/2020/ ). It will now be run entirely online, on top of the Mozilla Hubs platform. And we’re working with many speakers affected by the GDC and SXSW cancellations to ensure that they can still reach their audience, but do so without risks to health.

If you want to add something to this interview, feel free to say it!

At Mozilla, we are excited about how the internet will grow and evolve. We don’t want people to have to give up control of their personal lives in exchange for great products that rely on personal data for ads. We are working hard to make sure that new technologies evolve in a way that champion privacy and choice.