An interview with NOKIA about volumetric videos and MatchXR

Hello, can you introduce yourself to my readers?

I’m Ville-Veikko Mattila, and I lead multimedia technology research and standardization at Nokia.  I’ve spent much of my career working on technologies that shape how we experience digital media, from video compression and immersive audio to newer areas such as volumetric video, XR and AI. What I find especially interesting is taking an idea from research, developing it with the wider industry through open standards, and eventually seeing it become something people can actually use. That’s very much where we are today with immersive media.

Nokia is mostly known as the company making phones back in the day. But what is its mission now? And what are you working on at Nokia these days?

Many people of a certain generation still associate Nokia with mobile phones, but today Nokia is a global leader in connectivity infrastructure, enabling the AI supercycle. We build the networks that connect intelligence, across fixed, mobile, IP, optical, core and data center networks. Our focus is on trusted, secure infrastructure that enables AI to scale and advances the next generation of connectivity.

Research and standardization are an important part of that. We develop new technologies and work with the wider industry to turn innovation into global standards, helping ensure that new technologies and experiences can work across devices, platforms and ecosystems.

In my area, that means working on the technologies powering the next generation of multimedia experiences, including advanced video compression, immersive 3D media, spatial audio and AI-enhanced video. For XR, we’re exploring technologies such as volumetric video and Gaussian splats that can enable highly realistic and interactive 3D experiences. An important part of our work is taking these innovations from research and helping develop the technologies and open standards needed to make those experiences practical, interoperable and scalable.

You are going to MatchXR. What will you be exhibiting or talking about there?

MatchXR is one of the leading events for XR and immersive technology in the Nordics, so it’s a great place to show how some of these technologies are moving from research towards practical applications.

One of the technologies we’ll be showcasing is our latest work on adaptive streaming of dynamic 3D Gaussian splats. Gaussian splatting enables us to create highly realistic volumetric representations of people, objects, and environments that you can experience interactively in three dimensions. In our demonstration, visitors will be able to experience high-fidelity volumetric video through lightweight AR glasses.

What’s particularly important is what’s happening behind the experience. We’re demonstrating how this kind of photorealistic 3D content can be compressed and streamed efficiently using standardized technologies and existing streaming approaches. Ultimately, our ambition is to make volumetric media as streamable and interoperable as video is today, so immersive experiences can work across devices and platforms and eventually scale to much wider use.   

What are your expectations for the event?

MatchXR brings together a really interesting mix of companies, startups, researchers, and technology developers working across the XR ecosystem. For me, that’s one of its biggest strengths because no single company can build the future of immersive media alone.

There are many exciting opportunities ahead, including the industrial metaverse, where immersive technologies and digital twins can change how people interact with physical environments and complex systems. But the opportunity for XR is broader than any single use case. Creating truly scalable experiences requires progress across devices, content creation, multimedia technologies, AI, connectivity and open standards.

Events like MatchXR give us an opportunity to exchange ideas, understand where the industry is heading and hear about the challenges developers and users are encountering in practice. I’m particularly looking forward to seeing what others are developing, getting feedback on our own work and identifying opportunities for collaboration that can help move immersive technologies towards wider adoption.

You work with volumetric videos, which are an impressive technology. But apart from being “cool”, what practical use cases do you see for them?

The interesting thing about volumetric video is that you’re no longer limited to seeing the world from wherever the camera happened to be. You’re capturing a person, object, or environment in three dimensions, so the viewer can experience it from different perspectives.

That opens up a very broad range of applications. In industry, for example, you could capture an expert demonstrating a complex procedure and allow people to experience that training from different viewpoints. In healthcare and education, it could enable much more immersive teaching and simulation. And there are equally compelling consumer applications, from experiencing a sports or entertainment event as though you were there, to exploring cultural heritage, products or places in three dimensions.

We’re also seeing interesting possibilities around telepresence and AI. Realistic 3D representations could make remote collaboration feel much more natural, while volumetric content can provide rich 3D data for training AI systems and creating simulations.

So, there isn’t really one killer application. What’s interesting is that the same underlying technology can support everything from industrial training and digital twins to entertainment, communication, and new kinds of immersive digital experiences.

Why are volumetric videos not mainstream yet? What technical achievements are needed?

As with any new technology, it takes time to become mainstream. What we are seeing with volumetric video looks incredibly positive, though. Many companies are already bringing this technology into their services or products, some of which are consumer-facing. As more and more consumers are experiencing volumetric video, it is only a matter of time before it becomes mainstream.

Technology-wise, there are no fundamental roadblocks preventing adoption. With earlier generations of volumetric video technology, the quality of the experience was an issue that prevented adoption for many high-fidelity use cases. With the adoption of radiance field -based representations, this is no longer the barrier it once was.

Generating and consuming volumetric video no longer requires expensive dedicated hardware, both can now be done with your smartphone, which means that the barriers to both creating and consuming this content are also much lower than they were before.

Overall, the volumetric video industry has evolved significantly to overcome many of the challenges that were preventing its widespread adoption in the past.

Will average consumers one day create volumetric content as easily as they create photos and videos today?

Yes and no. Thanks to AI, everyone can create rudimentary volumetric content just by using their phones. For higher quality, realistic, dynamic experiences, a dedicated capture system and specialized workflows are still needed. But this follows the same pattern we see with professional 2D video production vs consumer-created video. There will always be dedicated studios generating professional content, while consumers will be shooting their own content and sharing it online through their social networks. AI might also play a role here, possibly allowing anyone to create high-quality dynamic volumetric video, simply by using a prompt or providing 2D video as input. 

How is AI revolutionizing the volumetric video field?

AI is interesting because it can help us at almost every stage of the volumetric video process. It can generate 3D models from relatively limited input, reconstruct scenes more accurately, and reduce how much capture equipment or data you need to create the experience in the first place.

It can also help us process, compress, and stream all that 3D data more efficiently, as well as improve realism and enable new possibilities such as synthetic environments and digital humans.

Ultimately, AI has the potential to make volumetric content much easier and less expensive to create and deliver. That’s important because reducing that complexity is one of the things that could help immersive media move from specialist applications towards much wider adoption.

Finland announced a National Metaverse Initiative in 2023. How are things going?

Finland has made significant progress in building a collaborative metaverse ecosystem.

The initiative has helped bring together companies, universities, research institutions, public organizations, and startups around common goals. Rather than focusing on hype, Finland’s approach has emphasized industrial applications, human-centric technologies, sustainability, and digitalization.

Several projects involving digital twins, industrial XR, virtual collaboration, smart manufacturing, and immersive training have emerged through this ecosystem approach.

The Nokia Veturi Industrial Metaverse LEAD program complements this national strategy, bringing together an ecosystem around areas including XR, advanced connectivity and AI-driven applications. It is a good example of how this collaborative approach can help turn research and innovation into practical industrial applications. You can find out more here about the Industrial Metaverse LEAD program.

What is Nokia’s role in Finland’s metaverse ambitions?

Nokia contributes foundational technologies and provides ecosystem leadership through the Nokia Veturi Industrial Metaverse LEAD Program, which connects over 200 partners through Co-Innovation projects and subcontracting with Nokia. Public funding from Business Finland is helping to increase investment in Finnish R&D within Nokia and accelerating ecosystem synergies. Metaverse Finland Ecosystem (MEFI) has been a key contributor in the formulation of sub-ecosystems and consortia in the field.

For us millennials, Nokia means the legendary indestructible 3310 phone. Have you ever seen a broken one? Do you still use them sometimes inside the company?

The Nokia 3310 remains one of the most recognizable technology products ever created, and it still makes people smile whenever it is mentioned. While we’re no longer using 3310s as our everyday phones, I think the qualities people associate with it, reliability, quality, and trust are still very relevant to Nokia today.  It’s great that people still have such affection for something Nokia created more than 25 years ago.

And yes, despite its legendary reputation, there might be a few broken 3310s somewhere out there, although they seem surprisingly difficult to find!

Anything else to add?

The most exciting aspect of immersive technologies is that we are moving beyond demonstrations and prototypes towards practical adoption.

When AI, XR, digital twins, and advanced connectivity come together, they create entirely new ways for people and organizations to understand, interact with, and improve the physical world.

At the heart of this evolution is the increasing convergence of physical, digital and human worlds, with people at the center. AI and advanced network technologies will be key enablers in bringing these worlds together across many different sectors. We believe this transformation is only beginning, and collaboration across the ecosystem will be essential to make it happen.