Noctvrnal: audio in AR/VR is 51% of the experience

I had the amazing pleasure of interviewing Helena McGill and Anna Wozniewicz of Noctvrnal, a startup focused in audio and music production, with special attention to immersive experiences.

When I develop a VR experience, I usually give a lot of attention to the visual aspect, while I often leave the sound and music always as the last thing to care about. After having talked with Helena and Anna, I realized that I’ve always made a huge error, since spatial high-quality audio is fundamental to improve the retention and the appreciation of the users.

In this interview, you will discover why audio is so important and you will get also various insights on audio in immersive experiences and on the problems of being a woman working in XR. Have a nice reading!

What does your company Noctvrnal do? What are you currently working on?

Noctvrnal is an audio post-production studio based in Los Angeles. Co-founders Helena McGill and Anna Wozniewicz are visual storytellers with a background in sound design for narrative film. After noticing a lack of female entrepreneurs in audio and music production, the duo decided to launch Noctvrnal in January 2017 and have since expanded into the XR and immersive tech space. Noctvrnal is bringing immersive sound design to the forefront of virtual and augmented reality by pushing the boundaries of current techniques to create a more visceral experience. The team is passionate about creating new worlds through sound, and their work has been featured at MIT, USC, and VRLA.

Currently, we are working on a number of VR experiences, a permanent public works installation in Denver, some experimental immersive theater pieces, a series of audio pod-plays, and a sensor-activated audio-reactive installation.

You are an audio designer… why do you think that audio design in an AR/VR application is important?

Audio in AR/VR is 51% of the experience. As in real life, when you look around, your field of view is limited, but you’re still hearing the full 360 degrees sound field around you, so it needs to be realistic and not pull you out of the experience.

Is it also important for enterprise applications or only for games and movies? Why?

Spatial audio is important for all AR/VR experiences. Research has shown that spatial audio keeps users in an experience for 40% longer. You want the sounds to mimic real life and add realism/immersion to the experience, whether it’s for entertainment or enterprise applications.

We pull a lot from our research background in psychoacoustics in order to design effective auditory UX/UI for enterprise. This is especially important in the military, training, and therapy realms, and we draw from our background in psychoacoustics in order to design these systems.

So, how the audio for an AR/VR training experience should be designed? What pieces of advice can you give to us?

For any AR/VR experience, you want to keep psychoacoustics in mind – placing sounds in a way that will draw viewers’ attention, or making sure the sound effect’s duration is enough to hold their attention and allow them to locate the source. For example, sound sources are more easily located along a horizontal axis, but our brains have a harder time perceiving variations in height. Use each experience as R&D for the next one – every experience is different, so figure out what worked (and why), what didn’t, and how you can improve for the next one.

And what are the differences of designing the audio for an AR experience from designing it for a VR one?

Depends on the experience itself. The main difference in workflow, whether AR or VR, is if the experience is interactive and being built in a game engine, or has a more linear/cinematic timeline. Also, within a game engine, you can simulate room reflections and occlusion more accurately for VR, because the objects are all virtual. For AR, most experiences exist as mobile AR and the audio is an afterthought, because people don’t always carry headphones around with them. We’re really excited about BoseAR and its potential to bring more attention to hands-free, spatial audio.

Are you excited about the HoloLens 2? Why?

Very excited! We work a lot with enterprise and training, so the Hololens 2 is going to open up a lot of opportunities and potential for advancement in that sector.

How do you perform your work?

We like to be involved as early as possible in the script development process. That way, we can include audio triggers as part of the story and use them to direct the viewer’s attention. We capture audio on set for cinematic VR, create sound objects for game engine-based experiences, and work with ambisonics (sound fields). Our primary tools are DAWs, spatial audio plugins and SDKs, as well as traditional and ambisonic microphones.

What in your opinion still lacks from current AR/VR technology to make the audio fully realistic?

Biggest issue: many experiences are still not using spatial audio; it’s just in stereo. We’ve also found that spatial audio isn’t fully understood by the creative teams, or isn’t utilized to its fullest potential, i.e. to engage the user with the full 360-degree space.

Spatial audio SDKs are rapidly improving in terms of realism. However, the next step will be headsets that measure HRTFs in real time.

What is your opinion on the recent audio AR glasses like the ones by Bose?

They’re a great start and a huge step forward for spatial audio applications. They’re making spatial audio more accessible to everyone, and putting an emphasis on the audio side of things, rather than focusing on visuals first.

What is in your opinion the best AR and VR headset for what concerns the audio quality?

It really depends on the headphones you’re using with that headset. Standalone headsets, like the Oculus Go, have decent speakers built in, but oftentimes for demo purposes at crowded events, the noise floor isn’t taken into account.

There is a lot of buzz about diversity in augmented and virtual reality. You are a woman in XR. What do you want to say about the topic?

Being on the crossroads between three male-dominated industries (tech, entertainment, and audio) definitely isn’t easy.

We’ve had to expend a lot of energy on simply proving our worth before we’re taken seriously as audio engineers and treated as equals and collaborators. One of the biggest problems for women in XR, too, is access – most of the funding comes from men, so fewer women are getting funded than men, which creates a vicious cycle. We’ve found that our biggest projects and repeat clients come from fellow women in the XR space. Whether intentionally or not, women supporting women has been the biggest support system in immersive tech.

How do you envision the future for audio in XR?

The future of immersive tech and spatial audio lies in location–based experiences and experiential marketing events. Headsets are isolating, so the more we can bring people together and use tech to create social experiences, the more engaging the experiences will be. Some of the best experiences currently are audio reactive, or concerts in VR, or music-oriented games liked Beat Saber (the #1 game on Steam).

What about your future instead?

We’d like to continue pushing the boundaries of spatial audio and finding new ways to incorporate audio into experiences, whether they’re AR/VR, location-based, or completely experimental content. We are storytellers at heart, so we’re always looking for ways to challenge current techniques and use sound to convey stories in new, engaging ways.

I hope that you have loved this interview and that it has helped you in getting more insights on audio in Augmented and Virtual Reality experiences. And if you want to get to know better the fantastic girls at Noctvrnal, have a look at their website or ask me for an introduction! 🙂