Mary Matheson speaks about storytelling in VR and the importance of 360 movies

Mary, I know that you’re a great content creator, you’re an award-winning storyteller, but there are still some people that don’t know about you, so can you please introduce yourself to my readers?

Yes, sure. I’m an immersive director. I usually make factual documentaries. I’m also a professor of practice for Arizona State University, which is opening a new center in downtown Los Angeles actually. I teach a master. I’ve been working in immersive production for about seven years now. I moved over from regular filmmaking into making virtual reality documentaries.

How have you switched between traditional storytelling to VR storytelling? What’s your story? Why have you decided to experiment with VR?

I think that I was always looking for new ways to tell stories. I was a foreign correspondent originally. I was a journalist, so I was always trying to tell stories from other countries and bring them over to Europe or the United States to the audiences there. What was interesting about virtual reality is that instead of bringing my characters or stories from other countries back to Europe, I was able to bring my audience in Europe over to the countries where I was filming and working.

It was a very extraordinary experience the first time I myself watched something in virtual reality because I understood straight away that you could create a connection between your audience and your character very quickly and with greater depth than regular filmmaking. That was how I was captured into this world, and realized that it was a really powerful medium. It was a really powerful tool to tell these stories.

Using these tools, what are you working on lately? I’ve heard that you are creating immersive series about new NASA’s Artemis mission. What is it about?

Yes. I can’t say too much because it hasn’t come out yet, but it’s using virtual reality as well as augmented reality to let audiences play with the story. It’s going to be a really fun experience where you can feel like you’re part of the next mission that’s going to be going to the moon. Again, it just brings the audience into the story so that you feel like you’re really part of it.

When you go and watch a film, often you’re there. You are looking through the screen and you’re observing, whereas with virtual reality, you can feel you’re actually there, you’re actually in the scene, in the story. We thought it would be really fun to do this with NASA with the next mission to the moon.

I can’t wait to discover more about it. I heard that NASA’s Artemis mission is also about inclusion of people. They are trying to put someone different from the usual white male on the moon. This is coherent with what you are working on a lot in the past. You always try to raise awareness about important problems like ISIS and like sexism, inclusion, diversity, etc… Why do you do this and why do you think that VR is good to raise awareness on these themes?

I think there are two reasons. One is that we’re at the dawn of a new age in this new technology. Everybody starts to talk about the metaverse. It’s this whole new industry, so why not try and create a new ethos? Like a new way of working, a new methodology of working. That’s why there’s quite a concerted effort to try and make sure that this new industry is inclusive, and it’s equitable, and diverse in the people who are making the stories as well as those who are distributing it.

Then for me personally as well in my work, generally in my documentary filmmaking, more and more, I’ve been working with characters to co-create the stories, to co-create their own stories. They have much more ownership over what’s being said about them. You don’t just fly in and take some shots. I write or make a film about what I think’s happening. It really is about them trying to tell their own stories. For me, that’s also what inclusion is about.

You are really working on a much more equal footing with people. Obviously, I’m like a mediator or facilitator between worlds and somebody who knows the technology and knows the techniques that we can use for the storytelling. It’s really got those two parts to it. One is actually the content I’ve always been interested in and worked with. Secondly, that here we are at this new door, and it’s a brand-new world. Let’s make it a better place, basically.

We got in touch because we have a common friend, which you are working with. You are together with MIAT offering a masterclass on immersive storytelling. You try to teach people how to create movies like the one that you do. Can you tell us something more about this masterclass?

Absolutely. I was so excited to be invited by MIAT, as you were saying, this fantastic organization run by Elisabetta Rotolo in Milan. It was really what I was interested in. We don’t want this to be in a very exclusive little area where only very few people are working on this. We understand this technology is happening now, and it should be learned by everybody, whether you are an artist, whether you are a filmmaker, whether you are somebody who is in advertising or education. You are going to be using these tools more and more.

We really want to train up and teach people how to use it. We set up this masterclass. Last year was the first year. It’s 60 hours. It’s like a full immersion boot camp. We have small groups, and we basically cover the fundamentals of immersive technology. Obviously, now we’re talking a lot about the metaverse as well and a little bit about virtual production. What we do is we teach the students who come on the course, all the techniques that they might want to be using to create their own story, or their own pitch, let’s say.

At the end of the week, they all come up with a pitch that they then pitch to some top commissioning editors. We have Liz Rosenthal, who’s an exec producer, and she’s also the curator of Venice VR Expanded. We have Myriam Achard from Canada, who is head of partnerships at the PHI center in Canada. Dan Tucker, he used to be the curator of Sheffield DocFest. He’s now an exec producer in his own right. This year as well, we have a new person coming who is from Somnium Space called Artur Sychov.

There’s a real variety, as you can see, but the idea is that anyone who comes on the course really gets hands-on with all the tech, but also is working towards producing something at the end of it, which actually could get them some work if they’re interested, or can certainly help them if they want to develop a pitch within their organization.

I think that’s something that can be very interesting for people who want really to learn this new medium of telling stories. This is not the first year you do this, it’s the second edition. How was the course last year and what have you learned yourself?

It was huge. You can imagine it was really intense because it was the first time we were doing it. It sold out. We had international broadcasters. We had the president and the vice president of Women in Film and TV from Italy. We had people from events organizations. We had sound specialists. We had video editors as well as filmmakers. It was a real variety, as you can imagine. I always say I teach like I cook, because I’m always terrified people don’t have enough food. You don’t have enough knowledge.

I make sure that you basically have everything that you need. For that, we also brought in other specialists. Obviously, I’m a director but we also brought in Dan Stankowski from the UK. He was teaching game engines. We had other people teaching immersive sound, augmented reality. We had someone from Facebook talking about the technology and how it works with your brain. We brought in people who were real specialists in the very specific technologies as well, just to give the attendees a real insight into the technology if they wanted to go down a particular route.

What do you think is the best added value of this masterclass that you’re doing? If you had to choose one thing.

The thing is it’s a really full eight days. You really do learn the basics of everything. I think what makes it particularly relevant and particularly special is that it’s very industry-focused. It’s very hands-on. There’s a purpose to it. You are not just going in and listening to somebody talking to you.

You are getting in there. You’re working with the cameras. You’re recording sound. You’re learning how to use the game engine. Also through the week, you have to have in your mind, “If I had a project, what would that project be like as I’m going along learning about all of these?” These are creative people, they ask themselves “What would I do, if I could do anything?” If George Lucas came to me and said, “Here you go. Here’s a million dollars. You can make what you want, what would you make?”

That’s really important because it’s very focused and practical. Everybody that’s really on this course, they’re already working in some business. They’re all professionals working in the field, as I said, and coming from all backgrounds. These are people who want to make sure they’re using this to progress within their careers, or if they’re artists, in their artists at work.

I really invite everyone to check out this masterclass (at this link) because it looks very cool. Changing topic now, and trying to think a bit more about your career, is there something about all your work that really made you proud of doing it, and why?

I think when I feel proud is when a change has happened for the people who I’ve worked with in the film. I think that as a journalist when I started out, I felt really like it was a very strange relationship where you would go in and do your work and then leave these people behind. Then you didn’t know what happened to them. That’s when I started working more with nonprofits or organizations that were actually working with the characters that I might be talking to so that there was some support.

Really the proudest moments I’ve had is when those people have benefited from that film being made or from the VR being made. In the sense that their lives have changed as a result of it, or suddenly within their community, they are respected because people have heard their story. They now understand what the issue is. In terms of peace building and that kind of work, that’s what I really– That makes it kind of worthwhile.

I’m trying to steal from you some information outside the masterclass. What are, in your opinion, the most important lessons to take in mind when shooting a VR or 360 movie or documentary?

The most important thing to think about is that you are not shooting a scene. You’re creating a world, because your audience is going to be in that world and they’re living in that world. One of the major things that we talk about and that we learn about in the masterclass is world-building and how you create a world. Then from your world, you can break it down into scenes. That’s how you might end up shooting or creating an environment in a game engine.

So, creating a VR movie is a bit like doing some sort of game design to create these worlds…

Absolutely. That’s exactly right. What’s so interesting is that I take techniques now from theater, from gaming, from radio. It’s not just film. You are learning techniques where you’re using sound to direct people or…. like a theater maker, you are thinking about the whole space, you don’t just think of what’s in front of you. You’re bringing in all these different techniques from all these different genres. It’s a very different experience.

And apart from these, what are the differences between shooting flat movies and 360 movies? Apart from this world creation, what are the differences? And what are the lessons to take in mind if someone that is listening to you and is used to shooting flat videos wants to switch to 360 instead?

One of the major things is that you don’t shoot in sequences. In a regular 16 by 9 film, you would think using the regular methodology that is you do a wide shot, a mid shot, a close up, etc… You don’t do that at all in 360 filming. It’s much more that you are creating a scene, let’s say, as opposed to a shot. The main thing, you would never try and do a close up in 360. What you need to do is bring your character or whatever the scene is closer to your camera. You leave the camera in the place rather than bringing it down. If you do a close up in a headset that feels horrific. It feels really uncomfortable. That’s the first thing.

Secondly, don’t move the camera unless you’ve got proper equipment because you’re just going to make people sick, and that is the biggest thing that people don’t like. They find it really hard to get their heads around that. The worst thing is if you move the 360 camera, and without doing it properly… it’s just horrific.

There is this little debate about if it is better the 360 format because it is super immersive or the 180 VR 3D format. Which one is better? Why, what do you prefer?

This is a great question. I prefer 360, I’m going to be honest, because I really love the fact that you can look behind you. You can look all around you. I think that it really does make you feel much more like you are there. However, 180 is great as an in-between. 180 is a really good way to start off. It’s much easier to move in 180. It really is almost like an in-between, but you can use the benefit of the stereo and that’s what’s cool.

I don’t know if you’ve seen it… There’s a great new lens that Canon has brought out, and that’s really fun to work with because it’s a 180 lens. You’ve got a lot of freedom. Again, when you move around, you really should have it on a gimbal, but you’re quite free to play with it in a way that with 360, it’s much more difficult to make sure that the shots are really good. I think for me in terms of taking your audience to another world, 360 is the king.

What do you answer to people that say that 360 movies in 2022 are not relevant, or even dead?

360 film is still my favorite because it’s so beautiful. I can understand why people say that because of the interaction. The interactivity is obviously limited. I think that will come, but I think that 360 is this whole genre that we still have to discover. We’re just learning how to make it. For me, game engines are nearly photo-realistic, but not completely. Certainly, if you have people in the scene, there’s nothing like the real thing for me. And it’s certainly not dead. I’m making a series at the moment. I’m making one film, a part of a series, which is a 360 series. I made a 10-part series last summer. I think it’s still alive and kicking.

Some people that want to enter your industry sometimes ask me for some advice, like what camera should they buy? What software should I use? I’m a developer, so I have no idea. I ask you to help me in giving an answer. What do you suggest?

Do you know what, the GoPro Fusion, it’s now called a GoPro Max. That’s a really great camera. I still use it for like… if I’ve got a drone shot or like… we are in the water or something like that. They’re really good quality. They do very good quality and they’re only about £300 or £400. I presume euros are more or less the same. I would always recommend that. I myself play with that quite a lot. When I’m shooting something that has a bigger budget, I would probably use an Insta Pro 2. It’s standard really now. But there were lots of other cameras around… My favorite was the YI Halo, which had 17 lenses. It was beautiful, that camera, but it’s now defunct. There’s a lot of them that are just not used anymore because the systems aren’t supported. The Insta Pro 2 is great. It’s a great camera. Those are really the two that would be the mainstay for me.

As for the software used to edit the videos… they just get edited on Premier Pro and then, stitched with Mistika.

Okay. Finally I have some answers to give to my friends when they ask me these kind of questions. I wanted to ask you another question about storytelling. There are lots of 360 storytelling experiences made by amazing creators out there. What are your favorite ones?

One of my favorites is Traveling While Black. I don’t know if you know that. That’s a 360 film. It’s actually really pretty simple and shot in quite a straightforward way, but they also composite in Archive in a really beautiful way. I think that what was so good about that film as well was that they brought something historical into the present and made it feel very relevant. I think that was a very powerful piece of VR.

I also watched recently Paper Birds, which has been made by Baobab Studios, and that’s a beautiful little film that really plays with scale and in a really fun way. Gloomy Eyes, again, I think is one of my favorites. Battlescar is great, that’s really fun… That plays with scale. I think what’s so fun about those kinds of experiences that you can feel like you are there in the game or in the story one minute, and then the next minute you are watching from the outside. I like it when there’s a playfulness like that.

I still want to ask you one thing. I’ve talked a lot about lessons, class… but also life, the professional life teaches us many lessons. What are some lessons that you want to say to people that want to do your job? Not the technical ones. We’ve already discussed about the tools, et cetera, but really… how someone that wants to do your job should approach the job? What are some life lessons to get from your job?

I think the biggest thing I would say to people is to jump into it, like join us. We want more people in this industry. We want it to grow. We want people experimenting, just having fun with it. I think what I found really incredible, and I’ve been a filmmaker for like 20 years…. what’s really fun for me about 360 in particular, but actually VR generally, is that you’re experimenting and playing and sometimes coming up with a technique that’s never been tried before. That’s kind of cool. We are the pioneers, and it’s really an exciting place to be at that cold phase.

It can be frustrating. Sometimes it’s a nightmare. I’ve been on shoots where my cameras died because they were all prototypes. It was too hot, so we don’t know why they died. It’s really hard. It can be really difficult and really frustrating. Equally, there are times when you shoot something and it’s amazing. People look at it and they have never seen that before, and that’s a very exciting place to be. That’s what I would say to people. The biggest thing is just to jump in with us, to go for it and try and do whatever interests you and push it forward yourself because this is such a new industry. There are no rules right now, we’re creating the rules.

That’s so exciting. How do you see the future of this environment?

It’s getting bigger, isn’t it? It’s interesting. People often ask me about what I think about Mark Zuckerberg and Meta and the metaverse. Is it a good thing or a bad thing that Facebook is so involved? I guess, again, I just think it’s good that people now know about what the metaverse is or people are asking about it. It feels like it’s growing. The statistics show it’s growing: the numbers of headsets and people working within the industry are all growing. I think there’s still a way to go in terms of mass adoption, but certainly, more and more people are getting headsets, and in a sense, we need to be creating the content for them.

Totally agree. We’re at the end of this interview. The time has flown so fast. As usual, my last question is: is there anything else you want to add, you want to say to my viewers, to my readers? Say whatever you want!

Thank you. I think, again, come and join us. Come to the masterclass if you want to learn about it. It’s a really fun place to be. You meet other creators and you work together in small groups. Come to that, but if you can’t come to that, there are lots of other ways to get involved and just explore and have fun. Get a headset, get into it and come to be part of it.

That’s great. I totally agree with you. It’s a great time to be in VR and experiment. The first thing to start is just… start without worrying.

Exactly.

Get the devices. Start experimenting. Make a mess, and then you will create something good in the end. Mary, it’s been so great speaking with you. Thanks again for taking the time.

Thanks for having me.