Gianluigi Perrone: “To make a VR film you need to know the DNA of VR”

Hello Gian, introduce yourself and your book “Dogma VR – Storytelling in Virtual Reality”!

I’ve been teaching screenwriting and storytelling here in China at Beijing Film Academy, and occasionally other Academies, because I’ve been working as a writer and producer for 15 years. Once I started creating narrative in VR, I framed my idea in a sort of masterclass or workshop, and I did some conferences in dedicated events. Simone Scafidi, one of the supervisors of the Luchino Visconti Film School in Milan proposed me to have virtual reality storytelling classes there, and we did an intense teaching unit for the students of multimedia classes.

Someway, Dino Audino, which is basically the Bruce Springsteen of film teaching books publishing in Italy (they released fundamental books from academics like Vogel, Katz as well as popular directors such as Maurizio Nichetti), knew about me and we met in Rome. We spoke about the book which was supposed basically to be the Dogma VR classes, but with a very simple approach as they have as target film academies but also schools.

Along with the writing, the book changed a bit structure: at the beginning Dogma VR should have been the main point of the book, not the resulting point, but I trusted their experience as an expert publishing house, and so I changed it. Later on, I received the proposal for translating it in several other languages and currently the book is being translated in English, which is mandatory for a worldwide release. The new version will have a different structure and it will have a lot of additions about social, gaming and even philosophy. It will be available on Amazon soon.

I read it and I found it very interesting! Let’s start from its end when you talk about the Dogma VR. What is the Dogma VR and what are its 12 rules?

I made a synthesis of the list on a dedicated website, www.dogmavr.com, which I’d love to explore here with you.

Basically, my first questions as a creator approaching virtual reality have been: “what is similar and what is different from traditional films?”. What I found out is that all the differences are related to the construction of the frame, that in immersive 360 is able to create the feeling of embodiment, giving the impression of extreme realism.

From this idea came out the fact that VR experiences can be “more real than reality”. From this threshold, I started to walk into the language of realism, which in narrative feeds what we call “suspension of disbelief”. Basically, when we wear the headset and experience, for instance, porn, we know that we are not actually there with the girl, but we deeply want to believe it so the chemistry in our body answers in the right way. This is the easiest way to understand it.

Why do we need a Dogma in VR?

In the ancient narrative, through centuries, storytellers identified patterns, figures, models and structures. The same happened with movies. It is interesting they all have something in common but with a natural adaptation on the storytelling medium. I read several books about VR storytelling and nobody did a similar job of identifying and writing about these patterns in a practical step by step guide, so I thought doing it myself.

Also, the term “Dogma” comes from Lars Von Trier/Thomas Vinterberg’s manifesto “Dogma 95“, which has been a very influential model of storytelling and production based on realism for films. The rules for VR are different, but the approach is the same. So the answer is: we need a method because in order to make a VR film you need to know the DNA of VR. Otherwise, you can just put the camera there and push a button, but it is not the same thing. I prefer the real deal.

In the book, you describe how a director should shoot a 360 movie. If someone from my readers comes from traditional cinematography, can you explain to him/her how the processes of creating a movie are different in 2D from VR?

There are many differences and some of them are quite complicated. I tell you that a traditional cinematographer should have a really open-minded approach to understand 360 videos because the first rule is to ignore the frame. Considering that normally a cinematographer spends his career studying the 16:9 frame, it is a huge difference.

Also, keep in mind that the cinematographer in a 360 cinematic VR experience should also be a stereographer, which means being able to use a rig dynamically (in a narrative sense). In The Irishman, last Martin Scorsese’s film, Rodrigo Prieto, cinematographer, used a 360 video to create a cinematography that could allow performing the de-aging of actors without losing the performance, for example.

What is the workflow that you follow to create a VR movie from the inception to the post-processing?

This is a time of experimentation so I try to evolve and improve my workflow every time I start a new movie.

I always ask my students to perform a very interesting experiment, asking them to imagine a scene from a film they loved and turn it into VR, if it is possible and especially if it makes sense. This is the first approach to an idea: “Why in VR?”

In the book, there is a chapter that lists some different virtual reality examples in order to make clear why these experiences have been made in VR. In a book the keyword is “structure”, in visual art is “harmony”, in a film is “engagement”, in games is “control”, in VR the keyword is “emotion”.

Then you have to study the location. The location is a character more than the characters themselves. Then there is a different approach on animation, visual effects and tools. There is a chapter dedicated to it and it is very complicated to approach VR because it is way more work. For example, if in a film you see a special effect for one second in a frame and then the film cuts to something else, in virtual reality you need to finish the effect until the end.

The tools that I usually use are Adobe Premier Pro, Adobe After Effects, Blender, Maya, Unity, Unreal. It depends on who am I working with.

I’ve read that you say that sound is of paramount importance for VR movies. Why do you think it? And how to provide great sound for your movies?

My first approach with spatial sound has been when I watched Slumdog Millionaire by Danny Boyle. The composer, A. R. Rahman is a pure genius. I had the chance to talk to him, and his approach to creating music has been groundbreaking. It is not a coincidence he experimented with Danny Boyle as he is somebody that used the sound in a sensorial way.

Let’s try to explain something complex in an easy way. Everything is made of vibrations. Literally everything that surrounds us. Above and below. So spatial sound can make possible to create presences in the area where we set the sound design. This means that one sound or one vibration perceived by your body can lead you in one direction or even give you one feeling.

The experimentation of this came when I realized Transcend, which is a VR experience about meditation. In the beginning, I wanted to stimulate libido creating vibes in tantric areas but then I changed my mind and went on something more spiritual. So we recreated the process of meditation with a Tibetan monk, showing what happens inside and all-around with visuals and sounds. The monks make some noises with their voice, a sort of chant, and they use the Tibetan bell for the vibrations. We worked on the spatial sound to pass through some areas inside the body where are positioned the chakras, the centers of energy of a human being. They are basically doors where the energy passes. Stimulating them, you…transcend. It works if you concentrate well, and in the end, there is a surprise. Somebody came out crying from the experience and not in the way the audience used to do when I was doing horror films.

You also talk about how to solve the fact that in VR you don’t have “framing”, but the user has all the dome around him, that can distract him. As a director, how do you solve this issue?

It is not an issue. It is just the identity of virtual reality because the frame is indeed a limit. We put images into a frame because we can’t incorporate it. Without a frame, there is no cut. Without cuts, there is no editing so the experience is a long shot.

How to prevent motion sickness and dizziness of your viewers watching the film? The Limit from Robert Rodriguez for instance is cool, but very nausea inducing…

It is a technical thing and unfortunatelly it depends from person to person, so it will be very hard to create a universal standard. Motion sickness comes from the neurological impact of the brain that identifies there is something irregular happening between the information you receive from the visual department, the eyes, and the equilibrium department, the ears.

Ok, that’s a long story and it’s not in the book. We are working on it on an experimental level. There is a concept called “plastic brain”, and explains how the brain works for learning things it doesn’t know and to adapt to situations. The brain is able to create bridges and links where they are not. It just take time.

What is the most common error of your students that start approaching the VR movies world?

Oh, not only students but also creators. The error is to look at VR the same as another medium, like gaming or films. I’m pretty sure you had to explain to a lot of people what is virtual reality and they didn’t get it. The most dangerous thing is when they think they got it but they don’t. There is a limit to overcome, which is the frame. Of course to understand VR, you must try it.

How much does it cost to create a short VR movie from the start to the end?

Ok, good question. I want to tell you an anecdote. Since several years ago, we do in Cannes the Immersive China conference to try to make people interested in the Chinese immersive market and tell what is going on. The first year, people asked me about the costs and I told them “well, it depends… but better you stay under 30000 dollars below the line“. Somebody in the audience (who is now a friend) wanted to kill me…. but after some years everyone understands I was right and actually I was even above what is permitted. Unless you don’t have the marketing budget for doing Spiderman VR, which doesn’t go a lot higher, you need to see immersive media as an industry. And like any other industry, it must succeed to grow.

If you spend one million plus another million to make marketing, promotion and win prizes around the world to not get anything back, you can go around celebrating as much as you want, but for the industry, you are a failure. And for the investors, you are out. This is a basic concept in economy, and even more in entertainment. There is a huge difference between budget and production value.

Also, you need to see what your investors need, which might not be necessarily ROI, but something which has the same value. As an independent producer, I am very used to perform this kind of acrobatics in feature films and indeed a short immersive experience of the production value of 300000$ is a walk in the park. Gaming is tough because there is a workflow you cannot invent, indeed. Alyx is groundbreaking in this sense because it is going to make money and it is made with the right money.

VR movies have a great problem of distribution (there are no VR cinemas) and profitability (there are few places where to buy them). How can we solve them? How can a VR director survive now?

Well, we do education and interior design as well, so it is not a big problem. Of course, the pandemic is going to hurt the global economy and it is going to be a bit hard. XR will be one of the growing industries because of the pandemic but be careful not to say “thanks to the virus!”. It is very important to know that yes, there will be more requests for virtual everything because “we need to stay home” but remember also that the economy is going down so people will tend to spend way less money for what is not strictly necessary.

Returning to your main question, and the main problems we have. Adoption and technology. We can’t have a widespread distribution because the headsets are expensive. Simple. They will be cheaper when there will be mass adoption, but after the most catastrophic economic crisis since WW2, I honestly doubt people will pay a thousand dollars to buy an headset. This would limit streaming and social VR which could be the businesses able to drive the industry.

What I mean is: if we cannot go out we could accept (sobbing) to attend a concert or an athletic show in streaming live in VR but not if we have to spend a fortune for that. The same for porn. The same for social VR. You guys have the hub on VRChat with VRROOM for the festival, we have the DogmaVR Factory on Sansar for exhibitions and events, but we know that for a full experience of the amazing features you need to have a proper machine. Otherwise, the end user will just stream on the desktop because it is not the same as live anyway. I think this is an argument to discuss inside of the industry.

This could put upside down all the previous premonition about the industry. The idea was that AR will lead to XR, but now, since AR works better outdoor and VR indoor, it could be the opposite. So headsets could win against smartglasses.

You like to direct movies that explore the darker side of human nature, like esoterism, erotism, horror. Why this choice? And why do you think this genre is great in VR?

Well, that’s nothing dark in sex… This question has many answers. We are educated by our society to not look at the dark side of things and just be positive. Indeed every philosophy ever existed encouraged the people to look into the dark side (the Abyss by Nietzche) in order to “come back to see the stars” as Dante said. It is cathartic.

Horror is in my background as a filmmaker and even before as a film lover. I am waiting for the VR industry to grow in order to do what I really mean with horror VR, which is killing the audience with a heart attack. Horror works well very easily because it is entertaining and it can tell a lot of true things through allegories and metaphors. Esoterism is basically the same reason but deeper.

Esoterism is the study of primordial signs in order to enhance consciousness. All the great artworks in history are esoteric. The Sistine Chapel is a huge esoteric opera. If you know that subliminal sign, your work is able to give “more than meets the eye” and the brain of the viewer will perceive it. It is perfect with VR because it is a plus to the empathic power of immersive media. It leads to the pineal gland of the people. I study esoterism since I was a teenager, and more deeply for 20 years, so it is a very important aspect of my life and I feel enthusiastic about my experience in bringing it into VR with so much accuracy. In the beginning, it was a try. My first 360 work, The 7^th Night of Thelema, is actually a ritual. We shot and engaged a ritual with a subliminal storytelling inside. For us, it opened the gates and I believe that with the right will (love under will) it can do the same to many people.

Then I like erotism because sex sells and sex is very honest as an emotional thing, so we will go more in that direction, but for sure not in a traditional way. We have a project in development called Bizarre VR. I see many people are doing experiences about spiritualism, but I don’t know how much they are accurate. It depends a lot on what are the intentions.

What are some storytelling VR experiences (movies or CGI) that you have seen and would advise to my readers? What movies made by others have amazed you?

All the ones that involve vertigo, since I suffer vertigo. I like Through You by Lily Baldwin. Very smart and sexy. I mean the film, of course. Technically, I appreciated a lot The Dinner Party and Battlescar.

You are Italian, but live in China. How is the 360 movie industry in China? What are the differences between doing films in China and in the West?

Well, to speak about China there should be another book. Let’s say that in the West there is more experience and method but also more competition and jealousy. In China there are the resources, but it is very hard to really go there.

In the Chinese circle, we are all in good relationships. The guy from Pinta Studios is even my neighbor.

In China, well, in Asia in general, as I work also with Korea and India, you need a cultural translator, which often is not a local. We set Immersive China as a bridge organization because the cultural shock is huge and now all the world knows what we know since a while: China has secrets.

It is a simple and pragmatic place where Hulk is called “big green man”, the minions “small yellow people” and Jennifer Lopez “big butt lady”.

If someone wanted to do your job of VR movie director, what would you advise him/her?

Learn how to do things, study a lot technology, art, economy and war. Be humble, but never submissive. Find your own mentors. Be competitive, but never jealous.

If someone wanted to live in China like you, what would you advise him/her?

Now it is going to be hard. My colleague a few, days ago told me that China is doing general cleaning. I feel now there is way more opportunity in the second level cities than Beijing, Shanghai etc

Tell us how do you envision the future of VR movies in 5-10 years.

The future will be like Mad Max for the Democratic countries and like 1984 for the socialist ones. Maybe starting very soon, as the epidemic speeded up a lot of things. Some industries will grow including the ones that work remotely like XR, but the product will have to offer something necessary, which perhaps means addictive.

Anything else that you want to add to this interview?

I would love my book to be translated in as many languages as possible, so if somebody has an interest in publishing it in his own country or just simply knowing more about it just get in touch with us at Polyhedron VR Studio.