MetaMovie brings interactive theater to VR

Some days ago, I got to know about an interesting VR storytelling project called MetaMovie. It is an experience that mixes Virtual Reality, storytelling, and theater in a quite unique way and I think that you should know about it.

VR can make you feel inside a movie, like the main character. In this blog, I have already outlined how this is awesome, since you can really live a story as if it were real, as if it were part of your life: for instance, in The Limit you can have an adrenaline adventure, in Defrost you can feel powerless, in Wolves In The Walls you can live as in the dreams of a little lady.

What is really interesting of MetaMovie is that it mixes this with theater: the characters of the story you are immersed in are not simple recordings, but are actors performing live just for their audience. This is more complicated to be organized (exactly as organizing a show in a theater is far more difficult than projecting a movie in a cinema) but has the advantage that is able to create a real bond between the actor and his/her viewer. Furthermore, the MetaMovie starts with interactivity in mind and this means that the viewer becomes completely part of the story and can interact with it.

To discover more this cool project, I have contacted Jason Moore, the mind behind MetaMovie, that will give you a long description of it. It will be a long and amazing journey inside VR storytelling.

Before leaving him the mic, I want to suggest you fund The MetaMovie project on Kickstarter, so that it can become a reality. It is so close to its goal…

Hello everyone! I’m Jason Moore, I call myself a cross-platform visual storyteller since I write, direct, and produce in film, single and multi-camera television, theater, and emerging media. I’ve had a lifelong obsession with wanting to not just watch a film, but experience it, from inside the magical world of the story. I know a lot of us share this dream. And as virtual reality technology became available to filmmakers, many of us have been experimenting to see what is possible with this exciting new toolset. However, to date, the results have been underwhelming.

I’ve watched and experienced many stories told using all varieties of Virtual Reality technology, and to me, they mostly come up short. Yes, there are some powerful 360 video documentaries out there, and some immersive stories told with gaming engines are clever and enjoyable, but nothing I’ve experienced yet comes close to what I’m after: I’m looking for a well-written story, set in a completely immersive virtual world, using cinematic techniques of design, music, sound, and light, with meaningful, believable, and emotional performances from actors. There’s nothing out there yet that can do this, and I believe one of the critical elements that keeps getting overlooked is the emotional connection between an audience and the fictional characters of the story.

When you’re watching a film on a screen, that fourth wall between you and the story actually helps you suspend your disbelief and emotionally fall into that story. We know it’s a story on a screen, our mind kind of ‘clicks’ and goes into ‘story-receiving’ mode and we’re ready and willing to believe the characters and emotionally connect to them and the story.

That same paradigm functions very differently in a virtual reality environment. VR shatters the fourth wall completely. If you are standing in a virtual space that looks and sounds and ‘feels’ real, the experience of watching pre-recorded actors (either photo-real or more cartoon style avatars) feels weird to me and I get very little emotional connection to the characters or the story. It’s fun to be inside a virtual space but I have yet to get sucked into a story the way I do with cinema. Virtual Reality is designed to be interactive: you want to move and explore, walk, run and fly, you want to hold and use objects, navigate that world freely. All of that inherent interactivity is what makes immersive VR so exciting. But when you stick pre-recorded, NON-interactive characters in this place, it goes against everything that VR can do and falls short of what traditional cinema does so well.

I always imagined that in my own fantastic journeys ‘into’ a movie the actors would be alive, breathing, talking to me, including me in their stories, letting me tag along, and acknowledging my presence. I don’t want to go into a fantastical world only to watch a bunch of robots marching through their paces reading pre-recorded dialog and totally ignoring me. For me, the story has to be interactive. The characters need to be alive. 

That’s why when I started the MetaMovie project, I knew it would have to use live actors. I describe The MetaMovie project as an ongoing series of experiments exploring completely immersive, totally interactive movies set in the virtual reality metaverse. MetaMovies are influenced by interactive theater, video games, cinema, and role–playing games like Dungeons and Dragons. Instead of trying to apply traditional rules of cinema onto VR, I’m trying to listen to what VR is telling me and use elements from a variety of sources. 

A MetaMovie is performed live, with a cast of live actors who are all using their own VR rigs in their homes or their office, to perform together, in the same virtual place, in real time. This allows our audience member, we call them the VIP, to have an experience unlike anything they’ve ever had before: a totally immersive, completely interactive, cinematic experience from the inside of the story. When you’re the VIP, you are given a role to play, a character, and the story unfolds around you. Our live actors are trained in improvisational theater, so whatever you want to do or say, we’re ready for you. If you’ve ever seen HBO’s Westworld, or David Fincher’s film The Game, you’ll have an idea about what the experience is like: a story that puts you, the VIP, right in the center of the action. 

Another term I’ve started to use for what I’m doing is Participatory Cinema. Our audiences become part of the story, it’s a shared storytelling experience. But unlike the type of shared storytelling that happens when gamers role-play during an MMPORG, this is not “user–generated storytelling”. I firmly believe that great stories begin with a great screenplay written by an experienced and talented writer. So for me, shared storytelling works best when there are professional storytellers running the show. Kind of like reverse Dungeons and Dragons: instead of a group of players with a single Dungeon Master who guides the narrative, in the MetaMovie we have a group of live actors who function as a group of Dungeon Masters, and we have one player at a time.

That’s an important part of the MetaMovie Project: for the moment, we take a group of actors and technicians and tell a story for one audience member at a time. This is not necessarily by design, it’s more a function of what we are able to do at this point in time. Since we give our VIP audience member complete and total freedom to do or say anything they want, it takes all of our energies to control and manage that freedom. Yes, it makes the experience difficult to scale, and I have solutions and ideas in place to deal with that, but at the moment I am less concerned with scaling and more interested in getting the experience to be incredible, amazing, transformative. And what I’m seeing is just that: our VIP’s are completely blown away by the experience. 

For a VIP, the experience begins with a short introductory scene where we introduce them to the world of the story and help them understand a little about how the story will work. We help them create their character, similar to a D&D character sheet: we come up with some basic ideas of who this character is, to help them get into the role–playing aspect. As a VIP, you will interact with all the characters, but there’s no pressure to perform: our main characters –  the protagonist, the antagonist, the supporting cast – they do the heavy lifting: we tell a story for you, with you, involving you, but all you need to do is go with flow: the experience doesn’t depend on you doing anything, but you can, if you want, do anything. We try to keep the story on the track that is laid out by our writer, and we’ll entertain some variations on it if you want, but we’re not a totally open world with a million story branches. We have a few and we do our best to encourage our VIP’s to follow our lead, and when they do, the experience is very rich, very immersive, and very fun. And when our VIP’s want to go somewhere else, we have ways on managing that, too. 

We have one MetaMovie in production now, we’re running live performances every week. It’s called The Heist, and it’s a tense, exciting, and action-packed piece where our VIP plays a bank teller who gets caught between a notorious bank robber and the undercover cop trying to take him down. We have seven live actors, a stage manager, and a live music composer. The seven actors actually change roles and avatars during the performance so our VIP actually engages with an even larger cast, around fifteen characters in total. 

The feedback we are getting is extremely positive. While we are a low-budget, boot-strapped production running mostly on faith, passion, and sweat-equity, we are able to provide an incredibly unique and fun experience for our VIPs. Many are overwhelmed by the experience: it’s very intimate and personal and it’s not anything like watching a film or tv show: it takes energy, it takes participation, it’s an active story experience. Similar to a video game in that regard: you are “on” the entire time, you can’t really tune out, get up and go get a drink, or check your phone: you’re live, on-stage, performing with a group of people.

Everyone does it differently: some prefer to be fairly passive, they kind of let us run the thing, they do or say what they need to but seem to enjoy letting us run the show. Others are very interested in exploring the interactive nature of it: they really get into the role-playing, they create their own little improvised mini-scenes, and they try to see how they can change and steer the story. And then we’ve had a few who really seem to feel free and wild and try to push us and almost ‘break us’: they are very curious what the outer limits are, what we cannot do, and they enjoy that challenge. And since we’re learning too, each performance teaches us. We have group post-mortem talks about how it went, what the VIP did or said, how we responded, what we might do differently next time.  We’ve had VIPs ask if they can go through again: once they kind of understand how it works they get excited and want to see what they can do differently next time. 

To answer the common question about how this project might be worse than a 360 pre-recorded video, yes, this is very complicated. I’ve directed plenty of challenging projects for film and theater and television but nothing I’ve ever done comes close to how difficult the MetaMovie project is. One issue is resources: I’m funding all of this on my own, so I just don’t have the financial resources to make life easier: my cast is essentially working for free, as is everyone else on the team. So I can’t exactly get mad when someone can’t make a performance or when some 3D models take a long time to create.

We’re running everything on the High Fidelity social-VR platform, which is still in Beta and things don’t always work the way they are supposed to. Our cast and crew are located literally all over the world, so just getting us all together in the same virtual space at the same time can be a challenge. And certainly doing a live performance is much less efficient than recording something and distributing it. In this way, we are closer to live theater, which is complicated and difficult and yet it thrives across the world because audiences crave that special energy of a live show and those of us with experience mounting live productions understand how to produce at that scale.

I’m lucky in that I started acting as a child in local theater, then through high-school, then I studied theater at UCLA before going on to my masters in Film Production. So while creating a live virtual performance is incredibly difficult, I know how to do it and the more the cast, crew, and I work at it, the ‘easier’ it gets (although it’s never, ever easy). 

One major difficulty is the question of how to scale. How to turn this experiment into something that can actually sustain itself: turn a profit, and reach audiences larger than one person at a time. I’m excited that we actually have some really cool solutions we are playing with. One thing we’ve done is to create a secondary experience, we call it the Firefly Experience. We give a group of audience members little Firefly avatars to wear. These are super cute creatures who are about 8 inches tall, with wings and a little glowing butt. We ask our Fireflies to mute their microphones and not disturb the actors or VIPs. They are free to fly, zip, and zoom through MetaMovie as it plays out, watching the action from anywhere they like: perched on a counter top, floating in the air above the scene, etc. While the Fireflies are not as interactive as the VIP, they give audiences some really cool and unique options: they have freedom to fly around and watch from where ever they want, they don’t have to worry about ‘performing’ and they can even observe parts of the story the VIP might miss – little subtle mini-scenes happening between characters who are not the main focus of the story.

Because while the VIP, Antagonist and Protagonist are all wrapped up in some intense dramatic moment, the other actors are all improvising and staying in character and often those interactions are really cool and fun but only the Fireflies can appreciate them. We’re having a blast as we roll out this new experience and I already have VIPs’ wanting to go back in as a Firefly, and of course, the other way around: Fireflies dying for a chance to be the VIP next time. 

Another thing we are experimenting with is having more than one VIP in the story at a time. That’s kind of the Westworld model: you have this fully populated story-world with so many actors and so many stories that you can bring in large groups of VIPs at once and you have the resources to entertain them. We’re not there yet, but it’s totally possible, and it’s where I see us in a few years. For now, we have another MetaMovie designed for two VIPs at the same time, and we hope to roll that out soon. There are so, so many problems with more than one VIP, the word ‘daunting’ doesn’t really do it justice. What if they decide to go separate ways? What if they stop role-playing and start talking ‘about’ the experience to each other (like a pair of tv watchers might do during a show). The list goes on and on. But that’s why I call this the MetaMovie Project: it’s not even close to being a final product, it’s a work in progress.

I agree with John Gaeta (great interview, by the way) in that I see our current cinema as something we will probably move away from. I personally have a deep love of film but the landscape of cinema is changing, as we all know, and I’m having a harder and harder time replicating the classic cinematic experience of going into theater and having immersive, satisfying experiences. And frankly the lure of the incredible feeling of being inside a gorgeously designed virtual world, where anything and everything is possible, and I’m experiencing the story from inside that world, taking part in that story: it’s totally obvious to me that this type of storytelling will become wildly popular. I mean, I don’t see myself making a traditional film anytime soon. I’m 100% committed to telling stories in the metaverse. 

Our Kickstarter has been very successful and we are really close to our $10,000 goal. But we’re not quite there yet and have just few days to get there. If your readers are intrigued or excited about these new types of stories I’m trying to tell, we would greatly appreciate their support.

And if we hit our goal, our immediate plans are to continue running our 1st MetaMovie, The Heist, while simultaneously producing two new MetaMovies. We have a supernatural drama called Bright Side of the Moon, and a sci-fi horror piece, Alien Rescue. Each new MetaMovie is designed to help us explore specific questions and issues around these types of stories. For example, Bright Side is designed to be an intimate, quiet, performance-centered work. We want to really dig into the VIP/Character relationship to discover just how emotionally powerful we can get. How well can we suspend disbelief? How much can a VIP feel for or care about a character? And Alien Rescue is our first attempt at a two-person VIP experience, as well as our place to see how far we can push our avatar design and performance: we have some really ambitious otherworldly creatures we want to bring to life. 

Ultimately, I hope to use these three MetaMovies as a larger proof of concept to present to investors in 2020, with the hopes of continuing to scale the project, reach wider and wider audiences, perhaps even creating feature-length experiences. And I hope that with the help of all the VR community, this will happen. Thanks to everyone that will choose to support us!

(Header image by Jason Moore)