SIF: Mandala mixes interactive theater with VR, my experience with redirected walking and much more!

Last post about my experience at Sandbox Immersive Festival! (If you lost the other ones, you can find them at these links: episode 1, episode 2, episode 3, episode 4, episode 5).

Today I will talk to you about other 4 experiences: Mandala, that has been one of my favorite installations; Andy’s World, that let me finally try redirected walking in VR; The Tide and The Emergence that were quite innovative. Are you interested? Then keep reading!

Mandala

Mandala has been one of the best VR experiences that I have tried at Sandbox Festival. It has also been the last one that I tried, and I have entered it so late that the Chinese policemen even came to the booth wanting to cut out the power to close the venue! :O

Mandala is an experience mixing interactive theater with VR developed by Sandman Studios, the company that is actually organizing the Sandbox Festival. It reminded me a bit the MetaMovie project that I described here on this blog, but instead of happening all in virtual reality in High Fidelity, it is organized in an installation in real reality. I’m telling you what happens inside, so be careful that this article will contain HUGE SPOILERS: read it only if you’re sure you will never experience Mandala in some exhibitions.

It is an experience for 3 people at a time. I entered the booth with two guys I had never met before and I found a beautiful girl telling us that it was a special experience, a travel into the esoterism and things like that. After that, she invited us to leave all our belongings in some boxes and then we were ready to go. She asked us who was going to lead the group and after some seconds in which we all looked in the eyes of the others hoping for someone else to propose, I decided to be the leader. For this, I was awarded the red color and I was the first one entering the experience. Just to let you know: actually, this “leadership” thing had no actual practical meaning in the experience… but it had a psychological effect. Because I told that I was the leader, throughout the whole experience I thought that I had to lead the group, take initiative and protect my team.

When we entered the main stage, we just found a space with some curtains and some weird props. I was given a backpack PC on my shoulders, a VR headset on my face and some OptiTrack gloves on my hands. After that we were all ready for the adventure, the VR experience started.

We three found ourselves in something that resembled an old temple. Every one of us looked like a simple silhouette avatar, a ghost, of a precise color: red, blue and green. Every one of us could see the other ones, could use the hands thanks to the tracked gloves and communicate via voice (because we were very close to each other). We started, pretty confused about what we had to do: in this experience, no one gives you the instructions of what you must do. So we started discussing what we were supposed to find in this old temple. And we started exploring the temple with our eyes and our hands.

It was interesting that Mandala mixed the virtual and the real world. In the virtual world, we could clearly see at the center of the room a bowl full of sand and when we got there, we actually found a bowl full of sand. It was fun playing with the sand both in VR and in real life.

After some investigations, I understood that every one of us had to push a symbol written with his own color (e.g. I was red and so I had to touch a red symbol next to the sand). After that we all performed this operation, some animation happened in the VR world and the visuals changed. Then, we got stuck again. Until suddenly, we heard a voice behind us.

Looking there, we saw a monkey-man. Was it real? Was it only virtual? Was he another participant? We had no idea. The monkey man started saying that he was very afraid of us because we were hungry ghosts (oh the irony of telling the SkarredGhost that he’s a ghost!) wanting to eat him. At that point, we were confused about what to do. Until I thought “hey, come on, this is my VR adventure, I am the leader… let’s have fun!” and started howling like a ghost (the (skarred)ghost howls while experiencing something to review for The Ghost Howls! This moment was like when Peter Griffin hears the titles of the movies…)

The monkey man started acting scared and so my buddies started pretending to be scary ghosts as well. The monkey man reacted to our behavior perfectly showing himself always more scared. Then he proposed us to spare his life for a moment to talk about something very important. I answered I was not sure, because I really wanted to eat him (I am an evil ghost!)… but in the end, we agreed and so the adventure could go on.

What happened was cool: the monkey was actually one of the actors of Sandman Studios and he was improvising in the classical style of interactive theater. He reacted to what we were doing (acting as scary ghosts) but then he smartly put us in the binary of the story again. And since I set up a funny mood, he continued in that style, being continuously funny with us as well. For instance, at a certain point, the monkey man introduced himself to us all and shook the hands of my two pals, but when it came to me, he refused to do that, since I was the bad ghost that wanted to eat him. It was overly funny.

After a while, he teleported us all in another world on the clouds and our virtual visuals changed completely. Where there was the bowl full of sand there was actually a tree stump now. It was great because when I went to touch it, there was no bowl and no sand anymore: someone, exploiting the fact that the VR headsets were occluding our visuals, had transformed that prop in another one. At that point, there was a stump in the center of the room, both in the real and virtual world. This has been one of the things that made me love this experience: without you noticing, it tries to keep the immersion strong by keeping coherent the relationship between the real and the virtual world.

A bit later on, the monkey hid in the stump and an evil giant came to find him. He asked us to reveal him where the monkey was or he would have killed us. Of course, I answered him “he went that way”, pointing a direction completely different from the one of the stomp. The evil big creature understood I was mocking him, so he menaced me to kill me using a long virtual spear… that was actually also physical, because when he pointed the spear towards me, I could really feel it on my body. Even if I knew that everything was fake, this gave me some discomfort.

As a team, we discovered that we could defeat the creature by grabbing the eyes that were floating all around him. When he was defeated, the monkey came out again, thanking us for having not betrayed him. Then he sat down with us and we talked about some philosophical things of life. After that, we all hugged virtually and physically, the monkey disappeared and then the experience ended.

I loved Mandala for these reasons:

• It mixes two different elements like virtual reality and interactive theater;
• It is an experience that you make with other people, and with these people, you develop a bond because you have lived a magical experience together. You can make new friends;
• You are the one that sets the mood of the experience. We started behaving in a funny way, so the actor of the monkey acted in a funny way as well, and so we had a great time together. I laughed a lot while living Mandala;
• Since it is improvised, every time you live it, it may be different from the other ones;
• The experience feels more realistic because it doesn’t follow a fixed binary: yes, there is a fixed story that you live, but you can modify it a bit with your behavior, so it feels more real;
• It mixes real and virtual props in a great way and this increases the realism of the experience;
• It has a very positive mood, with the monkey that wants you to think about the important things of life.

The only two downsides of the experience have been:

• It mixes various elements of Asian culture in a non-coherent way;
• The tracking technology was not working that well: the green ghost disappeared at a certain point of the story! The creators highlighted that actually, the installation was still a prototype, so this kind of problems was still normal.

I liked a lot Mandala, it was a very positive experience that I lived, and when I think about it, a smile always appears on my face.

Andy’s World

Andy’s World is an explorative experience that you make with another person. There is a story that takes you to various places… and I would really like to tell you what it was about, but it was all in Chinese, so I actually have no idea! For this reason, I will just tell you about the technical stuff.

Andy’s World is an experience that lets you navigate inside lots of different environments: spaceships, environments full of lava, of ice and other cool stuff of this kind. The graphics are good, but lack some kind of “coolness factor” that would make me define them as fantastic. They seem a bit standardized, I don’t know how to explain this.

You live Andy’s World equipped with Vive headsets connected to backpack PCs that you wear on your shoulders. Thanks to Leap Motion tracking, you can actually see your hands in VR and you can use your hands to activate some objects that make you go further in the adventure. You have no hand controllers, so the only way of navigating the environments is walking.

The experience is all about showing you how it is possible to explore a complex environment by just staying in a 5mx5m booth. So, you have the impression of moving forward a lot, going through curved corridors, moving up and down on moving platforms, entering spaceships, etc… while actually, you are always only moving inside a small booth.

One of the mechanisms through which this is possible is redirected walking. Basically, the VR system rotates your visuals slowly across time so that you think you are going straight while actually, you are physically moving in circle around an imaginary point. This way, you can live VR experiences that use big VR environments by just staying in a little physical space. The company implementing redirected walking in Andy’s World is a startup of Shenzhen’s ViveX venue that I met when I attended the VEC.

I had always wanted to try redirected walking because it always seemed black magic to me: something that tricks your brain in believing that you’re walking an infinite corridor while actually you’re moving in circle like an idiot is surely fascinating. And some friends trying it actually told me that it is an unbelievable experience.

Sadly, trying it, I haven’t come out much excited by it. Maybe it was because the volunteers following us during the experience made us walk too fast, maybe it was because the heavy graphics made the program run sloppily, maybe it was because the corridor we had to follow virtually was not straight but curved, but my body realized very well that something strange was happening.

I can’t explain well what I felt, but I’ll try doing my best: I felt my body very unbalanced… like if I was walking with one leg don’t working very well, like if I was limping a bit, with my body posture that was not coherent with what I was doing. I had a bad balance and once even had the sensation I could fall to the floor because of it.

The reason is this one: the redirected walking may trick your visuals easily, but what about your body posture? If I walk straight, my body is in a perfectly symmetrical balanced position, while if I walk in circle, I put more weight on the leg that is closer to the center of the circle. If I’m seeing myself going straight, but actually I’m walking in circle, my body will have the asymmetric balance of the walking in circle, but the visuals of walking straight. This sensor mismatch makes my brain and my balancing system go crazy.

Surely there are tricks to reduce these bad sensations (like making the users walk slowly or walk along a circle with a big radius), but this proved to me that redirected walking is not a perfect solution for all VR experiences requiring a lot of space. Sometimes it can be used, some other times not.

The Tide

The Tide is a storytelling experience about some big evil fishes that start populating the seas and the costs of Japan. These fishes are big like whales and they make ships to sink and eat people, so they are not very kind.

The experience follows the adventure of a young fisherman that is on a ship that gets sunk by these evil whales. He got almost eaten. Mysteriously, he finds himself alive on the coast and then he starts trying to understand how to continue his life after this unlucky episode. The experience ends with a guy inviting him to follow him to save his life and then a “to be continued” appears.

I admit that I found the story a bit weird and also very short, considering that it is just the beginning (probably it was only the episode 1). What I found intriguing instead was the graphical style. The Tide was real-time rendered, but was all drawn in comic-style. It seemed to me to be inside a Japanese manga.

And this analogy revealed true even for what concerned how you lived the experience. You could see just a tiny bit of action, and then you had to press a button to make the experience go on, with a new point of view and another little shot of action… exactly as if you were reading the comic frames one by one. That was cool.

The Emergence

The Emergence was a quite original experience. It is very complicated to define it, so I will copy-paste the definition of the authors:

Emergence is an open-world environment, expressing the primal desire to maintain your individual identity whilst being part of a crowd.

As you immerse yourself in a crowd of thousands, shafts of light beckon you closer. As you touch the light, the environment – its atmosphere, its gravity and the choreography of the crowd – transform in powerful ways, continually challenging your perception.

Showing 5000+ intelligent human behaviours, Emergence offers a powerful, unique experience of a crowd, that is only possible with the latest graphics technology.

It was interesting seeing these scene in 3rd person with all this big crowd of human beings behaving in a weird way and all following me (an enlighted character) or other humans. I could move with the thumbstick of my controllers. I could see some light rays, and if I touched them, everything changed: all the colors around me changed, all the people started behaving in different ways. With the last ray, we even all started flying! It was cool. A bit weird, but cool.

It is not something that I would do for hours, but the experience was quite short, so the duration was perfect to always keep me interested in it.

---

And that’s it! This was my last post about the Sandbox Immersive Festival! It has been a great event, and I really hope to attend it also next year! I want to thank especially Eloi, Eddie and Coco to made my participation possible this year… I felt really enriched by it.

I also hope that you loved performing this journey into VR storytelling with me… and if this is the case, subscribe to my newsletter to not miss my next posts!