Google storytelling features: Youtube VR180, HeatMaps & StorySpheres

Google is surely a believer in virtual reality: it is the inventor of the most sold VR headset (Cardboard); the creator of one of the two high-end mobile headsets (Daydream) and of a new generation of standalone devices (Daydream Standalone); it is also promoter of 360 videos and its popular video portal Youtube now fully supports 360 videos… even with Snoop Dogg’s commentary.

Some days ago, Google has announced some new features for its storytelling facilities. Let’s see them together.

The VR180 format

First of all, it has introduced the new VR180 format. “New” is a funny word, since it is a format that adult virtual reality platforms employ since a lot of time, but it is surely new for Youtube. What am I talking about? Basically, VR180 is a format which:

• Lets you see only what you have in front of you: so you have 180-degrees-only vision. If you’re questioning what can you see in the other 180 degrees… well, usually it is all black there. Of course this is an immersion breaker, since your eyes can clearly see that the simulated experience occupy only 180 degrees of space… it is like you are in the outer space, having a window through which you see a virtual world. It’s surely not realistic.
• Is completely stereoscopic: every eye has its own dedicated image, so the result is a 3D experience. You can perceive which objects are nearer and which ones are further, so the immersion is increased, since there’s a clear sense of depth of the environment surrounding you.

Usually standard 360 videos have the advantage of having the full dome of 360-degrees of vision, but they’re flat, with no depth. The brain is able to infer the depth of objects by their size, but the experience is far from realistic. Honestly, I don’t like 360-degrees videos, because watching them i feel like being inside a sphere with a texture projected onto its surface… I don’t feel really like inside the action, I don’t feel the presence. I like VR360 videos (like Ashes to Ashes), that have the full degrees of vision plus the 3D of depth, but recording them is really hard and requires expensive cameras (Mandt VR director has provided me a comparison of such cameras, if you’re interested). Unluckily, due to these problems, 90% of 360-videos are 2D ones.

VR180 has the disadvantage of not having the full 360° vision, but:

• It’s stereoscopic… and trust me: 3D makes everything immersive. Without depth you don’t trust the environment around you, so having 3D is a must;
• No one actually looks back: some times ago I’ve read some interesting statistics on 360 videos and the harsh truth is that almost no one gives a heck about what’s behind him/her. Turning the head is too tiresome, so unless there are some compelling reasons to do that, everyone just keeps looking forward. So not having 360, actually is not that great loss;
• It helps the director to have a clear idea of where the user is looking: one of the greatest problems of VR movies director is focusing the attention of the user: what if an important part of the story is happening behind me but I’m looking forward? With 180 you’re sure where the user is looking at;
• It’s easier to be shot: with 360 the camera records everything around it and this can be an issue, since it records the director, the tripod, the movie crew, everything. With 180 it’s just a forward recording, like in traditional cinema;

VR180 in my opinion is the trait d’union between traditional filmography and future VR one. It is something that offers depth and head tracking, but with rules similar to the ones used by directors at present time. It’s like 3D-cinemas. It can be the bridge between current movies and the ones of the future.

Google has provided us a playlist of videos employing this new format. Honestly I’ve had some issues playing them due to the classical WebVR problems and other strange stuff: on my PC the videos showed as standard Youtube onse, while on my laptop they show in stereoscopy…. mah. Anyway, I’ve managed to give it a try.

At the contrary of adult experiences, I’ve noticed that head tracking is disabled. This means that the video is sticked to your eyes and you never realize that your field of view is limited (you never reach the boundary of your vision space, since you can’t rotate your head) and you never see all the distortions (usually the experience is very neat if you look forward and the more you rotate your head, the more it gets distorted), but this comes at the cost of motion sickness (seeing always the same image regardless of the orientation of your head isn’t something our brain likes that much). My bet is that they’ll enable it in the future and the above GIF seems to confirm this.

A final news: this new format will support live streaming, so we’ll see a lot of VR streams in the future!

VR180 Cameras

Google knows that has no sense enabling this new format if it doesn’t help video creators to shoot this kind of movies. So it is partnering with companies like Lenovo, LG, and YI Technology to create cameras able to shoot this kind of videos out of the box.

On Lenovo website we can read this description for the upcoming camera:

Capture your own VR pictures and VR video with a simple point-and-shoot camera, which can be easily shared and livestreamed with your friends through YouTube. You can experience your content through Lenovo’s standalone VR headset enabled with Daydream.

As you can see, the idea is clear: make shooting and streaming of VR180 videos super-easy. It should be something easy to do, as it is today recording movies with our smartphones. And in fact my idea is that in the future these will be integrated in our smartphones, so we’ll be able to shoot VR videos with the device we all have in our pockets.

As you can see, Google is trying to build an entire ecosystem, encompassing all stages from videos creations to fruition. Virtual reality is a new field where there are no standard tools to produce content, so this is fundamental to make the technology to succeed. Do you remember my interview with PornFoxVR founder? He talked about the fact that at present times there is not the right hardware for shooting their movies, so they all resort using a couple of GoPro, one next to the other. This duct-tape solution works, but it’s far from perfect. Google is planning to offer exactly something like that, that resembles two GoPros side-by-side, but is a complete and polished product. Look like the PornFoxVR rig and the mockup of the Google-Lenovo cameras are similar…

Youtube VR Heatmaps

VR video creators will be able to see which are the points where the users are focused the most, thanks to Youtube Heatmaps. Google has announced this feature some days ago: it will be available for “360-degree and VR videos with over 1,000 views” and will let the video creators understand which are the areas of the videos where the users pay the most attention.

A reddit user complained about the fact that his videos, although they have far more than 1000 views, still haven’t this feature available. The suspect is that Youtube is slowly delivering this new feature to its users, to be sure that it hasn’t big issues.

This tool is interesting because:

• Lets video creators learn how their spectators watch their videos: so, for instance, if the video has been conceived so that the users are focused on the action and on the contrary all of them just looks at the cute marmot eating biscuits in the background, well… this means that something is going wrong. The director can learn from his errors and improve his next videos thanks to these heatmaps;

  

• This will let us understand better the generic behaviour of a VR videos viewer. Google has already learned some important lessons, like this one:
  

  The defining feature of a 360-degree video is that it allows you to freely look around in any direction, but surprisingly, people spent 75% of their time within the front 90 degrees of a video. So don’t forget to spend significant time on what’s in front of the viewer.

  Remember the VR180 format announced above? Well, I have some strong suspects they invented it because of this statistic they extracted from 360 videos published on Youtube…

• It will make VR advertising easier. Google is and advertising company, so for sure this is something that interests them. First of all, it can be used to understand how much an advertisment is effective: when we’ll have VR advertisment videos before the videos we want to watch, thanks to heatmaps Google will be able to understand what the user has looked at during the ad and so understand if it has been effective (for instance because the user has looked at the brand) and so charge more the publisher.
  I think that this could also become more scary: Google could analyze what I look inside videos and then offer me more targeted advertisement on this. Let’s all remember that in the future we’ll have pupil tracking inside headsets… Ok, I don’t want to think about a distopian future and alarm everyone. But surely analyzing what every user is looking at in a VR video is great for advertisement.

StorySpheres

Some days ago, I went to Google Maps to see the timetable of buses next to the office and Google proposed me to enter a 360 experience. Could I refuse it? So I entered the magical world of Google Story Spheres.

This experience regarded the park of Uluru-Kata Tjuta, in Australia. That is the Ayers Rock, for people not practical of strange Australian languages.

It seems that the one of the Ayers Rock is a very sacred zone for the Pitjantjatjara people of Central Australia (source CNET), so Google had to spend a lot of time to make the recording in a way to not offend the locals. Apart from creating a standard Google Maps reconstruction of the place, they decided to also create a Story Sphere.

StorySpheres is a framework to create a virtual tour in a way to communicate a story. It is basically like a collection of 360 photos that have some trigger points to go from one to the next (like the arrows in Google StreetView) and some interest points that contain audio streams that convey the story. There’s also background music for each location. If you’re thinking that is something like InstaVR… well, yes, it is. But it is more dedicated to storytelling than to make a mere virtual tour.

StorySpere isn’t actually something new: this 2015 article already talks about them. But I think that now it is the time they could be used for something bigger. I mean, the technology of StorySphere could be really powerful if embedded into other VR technologies and used through VR headsets:

• VR Youtube videos could have tag in specific places of the dome and this tags, if triggered, could point you to another video, to some in-depth explanations, to some ads, and so on;
• Google StreetView could use them to show interactible triggers for the shops: so if I looked at the point-of-interest of a shop, I could be redirected to the page of that shop. Furthermore, people could also make stories using this feature. For instance a boy could make a storytelling experience for his girlfriend taking the StreetView 360 photos of the places where they met the first times and then say something romantic that she can hear if looks at the the trigger points.

I’m curious what StorySpheres will become in the future… in my opinion StorySpheres per-se could die, but the technology behind it can be really useful for Google VR experiences. In the meantime you can learn how to create a StorySphere from Google instructions.

And that’s it for today… what’s your opinion on these Google’s new tools? Let me know in the comments!