How to show a video in a Texture with Unity on Android

Yesterday I’ve been asked about how to display a video inside a hobbist AR application in Unity, so I’m writing this tutorial to help people having the same problem.

You know, in Unity to display a movie inside a texture, you have to use a MovieTexture. MovieTextures are cool and work flawlessly, but… they don’t work on Android. The only thing that you can do in Android is showing the movie full screen using methods of HandHeld class. But… if you are going to develop a mobile VR or AR application and you want to show the movie inside a quad (maybe because this is the screen of a TV inside your game), this is absolutely useless. So, how can you do it? I can give you some hints.

1. Develop your own solution. This is the preferred answer if you have time to employ in doing it and/or you want to learn how this stuff works. Or if this is a critical piece of your applications, so you develop it once for all and then you use it forever. Here you have to find a way to open your movie file (OpenCV can be an option), then read every frame and put it on a texture using Unity methods. Of course some optimizations have to be performed or everything will come out super-laggy (very low FPS).
2. Buy a plugin that already does it on the Asset Store. Looking a bit on it, I found for example this one (but I had problems when I tried the free version) or this other one (haven’t tried by myself). As you can see, they seem quite professional solutions, at a high price (at least 60$). Use them if it is necessary for your job to have videos shown as textures in Android: you spend 60 bucks but you spare a lot of time that you would have employed in scenario 1.
3. Use a quick-and-dirty solution. Once, when I had only Unity free, I used this trick to display a movie. Basically you save the movie as a set of pictures (a picture for each frame) and then you read and show all this images, one after the other, as textures of your object. Of course this solution is far from optimal… it requires a big disk space, doesn’t provide you with video sound and it is slow in displaying the images… but it can be ok for some usages.
4. Steal video mechanics from Oculus… this is the approach that I will describe you with greatest details, since I think that it’s very interesting. Oculus has provided us a sample script to show videos inside a quad both in Oculus Rift and in Gear VR… so why can’t we re-use it? I should warn you that their script is very rough and has some issues, but for some usages (like prototyping or making programs just to have some fun with friends) it can be good enough.
   I’ll show you how use it with Oculus + Gear VR, but it can work even without Virtual Reality… Ready? go!
   1. First of all, download Oculus Samples Project from here. Unzip everything. Let’s assume that you have this project in folder C:\OculusSamples.
   2. Now, go to Unity and start a new project. This is the one where we will use the media player. Import Oculus Utilities into it and set everything for VR usage, like setting the Virtual Reality supported flag (you can skip all these steps if you plan to use the system without Oculus VR… but as I’ve told you, I’m making this tutorial for it). Assume that we have this project in folder C:\VideoSample.
   3. Copy C:\OculusSamples\Assets\Plugins\Android\libOculusMediaSurface.so to C:\VideoSample\Assets\Plugins\Android\libOculusMediaSurface.so . DO NOT CHANGE the destination folder of this file.
   4. Copy all folder C:\OculusSamples\Assets\SampleScenes\Rendering\Movie Player to C:\VideoSample\Assets\MoviePlayer\ . You can change the name of this folder if you want.
   5. Copy C:\OculusSamples\Assets\StreamingAssets\HenryShort.ogv and C:\OculusSamples\Assets\StreamingAssets\HenryShort.mp4 to C:\VideoSample\Assets\StreamingAssets\HenryShort.ogv and C:\VideoSample\Assets\StreamingAssets\HenryShort.mp4. DO NOT CHANGE the destination folder of these files.
   6. Remove from the scene the standard camera and substitute it with a OVRCameraRig prefab from Oculus plugin. (again, this is useful only if you’re using it in VR).
   7. Add a quad in the scene. This will be our screen onto which we will show our video. Again, it is not necessary to use a quad, you can use even a cube or whatever model you prefer, but I think that if you have a movie, you want to show it on a flat surface.

      

   8. Since we’re showing a HD video, scale the Quad so that x/y factor is 16/9.
   9. Set the Quad material to be C:\VideoSample\Assets\MoviePlayer\Materials\movie_surface_hd_mat . This is the material that will have as texture the frames of your video.
   10. In the Mesh Renderer behaviour of the Quad, Set Cast Shadows to Off and unflag Receive Shadows. With shadows on, older plugin versions failed to show the videos.
   11. Add the MoviePlayerSample behaviour to the Quad. As movie name, set HenryShort.mp4
   12. Add an AudioSource to the Quad.

       

   13. Ok, now try the scene inside the Editor. It should work. If it does not work, re-check everything twice. If you see a low quality video, it’s ok… Oculus has provided the OGV version of this Henry short movie in poor quality.
   14. Save the scene and build for PC. It should work again.
   15. If everything works on PC, let’s try on mobile, where lots of things are more tricky. Switch to Android platform and build as you build all Gear VR apps (so remember to change the bundle identifier and all this stuff). Deploy and run it… if you can see and hear Henry, you should feel happy!
   16. I think that standard video is not the one you want to show. So, delete Henry videos from the StreamingAsset folder and add your video. Here things become a little tricky. Since MoviePlayer is only a sample, it is not a full solution for your needs. You have to:
       1. Provide the video in two formats: OGV, to be used on PC; and MP4 to be used on Android. For video conversions, you can use any tool (I use XMediaRecode)
       2. Both videos have to be in the StreamingAssets folder and have the same name. StreamingAssets is necessary because it guarantees that your application will retrieve the asset on every platform (more info here);
       3. Make sure that the videos (at least the one that goes on mobile, where every little problem makes the media player go crazy) have this format: Video: MPEG-4 AVC / H.264, YUV:4:2:0 Planar 12bpp; Audio: AAC 48Khz 2 Channels. Again, you can use XMediaRecode to convert your videos to this format. Other formats may work, but I do not guarantee it (one video of mine had audio at 44Khz and the system went completely crazy). So have a try with your format and if it does not work, convert to the one I provided you;
       4. Set the full name of your mp4 movie as video name in the MoviePlayerSample behaviour (as you did in Henry where you wrote HenryShort.mp4) .
   17. If you want to modify other stuff (e.g. put videos in subfolders of StreamingAssets) you have to tweak directly the MoviePlayerSample script. Good luck with that 😉
   18. And that’s it… deploy and everything should work with your video too. As I’ve said, this method should work with or without VR!

       

Hope this little documentation could help lots of people having problems in showing movies in VR/AR applications! If this helped you, please like and share! 😉