When browsing the VR news, it happens that I find something about “holograms”. In these pieces of news, this term is not used in the general-purpose meaning of “augmentations”, that we all know and use when talking about Augmented Reality, but in its actual physics meaning. For instance, in this blog post by Facebook, the researchers mention the innovation of using a “holographic lens” instead of a classical optical lens, and how it should be able to reduce the size of an XR headset.
I confess to you that I’ve never understood completely the meaning of this world, and the articles that mention it usually don’t help, because they just say that holograms are the ones used to produce that special 3D effect on banknotes and credit cards… but honestly, I’ve never had clear how the tiny silver film showing me the logo of my bank can be used to create a lens or a 3D lightfield, for instance.
Today I decided to take the bull by its horn and try to have a basic understanding of what a hologram is, and why it should matter in XR… and as always when I learn something new, I’ve decided to spread the knowledge by telling you what I’ve just learned. Are you with me?
What is a hologram?
We can say that a hologram is the encoded version of a lightfield. Or that it is a special kind of photography that can encode completely how light scatters around an object, so that not only the intensity of light (like a classical photography), but all the properties of the light rays that come from that object are recorded into it.
Let me make an imaginary example to explain this better. Imagine that you take a photo of something with a special holographic camera, for instance, a beautiful flower garden. Then you take this picture, that is 2D, and looking at it, it is like you always had a 2D window through the 3D garden wherever you are. It is a flat picture, but looking through it from whatever point of view you can really feel all the colors, the reflections, the depth cues, the parallax, of the garden you have taken the photo to. A permanent window to that garden. This is what a hologram can be.
Does it look like Harry Potter’s Magic to you? Well no, it’s science.
The standard way of creating holograms
So how to create a hologram? There are many ways, but I’ll just detail here the classical one, the one that Wikipedia and most of the other websites use to explain the process.
You have a laser light (with a clear defined frequency and orientation), and you use it to illuminate an object. When hitting the object, the light scatters all around its surface, and the modified light rays go around in the space. If we manage to capture these light rays that reflects and refracts out from the object, onto a light-sensitive film, we obtain a traditional photography. Remember that what we see in our everyday life is not how objects are, but simply how the light rays of the sun (or artificial lamps) scatter around all the objects around us, until they arrive into our eyes, or a camera that record their intensity.
To make a hologram, we have to make something slightly different. We split the original laser light in two thanks to a beam splitter, and part of it goes towards the objects (where it scatters around), while another part remains untouched and gets directed through some mirrors towards the film that should record the hologram (imagine this like a special “photography” film). This special film gets so hit at the same time by the original untouched rays, and also from the rays that have hit the object and are scattering all around the space. On the film arrive so two kind of waves, one from the original beam, and the other one from the scattered beam, and when these two waveforms “collide”, they form an “interference pattern” on the film we’ve put to record the hologram. An interference pattern is a figure that gets formed by two waves interacting the one with the other: imagine when there are two sounds in your room that play at the same time. When they arrive at your ear, these two soundwaves combine and your ear hears something that is the combination of the two (and that maybe you perceive as an annoying noise). In this case we have two visual waves, but the reasoning is the same: these two light beams collide, and they form on the sensitive recording film black and white ripples that have certain shapes and colors depending on the characteristics of the rays that collide there (a bit like your eardrum vibrates differently depending on the two soundwaves that enter your ear in the previous example). The film is made with a special material that is able to record this interference pattern created by the various rays from the two different beams that are interacting on the film itself. It is like if the film was able to record the image created by the sum of the two types or rays that get cast onto it: the reference one, and the object one.
So after you have done this correctly, you have a silver film with a nonsense interference pattern drawn on it. If you watch it, you see nothing meaningful. Pretty useless isn’t it?
Actually no. I don’t know if you remember this from high school as well, but in optics, the processes are invertible: so if I cast a ray (let’s call it A) into a lens, I obtain a skewed diffracted ray (let’s call it B); and if I send the skewed ray B back into the lens, I obtain again the ray A. So, for a similar reason, it happens that if I have the reference beam B1 and the scattered beam B2 from the object, I obtain the diffraction pattern D on the recording film (This is what we’ve done at the previous step). And if I send the reference beam B1 on the pattern D on the film, I’m able to re-obtain the object beam B2. Let me explain this better: if I cast the exact same reference laser beam with the same orientation of the previous step directly onto the hologram that I’ve recorded, the film will emit exactly the light rays it had recorded from the original object, with the same intensity and orientation they had when scattering out from it. Illuminating the hologram with the same light rays of the reference beam, the hologram will emit the rays exactly as when they were recorded from the object from the previous step, as if the object was there and was illuminated. So I can see the object through the recording film, exactly as if the object was there, focusing on its depth cues, moving my head and enjoying its parallax, because the holograms will emit excactly the same “scattered rays” the object would emit if it were there. For my eyes, it is like if the object was really there. It is the magic window I’ve talked to you in the previous paragraph.
On the recording film, I have the hologram of the original object/scene. It has recorded the lightfield of the scene it had in front of it, and it is able to re-create its vision if it gets lit with the original laser with which the whole scene had been illuminated. It is like a photograph of a lightfield, a 2D window into a 3D world.
Holograms are pretty mindblowing. One of the properties that makes my mind to explode is that if you cut a hologram in half, you have still the whole picture. If you cut a real photograph in half, you lose half of the picture, because you lose the color information that you have stored in the photo. While if you cut a hologram in half, you have still the whole scene in both halves. A simple way of explaining this process is imagining the holograms like a window to a 3D world: if you look outside your window, you see the scenery in front of your house. If I close half of your window, you can still see everything that is in front of your house, you just have to move your head more to see everything, you don’t lose half of your data permanently. The same holds for holograms: you still have all the information of the scene you have recorded, you have just a smaller window into it.
Why they matter for XR?
Holographic displays
After this explanation, I think you already got one of the reasons why holograms matter. Our VR headsets are a window towards a 3D world, too, but they have just simple stereoscopy and no possibility of focusing on objects at different depths. If we could have a holographic display inside them, we could see reality in a new way, with complete depth perception and focus, parallax, etc… That would be incredibly realistic.
But here there are many problems coming to stop our dreams. First of all, how to create a holographic display, and then how to create the holographic content to show into it. Above I described to you a method employing lasers and silver films: the classical method of creating a hologram is good only for static objects and a single hologram requires a quite long time to be created. Here we need to create content on the fly at 90Hz for elements that are computer-generated, so the context is completely different. It is a bit like if we had to switch from the classical old photography on film to 4K Blue-Ray animation movies on our laptop.
The bad news is that there is no usable technology at the moment. Samsung is experimenting on true holographic displays for smartphones and in 2020 it claimed that the technology guaranteed either a viewing angle of 0.25 degrees in a 10-inch display or 30 degrees in a 0.1-inch display. As you can see, this is not much usable, not only in XR, but also in usual contexts. Also consider that at the moment most of the holographic displays are monochrome (usually green), and to have a full RGB display you would need to triple the required hardware (have 3 lasers for the 3 main colors).
The good news is that there is an ongoing big research in the field. In the last years we had some interesting news, like Samsung being able to increase 30-fold the field of view of a holographic display, obtaining a 10.1 inches prototype display with a viewing angle of 15 degrees at a viewing distance of 1 meter. Facebook had a completely different approach employing scattered rays, and it has been able to obtain a wide FOV inside a decent eye-box for a holographic display to use in a XR headset as well. And at MIT, a researcher has been able to generate in real-time 3D Holograms even on a mobile phone by using artificial intelligence.
It seems that slowly all the pieces are coming together to start having a full holographic system in the upcoming years. I don’t think this is going to happen soon, but in the canonical 5-10 years that we use to predict whatever thing we have no clear idea of what is the right timeline 😀 (Yes, in 5-10 years I will be rich!). Jokes apart, many researchers define 10 years as the limit time to have holographic displays, and I really can’t wait for this to happen. It would mean real 3D vision on all 2D displays, or glasses powered by holographic technology that gives you a realistic mixed reality with full depth cues.
Holographic lenses
In the beginning of this article, I mentioned a research paper by Facebook where the researchers talk about holographic lenses and that left me puzzled the first time I read it. Well, with our new expertise, we can now understand how a holographic lens works (and it is a genius idea!).
Let’s go back to our initial classical way of recording a hologram: the famous recording film records how lights scatters around the object that we are recording. We can see this “scattering” as the way through which the object “modifies” the incoming light: light starts in a way, then it finds our object, and then it changes intensity and direction, and our hologram records it. Then, when we light the hologram with the reference light, the hologram film emits the light as the object emitted it. That is, if I illuminate the hologram with the reference light, I obtain the refrence light as the object transformed it: the input is the reference light, and the output are the scattered rays, that is how the original object transformed the rays. It is a bit like if the hologram mimicked perfectly how the original object modified the light. The hologram is a perfect visual substitute of the original object, and that’s why we can see it as if it were there.
Now, imagine if the object that we record with the hologram is not a solid object, but a big optical lens. The tiny hologram would be able to record how the lens modifies the light, so we would be able to substitute the lens with a hologram of its. I know, it’s a bit mindblowing, but if you think about it for a moment, it completely makes sense. The hologram has recorded all the properties of the light that came out from the lens, so it can substitute the lens itself. It can mimick perfectly how the lens modifies the light. And since this holds also for big lenses, you can substitute the complex optical system of a headset with a thin hologram foil.
Of course, all of this is in a research stage, but yes, it is mindblowing… and a promising approach for the future.
A last caveat
I’ve detailed to you what are the real “holograms”, that is what science defines as “holograms”. You may find the world used in many other contexts to mean a 3D element, a 3D display, an augmentation, a Pepper-Ghost illusion (like the one that was used to show Tupac singing with Snoop Dogg at Coachella), but while these are all amazing things and we can call them “holographic” while speaking among us, they are not true holography according to physics.
Further references
Big kudos to Wikipedia for having a very well-written page on Holography. If you want to read more about holograms, I suggest you to head to it directly.
I hope you enjoyed this journey into holography as I did. If you are into physics and optics probably you have noticed some simplifications and some imprecise statements, but the purpose of this article was not to write a detailed essay, but to convey a basic idea into holography explained in a simple way and I hope that it can help many people in understanding better this fascinating field of research.
If this has been the case, please share this post with the other XR enthusiasts and subscribe to my newsletter to not miss my upcoming posts!