The sense of taste in virtual reality

When I was in Immotionar, my startup partner Gianni made a lot of presentations about what virtual reality is and usually he came to this slide when he reminded how VR is not all about visuals, but there are all five sense involved, including smell and taste, that are senses almost ignored in current simulations.

total vr immersion taste
The five senses involved in total immersion (Image by Gianni Rosa Gallina)

Usually he said that there are devices to simulate tastes in VR and told the joke “I’m not envious of people beta-testing these devices… I can’t imagine what happens in case of bugs” and everyone laughed.

Yeah, this was a great part of his presentations, because he remembered all the audience that there are out there a lot of things to consider in virtual reality simulations… that even if we all talk about Fresnel lenses and screen resolutions, fundamental part of our sensing of the world are completely missing from current VR experiences. But… these devices that Gianni wasn’t envious about testing… what are they? And how much there are out there? And are there other taste solutions for VR? Today I decided to try to answer all these questions.

If you lookup for taste emulation in VR, there is one name that will continuously pop up: Nimesha Ranasinghe. It seems like the best researcher of the world about emulation of taste and if he is not the best, he’s surely a very good one. At University of Singapore, where he works, he has made lots of work about taste emulation and taste communications.

Nimesha has created a very weird device that is able to emulate all the fundamental tastes that our tongue can perceive: sour, salty, bitter, sweet; furthermore can also emulate sensations like mint and spicy. This is possible thanks to a box containing both electrical and thermal controlling units, both terminating with two little metallic foils: putting the tongue between these two electrodes it is possible to feel the virtual tastes. In particular, this professor has found that sour, salty and bitter can be emulated using electrical stimuli (the different taste depends on amplitude and frequency of the electrical wave going through the tongue), while sweet, mint and spicy can be emulated using thermal stimuli (for instance, a cold metal can simulate mint). At the beginning they tried only with electrical stimulation and so weren’t able to emulate sweetness, but afterwards he had the idea of mixing it with heating, with very successful results.

virtual reality taste eating emulation
How to emulate the taste of food using thermoelectric stimuli (Image by Nimesha Ranasinghe)

Here you can see a video of his experiments of taste simulations. Notice how the box used for the simulation is pretty big and cumbersome.

This technology can have an enormous amount of applications. They could be divided in entertainment ones (like making beverages that change the taste while you drink them) or medical ones (like letting people with diabetes taste sweet food again). The final part of the video, when the girl sends a message to the man and then he licks the box makes me think also about possible porn usages of this technology… but this is surely due to my sick mind 😀 .

Nimesha’s researches are going on and now the weird lickable box has been replaced by a more comfortable Digital Taste Lollipop

virtual reality taste eating emulation
The Digital Taste Lollipop (Image by Nimesha Ranasinghe)

which is a connected with a box full of buttons that let you decide which kind of tastes you want to experience. Watch the video below to see the system in action. (You can read more about his work with the Digital Lollipop on his Ph.D Thesis)

His experiments with gustatory technology (just learned this term and wanted absolutely to write it) are not limitated to tastes emulations, but he is also thinking about tastes communications. I mean, if I want to send a tasty SMS to someone else… how can I do? Well, doc Ranasinghe has thought about a protocol called Taste Over IP.

VR taste social emulation
Sharing of taste in a social network. I already imagine how tasty memes could be (Image by Nimesha Ranasinghe)

Basically he has defined an XML format dedicated to taste that serves to define the type of sensorial experience that device has to make feel to the receiver. Of course both the sender and receiver device has to comply to some taste standards (if sour level 1 for my device is mild and for the receiver is ultra-hard, communication is terrible): if this happens, I’m able to send every kind of taste to every people I want.

taste emulation in virtual reality
TasteXML format example (Image by Nimesha Ranasinghe)

Finally people posting foodporn images on facebook will be able to attach also the taste of the dishes they’re photographing, so we can not only see their photos, but also tasting what they’re tasting. I think that all foodbloggers would get mad for this feature!

Some times ago they made an experiment in this sense, mixing the TasteXML with the thermoelectrical emulation of taste, to send a lemonade over the web.

In this case, a sensor detects the sourness and color of the lemonade drunk by the girl and communicates it to a special remote cup owned by the man. This way he’ll be able to drink a similar lemonade to the one of his friend. Notice that in this experiment also color is involved and that the research has tried to also create a more natural object onto which put the mouth (a cup, that is less scary than a box full of cables). Experiments have been pretty positive… but there is still a lot to do, since for instance the participants to this experiment have told that the original lemonade was actually sourer than the original one). But the fact that the emulation has been further enhanced with regard to the previous one makes me think that this work is going through the right path.

This is awesome for emulating taste… but how about the texture of food? That is… how do you emulate that a virtual food is hard, soft, gummy, etc…? Well, researchers of the University of Tokyo Arinobu Niijima and Takefumi Ogawa‘s have created Electric Food Texture System exactly for that. They use external electrodes applied on the masseter muscle, the main muscle involved in chewing, and these electrodes stimulate electrically the muscle so that to make it to contract or stretch to emulate the hardness of the food. Depending on the frequency and amplitude of this stimulation, we can emulate various consistencies of food. For instance they have tried eating cookies while having this device on, and the cookies seemed like gummy bears.

These two different researches can surely work together (and in fact the researchers know well each other), to emulate both the taste and the texture of food. So we can have a completely virtual eating experience. At present time they’re using these devices in a real world environment, but I already imagine having a device like that mixed with virtual reality environments to create an amazing VR banquet experience…

(Well, actually… these ARE virtual reality devices, that emulates only the sense of taste… remember that visuals are just a component of virtual reality experiences and that sight has not some kind of special importance over the other senses!)

Guys of Project Nourished are instead focusing exactly on this. Using virtual reality headsets to create a new kind of eating experience.

Website of the project shows how they want to create a new kind of eating by using:

  • A virtual reality headset to immerse you in a particular virtual reality environment that can enhance your eating experience (e.g. seeing yourself eating on top of a mountain is surely relaxing). Furthermore, this VR emulation can also be used to make you see you’re eating something that is not true: for instance you can be eating broccoli while you see that you’re eating donuts;
  • An aromatic diffuser to enhance smells of the emulated food;
  • A bone conduction transducer to make you hear the chewing sounds of the virtual experience. So, for instance, using the above example of broccoli, you could hear in your ears the sound of someone that eats donuts… this way the brain is more tricked in believing you’re eating donuts, since from the jaw sounds it could reconstruct something of the texture of the food you’re eating (something similar happens with Leap Motion that emulates haptics with sounds);
  • A spoon with which you can eat your virtual food. This utensil obviously is a VR controller and you can see it in VR as a spoon;
  • A glass, that is emulated in VR, too, to let you also drink while you’re in the virtual environment;
  • 3d printed food: these cubes made with hydrocolloid are 3d printed so that to emulate the texture of food: continuing the above example, its surface could be made to emulate the surface of a donut, so that your tongue perceives that there actually is something similar to a donut in the mouth. These molecular food cubes are crafted by specialists to have particular tastes.
virtual reality taste eating emulation
Project Nourished Items (Image by Project Nourished)

Project Nourished is actually a vast project involving a lot of different experiences mixing virtual and augmented reality with food. It is really ambitious and its webpage is evolving over time. Recently they’ve added concept photos of some addons to use Oculus Touch or Vive Controllers as augmented food experiences controllers.

eating in virtual reality vive
Project Nourished addon to eat using Vive controllers (Image by Project Nourished)
Eating taste virtual reality
A fork attached to Oculus Touch? Done! (image by Project Nourished)

But what are the purposes of these virtual simulations? Well, here again we have both serious and less serious applications. Entertainment can be the first environment, but this kind of simulations can also be used to let people allergic to a food to experience it without actually eating it: if I’m allergic to wheat, I can eat broccoli while actually feeling donuts; can also help in food education, making kids see awesome stuff when they eat boring food; can let people that are far away to eat together the same virtual food; can help a lot in therapies against eating disorders. Applications are actually a lot.

This multisensorial experience seems amazing and in my opinion it is completely complementary to the above technologies of universities of Singapore and Tokyo. If this geniuses partnered for an overall magic experience, we could really have an incredible emulation of a virtual lunch, including visuals, smells, taste, texture, sounds. I’d really like to try something like that.

virtual reality taste eating emulation
Will this be the food of the future? (Image by Project Nourished)

Apart from this ultra-innovative stuff, there’s another simpler way to influence taste in VR and it is to use visuals and sounds to emulate an environment that enhances the taste of some food. This is something that has been done as marketing experiences for instance by Guinness, Singleton Whiskey and Innies and Gunn. A reviewer of the Singleton Whiskey VR experience has been favourably pleased by it. The experience let him immerse in the Singleton Distillery environment and then guided him in the tasting. When he was told to smell the whiskey, he was presented with visuals of apples and cherries and he could clearly smell the fruit aroma. When he was told to sip the whiskey, images of chocolate influenced clearly his tastes. Even if he was not a whiskey conoisseur, he was clearly able to taste all the nuances of the tastes of the whiskey thanks to this amazing visuals. As always, our brain is really easy to trick and these experiences show us how even without applying any kind of electrodes, we’re able to influence the taste of what we have in our mouth thanks to other senses in VR.

And that’s it for what I’ve found about taste in VR. As you can see, there is not that much… but I think that thanks to the spreading of VR, more startup will pop up to investigate this very interesting topic. As Gianni would say, I wouldn’t like to be the beta-tester of these devices… but I really applaude to these silent heroes that are sacrificing their tongues for us!

(Header image by Project Nourished)

Skarredghost

AR/VR developer, startupper, zombie killer. Sometimes I pretend I can blog, but actually I've no idea what I'm doing. I tried to change the world with my startup Immotionar, offering super-awesome full body virtual reality, but now the dream is over. But I've still not waken up...