Mojo Vision: AR contact lenses are coming far sooner than you think

One of the most mind-blowing interviews I had in recent months has been the one with Mojo Vision, the company producing AR contact lenses that got famous during CES 2020. I thought it was an ultra-experimental company needing 20 years to release a product, and instead, they are far ahead from where we all imagined they were. AR contact lenses will come far sooner than we thought. Discover this and many other intriguing details in this article that summarizes the interview I had with Steve Sinclair, SVP Product & Marketing at Mojo Vision!

Mojo Vision

Mojo Vision is a company producing augmented reality contact lenses, that is contact lenses that you put in your eyes and that can not only fix your eye impairments, but also give you augmented vision. If you thought that AR glasses were already advanced stuff, you have to rethink your vision of technology 🙂

Mojo was born by the meeting of two people: Michael Deering and Drew Perkins. Michael has been the head of 3D graphics at Sun Microsystems for many years, and there he pioneered many technologies. He retired 15 years ago, and from that moment, he started working on the “ultimate display”,  that aimed to be very immersive, power-efficient, and perfect under every detail. Working in his basement for years, he reached the conclusion that the only way to build this was to embed the display into a contact lens: if the display is that close to the eye, it is easier to minimize the power and computational requirements of the whole system (that is one of the problems of current AR glasses). If the display is close to the eye, you need fewer pixels to reach the critical density and so the system is more manageable.

Drew Perkins is a serial entrepreneur who at a certain point of his life had cataracts. He had to undergo surgery and since the surgeon had to implant a lens into his eyes, he wondered why he could only have a lens that just restored his vision and not one that could also improve it. So he started doing some research and meeting people to try to investigate this topic. 

One day these two people met, and they discovered that they had the same purpose in life and complementary skill sets. So they partnered, and 5 years ago they founded Mojo Vision. Steve Sinclair, the guy I interviewed, joined them more than 1 year later to focus on what the company is building, for what customers, and how to bring the product to market. The total size of Mojo is now bigger than 90 employees.

Is its technology real?

I started the interview immediately with a harsh question: what has Mojo in hand to prove that it is not a scam, it is not a company that just raises money from investors to not deliver anything? I was excited reading the news of Mojo at CES, but many people were quite skeptical.

Steve answered me that they don’t want to over promise as other companies have done in the past (cough…cough… Magic Leap), but of course, they have to talk about what they are doing to generate awareness and attract investors. And they had prototypes and three different demos at CES, but since November 2019 they also have worn the lenses themselves in their lab tests!

While people outside of Mojo can’t currently wear the lenses, they can try three demos:

1. An AR contact lens installed on a stick: the user can take the stick in his/her hand and put the lens close to one of his eyes, to see the content of the microdisplay. The display is currently only monochrome (green), has 70,000 pixels (a good resolution, but it can be better), but it can give the users the idea of what is possible to see today with an AR contact lens. The demo, in particular, shows some dynamic content (sent from a computational box) at 1 meter of distance from the lens;
2. An augmented vision demo. Always with a lens installed on a stick, the user can walk inside a completely dark room. The room is actually lit with infrared light, that is invisible to our eyes, but is visible from the image sensor installed on the lens. Looking through the lens, the user can see the objects in the room directly like using night vision goggles, or he/she can see their edges detected by an algorithm;
3. A VR demo to show the user experience. Wearing a VR headset with eye tracking (Vive Pro Eye), the user can see what the experience of wearing AR contact lenses could be in a practical scenario.

https://gfycat.com/kindfeminineisopod

Now they are trying to build a system that includes all these characteristics (performant display, image sensor and eye-tracking UX) in one only device to demo.

Wearing Mojo contact lenses

Steve confirmed to me that he has worn Mojo contact lenses in the lab multiple times. He says that they are very comfortable for mainly two reasons:

• They are custom-built for his eyes. The lenses have to be tailored to the eyes of the specific user, so they adhere perfectly to the surface of his/her eyes and don’t move. This, of course, creates a great comfort sensation and also more precision in the augmentations, because they can be projected directly into the fovea of the user;
• They don’t sit on the cornea, that is full of nerve endings, but on the white of the eyes, that is far less sensitive to the presence of external elements.

Thanks to these characteristics, he can currently wear Mojo lenses even for 3-4 consecutive hours. This is impressive, considering that it is sometimes hard to even wear AR glasses for that amount of hours… I expected they were wearable for only minutes, not hours. Actually, Mojo employees are still not satisfied by this amount of time: they want lenses that can be worn for the whole day, and so they are investigating for instance how to build the lens so that more oxygen can pass through it to reach the eye of the user, so that the eye can “breath” more, for enhanced comfort.

I asked him when we can have lenses that are not custom-built, and he said that for now it is not possible, but in the future, when they’ll gather the data of thousands of eyes, probably they can find some kind of “preset shapes” of the lenses that are good enough for everyone.

What is there inside a Mojo contact lens?

For now the company is interested in building a baseline system that can be expanded later on in the future. This system should include:

• A microdisplay to let the user see the augmentations;
• An image sensor for seeing the surroundings and process them through computer vision (e.g. edge detection);
• Eye-tracking sensors (accelerometer, gyroscope, magnetometer);
• A battery;
• A 5Ghz radio communication antenna to make the lens communicate with an external unit;
• An ARM0 processor, that acts as a “traffic cop” for the data.

Since the lens must be very tiny, all the computations must happen outside, in a computational box that must be optimized so that to read the data sent by the lens, analyze it and send it back so fast that the total time from when the sensor detects something and the user sees the related information in front of his eyes is below 10ms. Because of these needed optimizations, this box currently can’t be the smartphone, but must be a custom-built computational unit. This unit can be attached to the smartphone to exploit its connection to the internet (maybe in 5G) to send some data to the cloud.

What is the FOV of Mojo AR lenses?

Mojo still doesn’t disclose the real FOV of the lenses, and Steve just shared with me the fact that in the VR demo about UX you could see the augmentations in a spotlight region in front of your eyes that had more or less this dimension:

Regarding the FOV, Steve underlines that it is pretty different from the FOV of standard XR glasses. With XR glasses we mostly notice the FOV when we move our eyes around, even with unconscious movements, but here the lenses are attached to our eyes every time, so there is no way that the eyes see beyond the borders of the FOV. The limited FOV on the contacts is so visible more as a limitation of the peripheral vision.

What about safety?

Let’s be honest: when we think about AR contact lenses, the first thought that we have is: “What if these things explode in my eyes?”. So I asked this question to Steve… well, probably not with the world “explode” inside, but hey, I think he got the idea.

Luckily, Steve still has both eyes. And this is for sure a remarkable result from Mojo. Or better, they seemed real eyes when I talked with him, but who knows, maybe they were cameras like with Robocop.

Jokes apart, Mojo cares a lot about safety. Contact lenses are regulated as medical devices in the US and EU, so they must undergo many checks and controls before reaching the market. If they are not guaranteed to be safe, they can’t be sold. That’s why Mojo is working with the FDA, which has authorized them to continue their research through the Breakthrough Device Program. Mojo Vision is collecting a lot of data from its first internal tests, and these data will be used to prove that the lenses are safe. Furthermore before every test, there is a strict checklist of things to control, and in the lab there are many optometrists that confirm that everything is ok and are ready to intervene in case of any problems.

Steve informs me that there are many things to care about to guarantee the safety of the users: the battery should be safe and should not leak electricity or explode; the device can’t become warm during usage; and so on. Even at the application level, they are thinking about the fact that the runtime should never allow an application to fill completely the field of view of the user or distract him/her during his/her tasks for security reasons.

Can Mojo lenses do VR?

When I asked this question to Steve, he answered me with a simple yet very effective answer: “if you wear smart contact lenses, if you close your eyes, you have VR”. I was like

He told me that he tried even this modality: he could play a card game with his eyes closed and a camera framing the cards that were on the table streaming the images to his lenses. I think that poker players can become one of the first users of Mojo Vision contacts 😀

Release roadmap

So when will we have AR contact lenses? According to Mojo vision, it is not 5-10 years, but before. Of course, the actual release date will depend on FDA approval.

The first lenses will be targeted to people with eye impairments so that they can help them see better. If these will prove to be effective, the company can start targeting specific enterprise sectors, like hospitality, logistics, military, public safety. Every task that requires you to have extra information in front of your eyes while having hands free and looking normal (no weird glasses) can benefit from these lenses. Mojo has already been contacted by many companies, but they have no info to share about partnerships.

After the enterprises have adopted the technology, it will come the time for consumers (Yay!).

First mover advantage

https://gfycat.com/unrealisticlankybellfrog

Mojo is very confident of what it is doing, also because they are the only company working with this kind of technology that has been admitted to the Breakthrough Device Program by the FDA. Mojo Vision is not only a tech company, but it is a quite unique mix between a medical device company and a consumer electronic company. It wouldn’t be easy for other companies like Apple to come and steal their job in a short time because to do that, they should undergo all the FDA approval process and waste a lot of time. This gives Mojo a quite unique competitive edge over the others: for a company like Apple, it would be easier to directly buy them.

Glasses vs contact lenses

So, are AR contact lenses better than AR glasses? Steve thinks there is not a true competition: if you are a glass person, you may prefer wearing AR glasses, if you are a contact lens person, you may prefer AR lenses. They can also be complementary technology and be used together.

https://gfycat.com/disloyalidioticgrison

SDK

My fellow developers, the sad news is that this technology is so early stage that Mojo has not among its top priorities the one of developing an SDK for his lenses. We’ll have to wait a bit before developing all the kinky AR apps that we have in mind for this astonishing device.

Expected price

So the big question is: how much will these amazing pieces of hardware cost? I asked Steve if the price was more towards 10,000$ or 1,000$ and he said that it is more towards the latter. They hope to create contact lenses that cost like a very high-end smartphone. He said that if someone wears contact lenses every day, he/she already spends $500-900 a year for them, so a price around $1500 for two AR contact lenses doesn’t seem so insane.

Future vision of AR/VR

Mojo sees AR and VR evolve towards “invisible computing”, where you don’t have screens that distract you like it happens today with the smartphone. We will have something always in front of our eyes, that will add virtual information directly inside our vision, but without distracting us. And to be informative, the virtual elements should be contextual to what we have around us and what we are doing at that moment. Virtual elements shouldn’t suck your attention when they are not needed, we need the right content at the right time.

https://gfycat.com/somelonejumpingbean

Regarding user interactions, currently, Mojo is working a lot on eye interactions, that are very powerful, but plans also to integrate voice and hands gesture later on. Multimodal control is the way.

The last lesson

Before finishing the interview, I asked Steve one last question. Since he has worked for Palm, Apple, and now Mojo Vision, always being at the cutting edge of technology, I wanted to know what is the biggest lesson that he has learned in all these years. He answered that the most important thing is “to focus”: if you are building something, focus on the value for the end-user. There are many things that you can do for your product, but when you decide what are the features to implement, you should ask yourself: “Should I do them and it is the right time to build them?”. Focus on the customers and what they find useful.

---

That’s a wrap. With this great life lesson, ends my long journey inside Mojo Vision. I really want to thank Steve and Alissa for the time they have dedicated to me and I hope I’ll be able to meet them in the US one day and try the prototypes they are building.

For now, I invite you to ask me all the questions you want about Mojo (I hope I can answer them), to comment your excitement for AR contact lenses here below, and to subscribe to my newsletter!