What is Google ARCore and how to get started with it

Google has decided to answer to all the hype Apple was having with ARKit releasing its own augmented reality framework, called ARCore. This is the third time a major company is announcing an augmented reality framework. The first has been Facebook, with its Facebook Camera; then came Apple with ARKit and now there is Google, too. So, the two most important companies that produce operating systems for smartphones and the biggest social networks are all working on augmented reality on mobile devices. This is the reason why all market analysts are all predicting huge numbers for this kind of technology in the next months.

If you still don’t know what augmented reality for smartphone is, the concept is very simple: it is a technology that let you put virtual elements in the real environment that you see through the camera of your smartphone. Since a video is worth a bazillion words, here you are a funny example.

This is an example made with Google ARCore: you put the funny virtual eyes onto the real wall and you can see them through the screen of your smartphone. Notice that the eyes remain exactly in the same real position as you put them, even if you move your smartphone or rotate it. This is smartphone based AR.

I admit I’m not a huge fan of this technology, as I’ve already written in a successful Medium article. The reason is: true augmented reality is the one you can obtain through a glass (like Hololens) so that you see the virtual elements in front of your eyes in every moment (and this allows for contextual information) and you also have your hands free. Anyway, I find the technology behind these smartphone frameworks really incredible, since performing on a mobile device, with only one RGB camera, a perfect tracking of the environment surrounding you to detect the pose of the smartphone, is really really difficult (I know something about computer vision and trust me… it’s super hard). If you want to know the fundamentals of how this works, check out this wonderful page that Google engineers have written.

So, ARCore is a framework that lets you use your smartphone to:

• Detect the pose of the device with regard to the environment around it, using some key feature points it finds into your environment;
• Find planar surfaces in the environment around it;
• Put virtual elements onto these planar surfaces so that they’re completely fused with the real world;

• Understand the average lighting around these planar surfaces and light the virtual objects onto it accordingly.

Basically, the same things that ARKit is able doing. When Apple released ARKit, we were all wondering why Google, that is experimenting with similar things with Tango since a lot of years, hadn’t ever thought about doing something similar. Actually, they were. Google has stated in its ARCore launch post that it was working on such a tech since 3 years, along with the one that powers Tango phones. And in fact, ARCore appears as a particular version of Tango, as we’ll see when I’ll talk about how to develop with it. Let’s all remember that Google is also working on the Worldsense technology that will power Google Standalone VR devices like the Vive Standalone, so they’re very good in developing tracking stuff (and I bet a bazillion dollars that Tango, Worldsense, and ARCore share a lot of the underlying algorithms).

So, which one is better? ARKit or ARCore? An interesting Youtube video made by FusedVR makes us see that… they’re basically the same. So, I think that fanboys of both platforms will say that their one is better, but actually, at this moment there is not much difference. Things may change a lot when Apple will release the iPhone 8 that rumors say that will have depth cameras and this would make Apple tracking far better (But also Google has its Tango phones…)

The great advantage of ARCore will be that it will be available with devices for different vendors, that will offer a wide gamut of prices: this is the strength of Android, that will make sure that this technology will be available on a lot of devices.

I admit I was super-excited by reading words like

It brings augmented reality capabilities to existing and future Android phones. Developers can start experimenting with it right now.

and

ARCore is our next step in bringing AR to everyone

I was super-excited. I really wanted starting experimenting with smartphone-based AR, making videos, tutorials! When suddenly…

ARCore will run on millions of devices, starting today with the Pixel and Samsung’s S8, running 7.0 Nougat and above

Google ARCore is currently available on ONLY TWO PHONES: Samsung S8 and Google Pixel. Reasons are unknown: surely the AR algorithms are demanding and require a powerful phone, but I guess that the S7 or the S8+, for instance, are good enough to run them. Probably has been a marketing choice to make people upgrade their phone. Or probably they target only Daydream-compatible phones (S8 and Pixel are both Daydream compatible). I really don’t know the reason, but this is so sad: they say they want to bring AR to everyone and then they actually bring it on only two models. Some people have tried to make it work with other phones (even Tango enabled phones like the Zenfone), but they verified that actually, it doesn’t work 🙁 .Also, their predictions don’t satisfy me: they say they’ll make more than 100 million devices AR-ready in the next times, while Apple numbers are at least 4x (someone says that by the end of 2018, ARKit compatible phones will be 500 million, others say 800 million!)

ARCore is currently in developer preview. So, if you are a lucky person owning an S8 or a Pixel… well, you can start developing for it. You can develop for ARCore using Android Studio, Unity, Unreal Engine but also web frameworks! And all the SDKs are completely open-source… you can find their source code directly on GitHub! (Yikes!) At the end of this page, you can find the links with all the info from which you can start developing for every platform. What surprised me the most has been the web platform: this has been a genius idea from Google. The boring side of AR apps is that for every AR experience you want to try, you have to download an app, maybe just to see a virtual element to pop out from a banner, something you just want to try once. So, Google is working on an API to make web AR experiences possible (I’ve read about a possible future WebAR framework, we’ll see): this way you can just go to a website with your smartphone browser and live the AR experience without installing anything! And they’re planning to make this work with both ARCore and ARKit... if you look at the page about how to get started with web AR, you can see that they’re considering both technologies. Super-cool, isn’t it?

So, to get started, go to this page, pick your platform and then follow the instructions.

Being a Unity developer, I’ve given a deeper look to the instructions on how to get started with Unity. The first thing to notice is that you have to download the latest Unity beta (2017.2.beta9), a build that has been released just to let the communities develop with ARCore. Unity version has been released a little later than the news, so some people started getting angry waiting for it 🙂 . Then you have to set some specific settings, some of which require you to enable Tango support on the build. This makes me think that from a game engine standpoint, Tango and ARKit are basically the same thing (and surely the two platforms will blend in the future when smartphones will add depth sensors to rear cameras).

Then it is possible to use ARCore APIs to get information about the point cloud detected by the device; to get the available planes; to perform raycasts towards physical objects; to attach virtual objects to the physical world. There is a sample scene inside the package to get started. The core feature of the package is that Google takes care of performing the difficult tracking stuff and the dev has only to think about the particular features of the experience he/she is developing.

After the project is ready, it can be built and deployed. It is important that before running the application, the user downloads the ARCore Service and then installs it with the following adb command:

adb install -r -d arcore-preview.apk

Many people have forgotten this step and weren’t able to try ARCore demos. Looking on Unity forums, I’ve also seen that some devs have had issues in running their ARCore experiments on their devices, especially on the S8. This shows us how this framework is still experimental.

If you want to see what the community has already done with ARCore, I advise you to go the official Google Experiments page! You can find nice things there… but as I always stress, I’ve still not seen a convincing demo on mobile AR.

I tried asking devs at Luden, which is making an ARKit game, if they’re interested in making an ARCore porting of their game, but obviously them, as all other devs, want to understand better this platform before taking any decision.

Anyway, ARCore, as ARKit, is enabling technology that will let developers create interesting AR apps in an easy way. What’s interesting is that thanks to the support of Apple and Google, soon millions and millions of devices will be ready to run augmented reality experiences and this will create a huge market for AR and will make people wanting more AR and this will pave the way for future AR glasses. So, kudos to Google for this wonderful tech!

And… what about mixing ARCore with Aryzon? We can have fully working cheap augmented reality glasses…

I’m very curious on your opinion about ARCore, so let me know in the comments or on the social networks! And as always, if you liked this post, please subscribe to my newsletter! 🙂

(Header image by Google)