Apple enters the AR/VR world at WWDC 2017

At last the time has come: after lots of speculations, of questions, of doubts, finally Apple has entered the AR/VR world. It has not been a disruptive entrance, like the one performed by Valve and Microsoft, but it has nonetheless had a lot of resonance in us innovators’ ecosystem. Apple, like a baby, has made its first steps in the immersive reality ecosystem, with two announcements, one in AR and one in VR. Let’s see them briefly.

Augmented Reality

Apple has announced a new framework of computer vision capabilities for the upcoming iOS 11, meaning that all iPhones and iPads will have it overnight, as soon as the operating system update will happen. But… what is this framework? Well, basically it is the same thing as Facebook Camera, but with a different name, that is ARKit.

https://gfycat.com/gifs/detail/disastrousidenticalbuffalo

(GIF from UploadVR)

With ARKit, you can perform SLAM using the single RGB camera of the phone, that is the phone can track its position and orientation in space using only the phone camera and this is awesome, considering that more complex devices like HoloLens need arrays of cameras to perform similar tasks (in case of Hololens, 4 RGB + 1 depth). Of course HL does a lot more stuff, since it is able to reconstruct full environments, while this software basically only maps planes and allows you to put virtual objects on it. As a computer vision guy, I really love this technology. And as you can see from the above GIF, the virtual cup is really stable (very few jitter), so the tracking works really well.

One thing that I found really cool is that this new toolkit will have ambient light estimation, that is the virtual objects will try to emulate the lighting of the real objects surrounding it. Of course this will be an estimation and I bet that it won’t be perfect, but it will be a step towards the solution of one of the biggest problems of augmented reality. When I tried HoloLens, in fact, one of the first things that I noticed is that the virtual objects are easily spotted because their lighting is completely incoherent with regard to the real environment. Apple is trying to solve this problem.

Apple announced that the ARKit SDK will have Unity and Unreal Engine plugins and this is an awesome news for us developers. Furthermore it is partnering with lots of companies (like IKEA and LEGO) to provide AR content for Apple devices in short time and this is awesome for consumers, since they want content. On stage they also showcased an enhanced version of Pokemon Go using ARKit and everyone went nuts.

Why is this a great news?

• Because this means that all Apple devices will support augmented reality out of the box and this will help a lot the AR ecosystem. The App Store is one of the best stores out there and Apple developers are really smart, so I guess that lots of them will start making AR applications and we’ll see a lot of AR apps to pop out. Creativity of people will lead to lots of new AR products we hadn’t even thought before. Someone on Quora is already asking what this ARKit will be useful to: I think that entertainment (gaming, Snapchat-like stuff, etc…) will be the main implementations, but also some business applications (like smart e-commerce, architecture exploration, art, etc…) will come out;
• Because this means that Apple is really committed to augmented reality, as rumors were saying. And I bet that this technology will be the same one that will be implemented into future Apple AR glasses (that will surely come). Acquisition of Metaio of some years ago is showing its first results.

Virtual Reality

Apple has announced its new VR headset, the iOculus! Ahahahaha no, just kidding guys, nothing that important.

Apple has “simply” announced that new iMacs will support SteamVR, hence all SteamVR headsets (Vive for now, but also LG headsets in the future). New 27-inch iMac will be VR-ready, thanks to the implementation of a powerful-enough GPU, the AMD Radeon Pro 580 GPU (that is roughly as a NVIDIA GTX 1060) and to updates to the new Mac OS High Sierra.

Mac was supported by Rift initially, but then, due to the low power of Apple GPUs and the little market share of this PCs (7% circa), Oculus dropped support for non-Windows platforms. Palmer Luckey commented upon Oculus support of Apple computers with “That is up to Apple. If they ever release a good computer, we will do it.”. Now Apple has released a good computer, but it has been Valve that added compatibility and not Oculus.

Valve as always is making a great job in keeping their platform open and compatible with most devices possible: SteamVR is stable for Windows, in beta for Linux and now it is on track for Mac, too. Really a great job, Gabe’s guys!

But let’s get back to Apple: as I’ve said, compatibility with VR has been possible thanks to a new powerful GPU. What happens if you don’t have that monster GPU? Well, you can buy an external add-on (an external AMD 580 with a fancy box) and start using VR even if you don’t have the latest Mac. Just plug it via Thunderbolt and start experiencing VR! About us developers, all these cool stuff will mean also a support for Unity and Unreal Engine VR SDKs, so finally Mac developers will be able to develop for VR (while iOS developers will develop for AR, due to above paragraph 😀 ).

As said, the VR-readiness has been possible not only because of the new hardware, but also thanks to some software updates. Metal 2, the company’s updated hardware-accelerated graphics API, will have all the required optimizations for VR pipeline, making VR possible. All these upgrades will be available with Mac OS High Sierra, that, as Road To VR says:

The OS launched as a developer beta today. A public beta will follow in late June, followed by the final launch as a free upgrade on all systems that support the current ‘Sierra’ version.

So, we’re talking about something really really close.

The real issue with this is price: this Macs price start from $1799 and if only the Radeon 580 will be confirmed as VR ready, the price to spend is $2299. In a moment where the VR ecosystem is trying desperately to reduce costs by cutting the cost of VR-ready PCs and headsets, raising them again seems a weird move. But we’re talking about Apple, guys. High prices are quite normal.

As you can see, these announcements are not that huge. Basically they presented a computer vision solution for their phones and a support for an external headset for their computers. Anyone expected headsets or glasses have been deluded. But there are some keypoints I want to highlight:

• Apple is investing in AR/VR. It was the only major company still not expressing a concrete interest in these technologies and now it entered the market. And considering that Apple moves only when it thinks that a technology is ready for final customers, this makes me think that they truly believe in AR/VR and this makes me even more certain on the fact that VR isn’t a “fad”;
• Apple has produced an AR framework and this highlights their interest in AR over VR. Tim Cook months ago publicly announced interest of Apple for this tech and now this interest has transformed in something more concrete. I guess what will happen in the next months;
• They showed an awesome AR technology. My bet is that they have a lot more in their labs, but they’ll show it only at the right moment, when the market will be ready;
• There haven’t been any announcement for a VR device. I mean, if Apple were interested in VR, it would have produced its super posh VR HMD, but instead it just made their PC compatible with external hardware of a third-party. Considering that Apple loves closed ecosystem, this makes me think that they have no interest in VR, so they have just added VR support so their customers can use VR. But they have produced no VR ecosystem (no VR app store, for instance), since they don’t believe in this tech. And being compatible with SteamVR, if they’ll change their mind, they could produce some kind of Apple VR headset compatible with SteamVR, even if I believe this won’t ever happen.

And that’s it. What do you think about these announcements? Are you excited by them or you don’t give a heck? Let me know in the comments…