Holoride details its in-car VR entertainment system and the opportunities for content creators [Exclusive]
I’ve had the immense pleasure of interviewing Marcus Kuehne from Holoride, one of the most interesting and original XR startups out there: they provide in-car entertainment via VR headsets, with the VR content that has the movements synchronized with the actual movements of the car. I’m following Holoride since when it has been one of the most popular XR companies at CES 2019, and I have even had the pleasure of trying its product at AWE USA some weeks ago. That’s why I wanted so much to speak with people from the company to ask them more details about the company, its vision, its stance of safety, and the opportunity for us developers in creating products for their platform.
This exclusive interview contains all of this, and through it, you can discover almost everything you need to know about Holoride. The first part of the interview is about the company and the product, while the second half is full of questions about developing content for its platform: if you are a content creator, you would find the second part incredibly interesting, while if you are not, probably you can skip reading some of these questions. The interview is quite long, but I’m very proud of it because it’s incredibly full of information, so I suggest you read it all!
Hello Marcus, it’s great to speak with you today. Can you introduce yourself to my readers?
Absolutely! Thanks a lot for the opportunity, Antony! It’s a pleasure speaking with you.
My name is Marcus Kuehne. I’m 46 years old and an avid car-, tech- and games- enthusiast. I’m also the co-founder and Director of Intellectual Property of holoride. I’m responsible for securing some of the core components of our technology for future competitiveness and enabling together with my co-founders Nils and Daniel our rockstar team to keep developing amazing XR applications for moving vehicles. Prior to co-founding holoride, I was working in various fields such as product marketing, development, and strategy at the German car manufacturer Audi.
I’ve been a VR fan ever since the 90s and its early beginnings, but like many, I was disappointed back then with the final performance. My “old” passion rekindled in 2012 with the Oculus Kickstarter campaign and even prompted me to pitch the use of VR technology in sales at Audi for which I was then also fortunate enough to take over the project management. In that position, we were granted early access to the tech and experienced the challenges and opportunities of it. Having this sort of insight helped us build a strong network within the international VR community and, ultimately, paved the way for holoride by spinning it out from Audi and making it an independent startup. Now and then our vision remains the same: change passenger entertainment experiences for good and substantially contribute to the growth of immersive technologies.
I am pleased to answer your questions – depending on the topic, also on behalf of our team of holoride experts
Let’s talk about holoride. What is it? Can you describe it?
holoride is an in-vehicle entertainment solution that provides passengers with games and experiences designed to adapt to the vehicle as it’s moving. We connect content with data points from the vehicle in real time, including physical feedback, like acceleration and steering, traffic data, as well as travel route and time. This works best with Extended Reality (XR) devices, because our technology provides a new level of immersion into any kind of VR content, creating a breathtaking, immersive experience for passengers, and significantly reducing motion sickness.
Knowing readership are VR aficionados and experts from this space, I would like to add another angle: Many VR fans are enthusiastic about high immersion, which is why 6DoF-supported room-scale VR has basically become the standard. This works great for many applications – but not for content where you fly, hover, float, or glide through an environment. Unfortunately, there is a trade-off as people get either simulator sick in normal VR setups or immersion suffers. Only motion platforms promise a remedy. However, they are technically complex, expensive, and require a lot of space. You can say that every person spends an average of 30-60 minutes a day in an excellent motion platform: cars. Why excellent? because their movement is real. So our thought was, if we reverse the ratio and adapt the VR content to the movement and position of the car, we should be able to offer an experience quality and level of immersion that is not possible at home for many entertainment fans. And that’s what we did – finally our approach turns the car from a relatively bad environment to one of the best environments for immersive entertainment/gaming and many more use cases.
How was this company born?
The origin of holoride’s technology is based on the insight that the synchronization of immersive experiences with vehicle position and motion data enables a completely new entertainment experience in a vehicle. Furthermore, it allows people prone to motion sickness to enjoy audiovisual entertainment on the road for the first time ever.
This, in combination with the fact that in-vehicle entertainment will be of critical economic importance once autonomous driving becomes mainstream, has prompted us to launch a neutral and independent startup with the aim of establishing a new mobility-entertainment medium with various industry partners, such as car manufacturers or content producers.
Why do you think that playing VR in a car is a great idea?
Although our tech works also well with usual screen-based smart devices, the quality of the experience and thus the added value is currently greatest in combination with mobile XR devices. Entertainment options in vehicles are significantly limited compared to the entertainment environment at home, and about 1/3 of people cannot use conventional audio-visual media on the road due to experiencing strong motion sickness within a short time frame. Passengers, especially those in the backseat, are relatively unobserved and usually need to be there for transportation purposes. In this environment, of all places, we are establishing the most immersive type of interactive entertainment while significantly reducing the symptoms of motion sickness, ultimately helping make travel time more valuable for all passengers.
What is the feature of holoride you are the proudest of and which is the one your test users are excited the most about?
People with a high affinity for interactive entertainment and gaming are very taken with the holoride experience — specifically, the unexpectedly high immersion that results from the inclusion of their own movement in the experienced content. Nearly everyone leaves the car with a ton of ideas on what they would like to experience with this technology.
In addition, many people who test the technology are surprised because they do not feel any nausea effects thanks to our motion synchronization. They often expect the opposite based on their previous personal experiences with nausea as VR users and vehicle passengers.
Let’s get technical. How does this synchronization between the car movements and the headset movements work? How is it performed? How do the various pieces of hardware communicate?
Communication between the vehicle and the headset is ensured either via Bluetooth or Wi-Fi. This is specific to the OEM manufacturer, since the electrical architecture varies quite a lot between different OEMs. The vehicle sends different data from different sensors via this wireless communication. After the headset receives all sensor information, the holoride service on the headset calculates the current vehicle location and state with a very high frequency and precision based on IMU, ESC, and GNSS data. [ESC = Electronic Stability Control, GNSS = Global Navigation Satellite System, A/N]
With this information, holoride synchronizes the headset movement with the vehicle movement and we can use it for a comprehensive motion synchronization of the user itself and parts of the shown content.
Furthermore, holoride’s service and the content are distributed separately. The service will be shipped with our holoride application which will then enable the localization. This technical setup was made so that we can add updates to the holoride service such as newer OEMs, better localization, or further features, without the need to force developers to update their content. So, older content that won’t receive any further updates will still work on OEMs introduced to the holoride platform after the release of the content.
Is it true that your technology removes motion sickness in VR? And can it also reduce car sickness in the passengers that are sensitive to it? Do you have data to prove your claims?
In general, simulator (VR) sickness and motion sickness are two sides of the same coin. People who are susceptible to the mismatch between seen and felt impressions suffer equally from both types of sickness. About every third VR user or vehicle passenger is strongly affected, and another third are affected in a weakened form and only after some time.
Since a key feature of our technology is a very latency-optimized synchronization between driving motion and at least parts of the perceived experience, we naturally counteract this effect in the long-term and have a significant reduction of corresponding negative effects.
We’ve conducted user research and found that our technology is significantly reducing motion sickness. This does not mean that holoride removes motion sickness or signs of nausea completely. There are just too many external influences that have an impact on how passengers perceive these potential side effects. However, we do strongly improve the experience for passengers and enable the consumption of audiovisual content with our unique technological approach for many people who have not been able to do so before.
What cars are currently compatible with holoride? And which ones will be in the short-term future?
Unfortunately, I cannot disclose any details here. But rest assured that we’re pushing hard to make holoride available in as many vehicles as possible. We’re pursuing a manufacturer-agnostic approach and aim to make as many vehicles holoride-ready. But the interest in the industry is high and we will share more and exciting details in the course of next year!
Where can we try your solution at the moment?
We’ve been quite busy these past few months, especially after announcing the release of our Elastic SDK at this year’s GDC [find the full presentation here]. Following this announcement, we partnered with Unity and Audi and for a US roadshow that brought us from Los Angeles to San Francisco. We’ve offered a number of game studios an exclusive sneak peek of the original title that we’re currently working on with Schell Games. Using a similar setup, we offered demo rides in the scope of the IAA Mobility in Munich to the public. And just recently, we concluded the IAA Mobility setup at Autostadt in Wolfsburg, Germany, with two Volkswagen ID.4s.
These sorts of events are a great way for us to engage with passengers and understand how they perceive the overall experience and further gather important feedback. If you’re interested in a holoride, send us an email to [email protected]
You were the most interesting XR startup at CES 2019, but after that, you mostly worked behind the curtains. Why have you decided to take a step back on a visibility standpoint? And what have you accomplished in these two years?
Well, I wouldn’t necessarily call it a step back given the overall global circumstances due to the pandemic.
In 2019, our focus was to share our vision with the world and create awareness around the holoride brand while stressing the disruptive technological approach we have taken to build a media platform for immersive XR content. It was an extraordinary year with exciting partnerships and projects in collaboration with car manufacturers such as Mercedes Benz, Porsche, and Ford as well as media companies like Discovery Channel and Universal Pictures.
While 2020 was not as packed with events and showcases, we’ve strongly focused on the development of our Elastic SDK and core product. With all its downsides, the pandemic also helped us to drive this process, bring together our stellar advisory board and it is paying off. 2021 has been a super intense year for us – in a positive way. We’ve won the SXSW Pitch incl. the Best in Show award which we’re super proud of. But we’ve also completed our Series A funding round which is helping us to further establish the holoride brand, continue our development pace, and ramp up talent. We’ve grown into an international team of more than 30 holoriders from over 10 nationalities – it’s amazing to see how we’re evolving as a company.
Building on this momentum, we are looking forward to the upcoming year, which we will dedicate primarily to the production of various content but also prepare for our highly anticipated market launch.
When I shared the latest news about your company on Linkedin, I read many enthusiastic comments, but also some serious concerns about the safety of the users. For instance, if passengers move too much, they can distract the driver. What’s your stance about safety? What are you doing to guarantee the safety of your users?
Having worked for a car manufacturer for quite some time, we naturally take safety issues very seriously. For example, there will be guidelines for our content creators that exclude certain behaviors and movements that are dangerous in the vehicle, such as “extreme” waving of controllers. Our manufacturer-agnostic approach is beneficial in this area because it allows us to define overarching safety standards and share insights on this important topic between the different car manufacturers via our product.
What do you answer to people that say that your solution may make the passengers isolate themselves and not interact with the other people in the car anymore?
If you think of longer journeys, especially in the context of vacation, we do not anticipate holoride being used the entire time. So there’s still plenty of time for the interaction between passengers and the driver.
We are also working on concepts that enable a shared experience — whether its multiple headsets or different personal devices. This way, our technology can even connect people — like online gaming bringing people together rather than isolating them.
Also, we’re looking at this topic from an ethical angle. Kathleen Cohen, one of our advisors, is very vocal about the implications new technologies such as ours will have on users, especially in the context of the Metaverse. We’re strong believers in content that makes people happier, more productive, and even smarter while on the go. Last but not least, we also provide a key technology to make future Metaverse applications – including social interaction and communication – accessible while driving.
I think that what you’re building will be amazing with the future self-driving cars, where people will have to entertain themselves when they are traveling. What do you think about it?
It all started with the very same thought for us, too. As the co-founders, Nils, Daniel, and I thought about how on-board entertainment, which is becoming increasingly important for car manufacturers, could look in the age of autonomous driving. We quickly arrived at the conclusion that immersive display technologies in combination with motion data enable a – literally – unseen experience. When looking at the stats, we’ve also realized that there’s already a market for it as 1.4 billion rides with more than one passenger on board take place every single day — over 60 million of them operated by the four biggest ride-hailing companies. In short, the market and demand are already there today and will increase significantly with the establishment of autonomous driving. However, holoride does not have to wait for this. Instead, the entertainment format and the associated ecosystem should be established as far in advance as possible so that it can then be scaled accordingly and provide a further incentive for autonomous driving.
A while ago you announced a partnership with Pico and Unity. Can you tell us more about it?
Pico and Unity are both innovation-driven pioneers in their sector which connects us in a special way and forms an important part of the basis for good cooperation with all our partners. Not only do we share a fascination for immersive technologies, but we also have a common interest in seeing the XR industry out of its infancy and into a substantial business segment. These two partners in particular are characterized by their ambition and international orientation, which fit very well with holoride’s global orientation.
Are you going to work exclusively with Pico or are you also thinking about other VR headsets? And what about AR ones?
With its high-quality headsets, entrepreneurial ambition, international orientation, and in particular, its strong position in the Chinese market, Pico is a very good first partner for us. However, our goal is to be headset-agnostic. Meaning we want to make as many consumer headsets as possible usable in the vehicles. That’s why we’re speaking with a number of other innovative headset manufacturers which also see potential in this area and are interested in selling their products as accessories in the future, e.g. via automotive retailers.
Since AR glasses will also benefit from our tech stack in a similar way, their potential integration in the future outlines an interesting opportunity for us as well. We are observing the developments within the market with great interest and look forward to the first mass market-capable devices being launched.
Being a developer, I’m very interested in the news about you releasing your Elastic SDK soon. I subscribed on day 1. Give me some anticipation: how will your SDK be? What will its features be?
We have announced our Elastic SDK at this year’s purely digital GDC. Therefore, the Elastic SDK has already been released and the current version can be downloaded via the holoride Creator Space. Once registered, developers can get access to a variety of tutorials on how to set things up and how to create procedural worlds using our graph system.
The Elastic SDK comes with a Vehicle Prefab that moves through the Unity scene where you usually attach the VR camera rig, your custom cockpit model, and everything that should stay in that local space. You also have access to the sensors to measure the forces applied to the car to make your scripts react to it. Everything outside the Vehicle Prefab can be modeled with a node graph-based scene generator that has similarities to Unity’s Shader Graph: Nodes download topological maps, process them, distill information and eventually spawn your custom prefabs and generate a terrain.
Currently, we’re testing features to make the distilled information easily available in your own scripts and presumably release it very soon.
How can we develop, debug, and test the applications I develop with your SDK? What is the workflow that we should follow to create holoride-compatible experiences? Should we buy a new Audi to test our apps? Or do you gift one to us? 🙂
There are many strategies to test and debug your app — starting with actual recorded replays of driving cars to see your game with all its procedural generation in action in the Unity play mode. Play-testing at your desk is also possible by choosing the file playback as the source of motion for your test build. Ultimately, we have developed a Dev Kit – an exclusive and limited prototype made for developers by developers. This Dev Kit will allow for real-world testing of the content that has been built with the Elastic SDK. You can apply for the Dev Kit via the holoride Creator Space.
And from a game design standpoint, what are your guidelines on how to design a solid experience that should be reactive to the movements of the car, without you knowing a priori what the path will be?
Dynamically traveling through the world offers infinite possibilities, but there are also challenges. Therefore, creators are advised to think differently in regards to game design. It’s about creating events that are triggered by the Elastic SDK through ambient awareness. Using the Elastic SDK analyzes particular areas the user travels through, meaning different events are created where they fit best.
The creator has full control to define which features and requirements fit best and which gameplay or storytelling-related events should be triggered in the situation that applies. Ultimately, this creates an exciting challenge for creative professionals to establish this new medium and play a decisive role in shaping it.
In short, you create the world and then you add more and more detailed and complementary events. It’s like cooking — you start with the core ingredients and then add particular spices that make the experience unique in the end.
Developers can join your “Creators Space”. What is it? What does it offer?
Our holoride Creator Space is the one place for developers, designers, and curious minds to get together for anything content and development related. Developers manage their access to the Elastic SDK, choose between a variety of tutorials, sample files, experience tools, and much more.
We are constantly adding new features to the Creator Space. Join the Creator Space via https://www.holoride.com/elastic-sdk
What studios are collaborating with you now?
Amongst others, we work very closely with Schell Games, the largest full-service education and entertainment game development company in the US. Schell Games has developed and produced highly successful VR games. Jesse Schell, Schell Games founder and well-known personality in the XR industry, and his team have become strong supporters, advocates, and friends through our collaboration over the past couple of years. This means a lot to us! Here is a small statement from him, explaining his view on holoride and making us proud and motivated at the same time.
A few weeks ago, Jesse Schell shared the stage with our CEO Nils at AWE in Santa Clara. We have also collaborated with other established partners like Rewind UK (which was just recently acquired by Magnopus) and smaller indie studios and have had very good experiences. Now that our Creator Space is open and we’re working towards our market launch in the second half of 2022, we are in contact with studios of various sizes about some very exciting projects. There will certainly be interesting news to share in the near future.
What kind of experiences are you looking for at the moment? Do you want mostly games, or are you open to every kind of experience the community can come up with?
From interactive experiences such as games of different genres, to well-being and edutainment applications, to work and conferencing tools, we can imagine a very wide range of content. In the gaming area, for example, we can implement 2D games on a virtual screen (with motion-synchronized environment), holoride adaptations of existing VR titles, or even completely new, fully immersive holoride titles.
In the other categories, there are many different ways of delivering fascinating content to customers on different output media with the widest possible motion synchronization and localization. With our reference content, aim to inspire professionals — knowing their experience, enthusiasm, and joy for experimenting, we expect the most surprising and inspiring content to come from creative professionals in the content industry.
As a developer, I love what you’re doing… but it is also true that we indie devs have always limited resources to develop new experiences. Why do you think a little or big development studio should join your creator program?
We realize that joining a new platform is a risk for any developer, regardless of size. However, the rewards that come with being at the forefront of a new technology outweighs the risk and offer a unique chance to create new genre defining IPs. We absolutely believe that this is the case with holoride.
As cars become autonomous, mobile computing platforms, everyone becomes a passenger and a potential consumer of in-vehicle content. Our proprietary motion-syncing technology will enable this content to be compelling, thrilling, relaxing, engaging, or educational (depending on the experience), while simultaneously avoiding the motion sickness that is inherent to any disconnect between real-world and digital display inputs. The possibilities for unique experiences with significant replayability are virtually endless (e.g., take a different route, experience a different “level” of the game).
The worldwide car market is, of course, enormous, and passengers are essentially a captive audience. We think that the opportunity to be first to market with compelling content in a massive vertical such as this should appeal to virtually all developers.
How are you going to help the developers that are joining your program?
We are committed to being the best possible partner and collaborator to developers – both on the business and on the creative side. I believe that flexibility and responsiveness are key to this.
On the business side, we want to offer developers a variety of professional agreement options which ensure a framework according to each individual’s or company’s needs. For example, we have a grant program to help the smaller, indie studios with their upfront development costs that will reward early adopters with larger grants and help mitigate their financial risk.
On the creative side, our product and engineering teams will work closely with developers to support the execution of the artistic vision and ensure that our platform can support their creative goals.
I’ve read that there will be a dedicated store for the experiences made for Holoride. Can you tell us more about it?
We are currently evaluating various concepts for commercializing the different content. For a number of reasons, we are leaning toward a subscription approach, which has also emerged as the preferred form of offering among our target group in all previous surveys. Revenues will be distributed among the various stakeholders, with content creators naturally receiving the largest share, which in our case, is divided up in an indie studio-friendly way according to the determined play/usage time.
Last technical question. Your blog post talks about “blockchain” that is somewhat used together with your solution. What’s the role of the blockchain here? Does this come from the recent NFT hype?
The highly secure, immutable, and decentralized characteristics of blockchain technology will enhance the overall holoride ecosystem, distribute revenues fairly among partners, and offer an unparalleled, personalized passenger experience. The possibilities are unlimited for how content creators can use NFTs on holoride’s platform, and blockchain technology will be integral in making NFTs part of the holoride experience.For example, car manufacturers will be able to track how much time passengers spend on holoride experiences, and by storing transactions in a blockchain-based ecosystem, we will be able to fairly and transparently compensate content creators based on user engagement. Blockchain technology and NFTs will help holoride scale while securing the integrity of our developer-centric and car manufacturer-agnostic approach.
There’s a lot of talking lately about “the metaverse”. How do you envision your company in the future metaverse in 10 years?
The metaverse as a central digital interaction space will be very diverse and will include both completely virtual and hybrid spaces. We are developing the structural basis for bringing this increasingly important interaction and experience level into the vehicle in a special form — an environment where many people spend a considerable part of their day. In the pre-autonomous driving age, passengers naturally want to be entertained in a way that is contemporary and competitive with their home entertainment solution.
With our own product we will initially focus on the gaming & entertainment sector and expand our offer from there, but also other companies can build their content and functions produced for the Metaverse accessible in the car via our technology.
If you could ask one thing to the XR community to help your company, what would it be?
Community support can be multifaceted, from following and sharing our social media channels, to actively contributing and creating content within our developer community, to closely partnering and supporting the development of this market on the part of headset manufacturers. Anyone who shares our vision and enthusiasm can support us in all their own ways. Feedback, suggestions, and criticism are always welcome, and if the idea of merging mobility and immersive media has totally grabbed you — like it has us — you can also browse our job postings to join us as a colleague in our journey to redefine fascinating in-vehicle entertainment.
Can you share some lessons that you’ve learned these years that may help other people in the communities that want to do your same job in the future?
When we started developing the idea around holoride as a cross-manufacturer solution, many skeptics said that the car industry is not ready for such cooperative approaches. Today, our approach is attracting the greatest interest, especially from OEMs, and we are working with many manufacturers — including those outside the VW Group — to be able to provide holoride entertainment to their customers.
Today our company has more than 30+ very dedicated employees with a diverse background of different nationalities and cultures – all of which are united by their enthusiasm for immersive technologies and fascinating entertainment. In the meanwhile, I am only a small part of it. That makes me incredibly grateful and also a bit proud. It shows that anything will be possible if you believe in your dreams and if you’re lucky enough to find the right people to make them happen.
Our vision is coming true bit by bit, even though there have been some setbacks and dry spells along the way. The XR industry itself has faced setbacks in recent years, but after the Gap of Disappointment, is now accelerating its growth. According to all our indicators, the patience and persistence of the market participants who remain committed has paid off, or will pay off with increasing probability in the foreseeable future.
I thank a lot Markus and all the Holoride team for the time they have dedicated to me and to inform the community about their company. Having met them in person, I can tell you they are all very kind people, so I invite you to reach out to them to evaluate a possible collaboration.
(And I also invite you to join my newsletter to not lose amazing posts like this one when they come out!)
(Header image by Holoride)
Disclaimer: this blog contains advertisement and affiliate links to sustain itself. If you click on an affiliate link, I'll be very happy because I'll earn a small commission on your purchase. You can find my boring full disclosure here.
