Frooxius talks us about the development of amazing VR metaverse Neos

I had the great opportunity to interview Tomáš Mariančík, better known in the VR communities as Frooxius. He has always been a great innovator and I admire his work since the good old Oculus DK 2 times, when he published “Sightline: The Chair”.

Now he is continuing this innovation work by building a completely open metaverse called Neos, where thanks to object modeling and scripting, it is possible to create everything you want. It is really an interesting project, also considering the very small team that is carrying it on.

But it’s better that I let him describe to you what it is with the typical passion of all VR enthusiasts…

Hello Mr. Frooxius, introduce yourself to my readers!

Hello!

I’m a VR game developer and one of the original Oculus Rift Kickstarter backers. VR has been my life for the past 5-6 years and for more than half of that I have been trying to build a metaverse I call Neos.

I am a big fan of all the experiences you created: Sightline The Chair was incredibly trippy, World of Comenius was innovative for education and now Neos VR seems one of the best metaverses out there! These programs are all so different but at the same time equally cool. How did you get the inspiration to develop these three experiences? And how do you manage to always do something innovative?

Aww thank you! I always love finding new approaches and outlooks to anything, even just for the fun of it to see where it goes. When I was in high school I would design and build some esoteric CPU’s (which I called Weird Processing Units) just to see what properties they would have and to better understand why the ones we use are designed the way they are.

I love unique creative or technical ideas, because they expand the way you think about things and overthrow a lot of base assumptions we have about the way things work.

With SightLine, I wanted to see how could VR make you experience something you never could in real life (and that our brains weren’t really prepared to process), essentially having a world with “broken” laws of physics. It’s why I backed DK1 in the first place, because I wanted to build that kind of experience just to see how it would feel.

World of Comenius combined that with another passion of mine, which is learning. School often frustrated me, but I absolutely love learning new things since I was a kid. I’d be always buried in encyclopedias or taking my toys (and decommissioned electronics) apart, nowadays I even get a textbook on something out of my area of expertise (like biology or astrophysics) just to get a better understanding for the field.

I wanted to make something that would fuel that kind of curiosity in others, a world that you can freely explore and learn about things as you go, not because it’s on some curriculum, but because you just want to play around or explore with your friends.

Neos VR actually grew out of that, in part because of the support we got from the Rothenberg Ventures (both financial and motivational) and in part because the content building tools and multiplayer framework/engine I was designing was very general to begin with, so it didn’t have to be used just for education.

You started when Oculus was only a devkit… what are the differences between developing for the Rift now and then?

I’d say the biggest chance is having the positional hand controllers, which helps with design of interfaces immensely. When you can just reach out and touch or grab objects with precision, it makes the interactions so much easier and much more powerful.

The headset itself is also significantly better. Having higher resolution displays and positional tracking gives more freedom as well. I remember not being able to read any text with DK1 unless it was really huge.

The SDK’s have also matured somewhat and been integrated into game engines, which saved a fair a bit of boilerplate work, streamlining the whole process.

However there’s been some extra bloat which made it less enjoyable as well. You have to jump through some fairly unnecessary hoops to test and publish your experiences (particularly on mobile headsets like GearVR or Go) and sometimes miss the lightweight nature of the older runtimes.

What pieces of advice you’d give to someone wanting to become a VR developer (or dev studio?)

I’d say the most important thing is to understand that VR is a different medium from the screen, so you have to be prepared to let go of a lot of practices you’re used to when developing traditional games and apps. VR requires you to rethink how are things built, so you really have to put your time into that, otherwise your VR experience will suffer.

Shoehorning traditional games into VR doesn’t work too well and will make VR feel like a gimmick. In my opinion the best VR experiences are the ones that can’t be experienced on screen.

Neos VR seems an astonishing project… do you mind talking about it a bit?

Well first, it’s the most ambitious and challenging thing I ever worked on and that includes making my own processor architectures 😀 Part of the challenge is explaining what it really is, because it’s so many things at once.

From user perspective it’s a social universe. You can meet other people, play with them, explore. You can import 3D models, images, videos, music into the worlds. And if you want you can also grab various editing tools, from simple ones like material gun, to more advanced and powerful ones, like the inspector or visual scripting and build out entire worlds and complex interactive items together with other people, in realtime.

It stems from all the different VR (and non-VR) projects I did before. I would always wish I had feature from this other project in this one. Or I’d have to spend time implementing multiplayer for this and that and integrating the same hardware over and over again, so wanted to build an engine that’d solve these problems once and for all and that would let me (and others) implement these within a single shared universe and have them all coexist.

For example, do you need to build a VR MRI visualization for medical use? Just write the specific bits, like volumetric rendering and reading formats. You get collaboration and persistence for free. Need to draw into the scan? Just grab a brush tool that was already built for a note-taking purposes and bring it to the other world.

To enable all this, I designed Neos quite differently at the fundamental level. I had to start really low, building out core programming blocks, that are analogues of variables, references, lists, trees, classes and so on and then build out the engine from these. It took significant amount of time during which I didn’t have much to show, but now it’s really paying off as we can add new features faster than anyone else, without having to deal with the bugs they do.

Describe me what are in your opinion the coolest features of NeosVR.

The best thing about Neos is the creative freedom it offers and how easy it is to build and change things right from within VR, in realtime and in multiplayer. It’s like being inside of Unity engine with your friends.

But you also don’t have to use any of the editing tools if you don’t want to and you can just hang out and play with things others have built. It puts everyone on equal footing and that’s one of the most beautiful things in my opinion. In the real world you can invite someone to your workshop to show them what you’re working on, they don’t have to be an engineer. But in software this is not guaranteed.

With Neos, we got people from different backgrounds and skill levels, from younger kids, to hobbyists to professionals and university researchers, all naturally interacting with each other and even helping out with things. For example a guy from California would join the researchers in Sydney, shrink down and fix a broken visual script on their vestibular system visualization. Or a kid would help a professional with virtual camera work for recording his tutorials. And then somebody suddenly spawns a three player chess and we play together.

People often tell me that software has to only target particular demographic, but we all work and exist in the same physical world, so the virtual one should let us do that too, but better. Seeing it happen on its own like that is one of the things I’m the most proud of.

What has Neos VR that in your opinion makes it better than the competition?

It’s related to the things I already mentioned above. From user perspective it’s the creative freedom and power it gives you. No other VR software offers it at this level, from within VR and in multiplayer. You never have to take your headset off, everything is in VR. You can build entire worlds and interactive items and you can have someone help you as if you were in the same physical space, working together in a workshop. Want to show something off? Just copy and paste it there or import from your disk.

This is possible thanks to the architecture of the system, which I don’t think anyone can rival right now. I put lot of time into designing it in a novel way to build a new programming paradigm and then build out the project with it, layer by layer. If you wanted to implement these kinds of features traditionally it would take far too long and would be too error prone.

What are its flaws instead?

I’d say right now it’s some usability issues, particularly UI’s, lack of more higher level building tools and some templates and guides to give people some direction depending on what they want to do. I‘m slowly working on making it easier to use for beginners and ease people in, but it still needs a lot more work. Often I don’t have the necessary frameworks or components to implement the UI and tools I want, so I have to build these first and that takes time.

Our community is also still small at this point, so often somebody can come to Neos and not find anybody online at the moment. Our members are very passionate about the project though and constantly bring more people in, so that’s also improving.

One of the flaws that are inherent to Neos’ design (and purpose) is some overhead. To offer the the flexibility, versatility and ease of use, there are subsystems which take certain amount of system resources to do their job and offer the abstractions they do. This is natural for any project of this type and I’ll be working hard to make that overhead as small as possible, but there will always be scenarios where you’ll need to write a custom solution from scratch and closer to metal.

But I think for a sizable chunk of experiences that overhead won’t matter and they’ll greatly benefit from being able to focus on their own unique functionality, without having to worry about multiplayer, persistence, hardware support and interoperability.

What surprised me the most of Neos is that everything is completely scriptable, as if it was Unity. Please describe me the scripting mechanism: how does it work? Can I also code using C#?

Yes, Neos itself is designed like an actual game engine, actually very heavily inspired by Unity and some other tools. At the core you have the data model – scene hierarchy, components, fields, references, lists, arrays, dictionaries and so on. All the engine and behavior components are written like this and have their properties accessible via the inspector inside of VR, giving you full control.

Neos has a visual scripting language, which essentially allows you to read, modify, add, remove and create relationships between these elements. For example to create flashing lights on your avatar as you talk, you can take the voice volume value and plug it into the intensity field of the point light source.

Or to make a weapon fire on a trigger press, you can wire the trigger press strength into the particle emission rate. Or check when it passes a certain threshold and start a timer which toggles visibility boolean on a beam effect and triggers a sound playback.

It’s a fully fledged programming language, so you can do a lot more complex things too. For example one of our users built a functioning calculator or a controllable drone that you can fly around with your controllers (or even fly inside of).

Because all engine components are also written this way and have all their properties accessible, you can do pretty much anything with this and wire anything into anything else (or even use it to generate new components).

You can use C# as well to write new components for example and have even greater amount of flexibility and power, but right now you have to distribute the code yourself for security reasons. That’s going to change in the future though, but requires a significant amount of work.

How many people are developing it?

For the development of Neos itself it’s still pretty much just me, but now we have our community members helping with lots of things, like building tutorials, 3D modelling environments and items (like a new hub), building cool items to showcase what’s possible with it and spreading the word.

And how many are your daily/concurrent users?

We have at least 5-6 different people who use and meet in Neos daily, with bigger groups coming in, like 10-15 at the time, but that’s a bit random. It still gets quiet during some parts of the day, but it’s been slowly growing over time.

Why in your opinion virtual worlds like Sansar, that are well made by an established company, are getting so few users?

I’m not quite sure since I don’t keep up with these things too much. From what I heard from some of the people coming to Neos one of the big factor is that they’re only half-way VR and don’t seem to want to embrace it fully, particularly for in-VR building. A lot of our creator users don’t like having to constantly go in and out of VR whenever they want to change something in their environment.

I went through something similar early on with my projects, where I was afraid VR wouldn’t be enough to support it, so I wanted to design them for both screen and VR, but that just ultimately sacrifices usability and versatility on both. With Neos I decided to go full on VR and just design things from scratch without thinking about desktop mode at all.

Is Neos VR already profitable? How do you plan to make it sustainable?

Unfortunately it’s not, but it’s slowly getting there. We have a Patreon page where people can support us and our patrons already cover the running costs and some of the bills, which has been a great help over the past months.

We are also offering commercial licenses for businesses and schools and some companies are already using it, like Czech branch of SIEMENS for visualization and employee training, or the Czech PwC exploring its use. Thanks to Neos’ versatility we can also adapt it very quickly to build specific commercial solutions.

But I think the main source of income will be the community itself and economy. We’ll be releasing our in-world currency NCR (backed by the Ethereum blockchain) pretty soon, which people can use to trade services and items or to simply invest into Neos. Our Patreon supporters have been getting NCR for their early support.

However the business side is more of my co-founder’s area, I’m more or less just building what I think is cool 😀

How much does it cost, for which platforms is this available and when will it come out from early access?

Neos itself is free for anyone on Steam. If you support us on Patreon you get extra perks like extra storage for your worlds and inventory and faster download speeds.

It is available for Windows and supports all the major PC headsets (through Steam VR and Oculus SDK). We got early builds for Oculus Go as well that our patrons can play with, but they’re not quite ready. We plan to release on Oculus Quest though and are looking to support AR platforms as well such as Magic Leap.

If you want to use it for commercial or other mass use, you should get the Neos Pro license (which will offer some extra business features as well), which is $75 a month, or less for bulk.

I’m not sure when exactly will it come out of early access, so all I can say is that when it’s ready. There’s still lot of work to do to make it more friendly to newcomers, make it perform better and offer more tools. I release updates pretty much every single day, so we’re moving forward quickly. We’d like to expand our team to cover more ground and go faster once we have the resources to do so.

How do you envision the future of Neos VR? Imagine it 5 years from now!

Oh my god 5 years! 😀 I couldn’t envision where it is now a year ago. But my hope is that in the future it’ll power all kinds of social interactions on both VR and AR and in all kinds of areas. You could have people just hanging out and playing games, students learning together, creators collaborating on all kinds of content, teachers and educators making interactive learning experiences and tools and engineers and scientists using it for visualization and communication about their work, both with public and in their work.

I hope we’ll have a booming economy so people can earn living by building things with Neos and bring the collaborative universe to every aspect of our lives. And I hope it’ll help people to communicate their ideas better and easier. There’s always this gap between imagining things in your head and being able to share it with others, which requires lots of skill. My goal is to make it easier to go from the idea to some implementation.

And most importantly I hope we’ll have all those people interacting with each other naturally. You get really beautiful and unexpected interactions by letting all those demographics coexist and people start riffing off each other’s ideas and creations, a sort of creative amplification.

There are seeds of these things already in the community, but it’s always difficult to predict how will these things go. But at very least it gives it a clear direction.

And what about your future, instead?

That’s pretty much the same as Neos’ 😀 When I’m not working on it, I’m usually hanging out in there with the community. I want to keep working on this for as long as I can, because it’s my passion project and the most crazy thing I ever worked on. As long as I got a good internet connection, tech to work with and food to eat, I’ll be happy.

Add whatever thing that you would like to say to my readers!

Thank you very much for having me on and to listening to my rambling!

If you’d like to check out Neos yourself, you can get it free on Steam. We also have official community at Discord that would love to help you get started and show you around Neos! And if you’re feeling really generous, you can also support this project on Patreon.

---

And that’s it with this interview. I loved the passion with which Frooxius talks about his ambitious project… a project that he’s developing ALONE. It’s impressive. Reading his words, I really felt inspired, especially by his idea of adding creativity (and a bit of craziness) to his technological projects and by his desire of fostering collaboration between all people of the world thanks to VR.

I hope that you felt inspired as well, and if it is the case, please help him someway in building this amazing metaverse!

(And please also help me by subscribing to my newsletter 😉 )

(Header image by Frooxius)