Why you should use now generative AI in your metaverse company. Or maybe not

We are living the generative AI hype, with everyone getting crazy about ChatGPT. And every day I read articles about how AI can disrupt the metaverse, how ChatGPT and metaverse are the perfect match, and especially about how the fact that if your XR company doesn’t have an AI strategy, it is already old and ready to die a terrible death. I have my point of view on all of this, and I would like to tell to you. I know that now you only read ChatGPT’s opinions on everything, but for once, also listen to the meaty Tony.

The incredible moment of generative AI

Generative AI is incredibly cool. The moment I tried ChatGPT and saw its potential, I was literally stocked. Not only it was able to answer my questions better than the usual chatbot, but it had superpowers like:

• being able to speak all languages
• being able to remember the context of a discussion and carry it on very well
• being able to write pieces of code in all programming languages with very good quality
• being able to understand and fix already existing pieces of code
• being able to understand if you were trying to do harmful use of the AI
• being able to write meaningful pieces of texts
• being able to give me an explanation about basically everything without having to look on Google

For the first time, I saw an AI that had an almost believable way of speaking, and that was able to substitute many tools that I’m still using now. The first one of these is Google: ChatGPT is able to give me an answer to everything in a human-readable way without me having to look on Google. And this is the reason why Google has rushed to launch Bard: for the first time in many years, there is a chance that Google sees its business disrupted.

ChatGPT is impressive, it is great, and it has a quality high enough to be already usable. I know already many people that are using it as a support for their development efforts.

But ChatGPT is not alone: StableDiffusion, Dall-E, and MidJourney are being used to create images every day now. There are many people using them to create images for their PowerPoint presentations, for instance. Some 2D artists are using these tools to create images that serve as starting points to create a final artwork. There are also some tools for generating music, 3D assets, and many other things. We’re living in a sci-fi moment where Generative AI can create assets, something that probably we were not expecting was happening this fast.

AI and the metaverse

The “metaverse” has been the past hype, and “generative AI” is the new one. It’s obvious that many people are mixing the two, and are wondering about the potential disruption of “generative AI in the metaverse”. The common consensus is that Generative AI can help create assets in virtual worlds, so content creators that don’t know how to do things can let the AI do that: for instance, generative AI can create 3D models for your virtual room. ChatGPT could be used to create intelligent NPCs that speak with the user. Also, this technology can be used to support the developers that are programming virtual worlds: every day we see examples like “this AI saw a game once and then it wrote all the source code of it”. There are actually many links between artificial intelligence and virtual worlds creation.

The futurists on Linkedin are all rushing to say that “If you don’t integrate Generative AI in your business today, you will be disrupted and your company will close”. But is it true?

Hype and technologies

I’ve to honestly say that I’m too old for this shit. It’s 8 years that I am in immersive technologies, and I’ve already seen many hypes coming and going away. Do you remember Web3? “Oh NFTs are going to change everything, artists are getting a fair share of their music, and the blockchain is the future upon which all our life will be based!” And then, after some fantastic use cases like people making NFTs out of monkey JPEGs and Paris Hilton arriving on The Sandbox, the technology has been basically thrashed away by the mainstream tech world. And the metaverse? The M-word? Well, after having made the success of many marketers with their amazing posts like “How we will go to the toilet in the metaverse” and having seen some people becoming Chief Metaverse Officers, which is a job that means nothing but is paid well, again, we saw everyone losing interest when they discovered they had to wait many years for it to become reality.

Now it’s the time of Generative AI and the posts about “how generative AI can generate the best porn of your life”. How do you think it will end? Yes, exactly that way: people will realize that generative AI doesn’t solve every problem of our life, and the hype will deflate, too. And all the metaverse experts become AI experts will become something else, like I don’t know… 3D-organs printing experts? I’ll just watch this hype come and pass by like all the others I have seen in my life.

Shouldn’t your XR company think about an AI strategy?

Don’t misunderstand me: generative AI is amazing, and I clarified it in the first paragraph of this article. And I think that if your role is the one of thinking about the technical strategy of your company, you should be informed about it and you should experiment with it. Exactly like all the other technologies, hyped or not. You should be informed about all the main technologies around you, how they are mature, how they are relevant for your business, and understand how and when integrating them. So yes, you should think about AI and how it can be relevant for your business, in the short, medium, and long terms.

But should you rush into integrating it? I don’t think so. In VRROOM we had a few brainstorming sessions about generative AI and how it could be relevant to our VR concerts business. The interesting fact is that both us developers and the artists came to the same conclusion that there is nothing disruptive for us in the short term. There are many nice-to-have, many things to experiment with, but nothing that “if we don’t integrate AI now, we are disrupted”. There are two options for this conclusion: either in the company, we are all stupid (and looking at myself in the mirror, it can be), or if you remove the fog and the hype, you realize that generative AI has still a lot to do to become practically useful in our field.

We see the long-term potential, but the long-term is not here yet, so what we can do is keep following the field, keep doing tests, and when we see things coming to the point that is interesting for us, then run to integrate them into our product. We have no rush. We will add AI-driven features over time: maybe something small now as a test, then something bigger later when the technology will be more mature.

So my advice to you all is to have a very practical talk about generative AI in your XR company, ignoring all the noise that there is outside, and understand what can really be useful for you, and how to start experimenting with it, without wasting all your budget in something that is not useful or is not ready-enough for you.

Why you shouldn’t rush in integrating generative AI

I guess now you are curious about why I’m not rushing to implement AI into my business. Why don’t I write all my articles with ChatGPT? Why do I still write source code? Why do we still manually craft a lot of our virtual concerts? Well, there are many reasons. Let me tell you just a few to give you some examples.

The lack of context

When you are writing source code, you usually don’t operate in a context like “write me a script that makes a Unity cube red”. This still happens a lot of time, and in these cases, ChatGPT could be a great substitute for a developer or could help in writing code someone that doesn’t know how to code (e.g. a 3D artist).

But most of the time as a developer you have to write code inside a larger context, knowing what the other developers are doing, having in mind what the rest of the application does and what is the overall architecture. You have also to keep in mind all the code policies and the company vision. It is not that straightforward. There is in your mind a lot while you are writing that code, and current generative AI doesn’t have it.

There is also the context of all the other source code that has been written. Of course, you could feed all this information to a ChatGPT-like system, but would you really want to send all your source code to an external American company just to have the help of the AI? Personally, I would never do that.

On the high-level side, when I write a software architecture, I have also in mind lots of business reasonings and technology reasonings (e.g. how technology is going to evolve in the future), that currently, no AI can mix together.

Context is relevant also on the artistic side: generative AI can create 2D and 3D assets for you. But do they all follow a common style and art direction? Are they completely coherent with the lore of your experience? Do they follow all the directions given by the art director? Usually not. Of course, you can start from the AI outputs and modify them to give this coherence, but you must have the skills. The promise of “everyone can be a creator without the skills thanks to AI” is not coming today. Making a full coherent virtual world without any kind of artistic knowledge is still almost impossible.

Some months ago I was also talking to a writer that mentioned to me “the human context” inside which art happens. Usually, when someone innovates in art, he/she lives in a human context that is historical (maybe there is a war) and artistic (all the art happening around him/her, and all the ideas exchanged with all the other creatives around him/her) and creates something totally new out of it. This is still something that AI can’t feel, and can’t do. Usually, it can make a mix of many current things, but it can’t have the vision of creating something totally new from a whole human context.

The creation errors

Not everything ChatGPT outputs is correct. And as I’ve written before if you have the skills, that’s great, you can fix it. But what if you don’t understand coding and don’t know what to do? Well, you are out of luck and you have to ask for the support of someone.

This is even more evident in the art field. When you try to create something with MidJourney, it usually looks creepy and uncanny. We all use those images because they are fun to use, and for now, they have no copyright protection, but rarely they produce what I had in mind. The fact is, I don’t have the skills to fix them, so I usually accept what I get.

You get what you get

Generative AI generates what it wants. But again, rarely is what I had in mind. There are many “metaverse” applications rushing to integrate generative AI, but when I see the videos, I am usually puzzled by the results. You can write a prompt to obtain something uncanny that you can put in your world, or as a texture of your avatar, or whatever. Honestly speaking, I would rarely use that.

When I speak with real artists, I spend a lot of time telling what I would like to see, and usually, it’s a long trial-and-error process… and still, at the end of the day, I rarely obtain exactly what I had in mind in the beginning. Imagine what can happen by writing just a text prompt to an automated system… almost never I get what I was thinking. Most of the time I use it anyway, but I’m not completely satisfied.

Talking about 3D models, there are some experiments about automatic generation, but the results are almost never optimized well for XR usage. And again, almost never follow the coherent art design that there is in the rest of the application.

Everyone is having fun using ChatGPT to write essays. But reading them, I find them totally heartless. It’s like reading those posts about “5 reasons why the metaverse is disrupting your business” written by marketers that have no idea of what is the metaverse (no one does), and what is a business, and what the word “reason” means. It’s very cool that ChatGPT can write, but I would never read a book written by such a system for now, because it’s so flat and boring. I‘ve discovered that many people read my blog because “it’s human”, because “they can feel me”. Humans want other humans, and for now, ChatGPT can’t give that feeling. So you can obtain a simple essay, but most of the time, it is good-enough, but never great.

This means that if an AI system generates things for your users, almost never your users get something truly beautiful, and almost never what they really wanted to create in the beginning.

Different choices

ChatGPT can disrupt Google, I totally agree with this. But still, there is something that Google gives you that ChatGPT currently does not do very well. And that is the power of different points of view.

When you ask something to ChatGPT, it usually condensates all the many articles it has read in a short paragraph of what is basically “its opinion”. Many times, it is enough. But there are times when you have to evaluate different options and the pros and cons of all of them. In my job as a CTO of an XR company, this happens a lot of times: I have to take a decision, and I have to evaluate the pros and cons of all the scenarios. To do this, I look on Google and try to find opinions on them. Usually, what I love finding is people debating on the topic on Reddit or StackOverflow. I’m not looking at a “solution”, because most of the time there is not a real solution, but different options with pros and cons. I take hours to do that, but after that, I have in mind exactly not only what to do, but also why I’m taking this decision, and why and when I should change it in the future. It’s time well invested. I would never get it with just some queries on ChatGPT.

Tool refusal

I was kinda shocked when ChatGPT refused to write a joke for me related to a friend. The system tries to detect when you are trying to use AI in a harmful way and this is good, but I found it wrong on so many levels that a tool refused to do something I wanted. It is like if I had a hammer, and the hammer decided which nails to hit and which not. I would never use it.

AI tools may be programmed to avoid doing something that they deem inappropriate. But if I want to add giant cocks to my room in a social VR world, I should be able to do it. Of course, the company could ban me for not following the community standards, but should be my decision to add them or not, not the one of a tool.

Your brain needs training, too

The more you offload your thinking to generative AI, the dumber you get. I think there is a threshold to keep in your mind about what to do yourself and what to give to do to an automated system. If something is very easy or is it something you just need to copy-paste, it’s good to make an AI do that, so you spare a lot of time. But if it is something that is better for you to know how to master and modify, then my suggestion is to do it yourself (maybe letting the AI help you a bit).

The unknowns of the IP

There are a lot of discussions about who owns the IP of the elements generated by the AI. If Stable Diffusion takes thousands of pictures from the web and creates a new one… should the IP belong to Stable Diffusion or a bit to all the artists whose pictures were used to generate the new one? The debate is still ongoing and I guess that soon there will be regulations.

The pleasure of crafting

As humans, we all like creating. The purpose of art is not the one of producing something, but of expressing ourselves.

I enjoy writing code. I like to write blog posts like this one: I’m not doing that to have a product, I’m doing that to express what I have in my mind and discuss it with the other people in the XR community. What would be the point of letting an AI system produce it? I don’t want to just publish a summary of all the web articles on this topic, I want to write my own personal opinion.

The same goes on for all the rest: in a concert platform, we could offer the possibility of having AI-generated music, and it could be fun…but people that make a concert want to express their music, want to express the music that they have written to communicate their feelings in artistic form. It’s a different thing.

AI seems good when you need a product, but for art, in my opinion, it misses totally the point.

But in the long term…

… AI will be able to do all of this. At the end of the day, if we manage to create an AI that works exactly like our brain, it could do everything we do. I totally agree, but this is not happening now… maybe it’s happening in the infamous 5-to-10-years in which we expect everything to happen (Vitillo’s Law).

But your users are there now, and your company should be profitable now. Not in 5 to 10 years.

So what to do now to integrate generative AI with XR?

I’m not going to write the use cases for you on how to integrate generative AI into your XR business. There are already millions of articles out there (some written by generative AI itself) on what are the potential use cases.

On the higher level of things, as I’ve told you, my suggestion is to be very practical. Think about the potential there is now for both your internal processes and your product. For internal processes, for instance, developers can use ChatGPT for writing simple pieces of code in an automated way and artists can use images produced by MidJourney as a source of inspiration. For your product, you can evaluate if some of these tools can already produce assets that have enough quality for your users (e.g. ChatGPT could be good to create a chatbot or a talking NPC).

Read a lot, and experiment a lot. Brainstorm with your peers. Share your results with the community and get feedback. Try to understand more or less when the various technologies can be relevant for your specific use cases. Ignore all the noise outside and forget the hype. And doing this, most probably you will realize that you have no urgency to implement many things now, but that you can just think about a strategy that for real will make generative AI useful for your business in the long term.

I‘m actually quite bullish on the potentialities of AI mixed with immersive realities. But I know that I have to wait the right time to implement the various ideas that I have in mind. So be practical, and over the years, you will actually succeed. Because it’s true, these technologies are going to disrupt all our XR and social virtual worlds field, so you must surely work with them. And they are not the only ones: for instance, 6G and edge computing are going to play a very important role, too. But you have to be practical and realistic to make your dream become reality.

… but investors want me to implement AI now!

If your investors can only follow the current hype without any kind of analytical thinking, then you have the wrong investors. Sorry about that.

In this case, ask ChatGPT how to deal with them…

(Header image generated with Craiyon)