Dr. Mark van Rijmenam outlines risks from the convergence of generative AI and the Metaverse: intensified polarization via personalized/filtered views and real-time deepfakes; covert manipulation through AI-driven NPCs and targeted in-experience persuasion; misinformation and identity-impersonation risks; and copyright conflicts from model training on scraped images. He proposes three countermeasures—education, technical/content verification, and regulation—to give individuals control over data, identities, and digital assets.
Chapters
If we have to look at the dark side, the bad side of the convergence of generative AI and the Metaverse, I think there are four challenges that we need to consider. These are polarization, manipulation, misinformation, and the issue of copyright. So let's discuss each of them briefly. First of all, polarization. In today's world, polarization is a problem because we all live in our own filter bubble driven by toxic recommendation engines. We all live in our own world and we believe that our own world is the truth. If I'm on LinkedIn, I think based on the content I get, I think everyone in the world understands everything about the Metaverse and generative AI, because that's all I see. Obviously, that's not the case now in a world in the metaverse world where we have this immersive Internet that we are literally linked, and in the Internet, it becomes possible to have, with sleek AR glasses, to filter out the people that you don't want to see. So imagine that you walk in the streets and say, I don't like women with blond hair, or I don't like men with brown hair. Let's just filter them out and replace them, I don't know, with a giraffe, that becomes possible. And just because I thought about it probably means that somebody at some point in the future will make it and people will use it. And I think that will drive polarization to a whole other level that we're not even used to today. And then of course, we have the issue of real time deep fakes, and that will be beyond what we know today. I'm currently working on creating a digital twin of myself. A digital twin that looks like me, sounds like me, moves like me, walks like me, and really sort of is me. So next time that you see me on stage, it might be my digital twin, but if I can do that of myself, you can do that too of me. So my images are online, my voices are online. So you can create me and you can be me in the virtual world. Now, people are gullible. So if people think they see me, hear me, and it sounds like me and it looks like me, they probably think it is me. If that happens to me, that would be annoying, but not problematic. If it happens to a CEO or a world leader, that would give a much bigger problem. Let's look at the short video.
I am not Morgan Freeman and what you see is not real. Well, at least in contemporary terms, it is not. What if I were to tell you that I am not even a human being? Would you believe me. What is your perception of reality? Is it the ability to capture, process and make sense of the information our senses receive? If you can see, hear, taste or smell something, does that make it real? Or is it simply the ability to feel? I would like to welcome you to the era of synthetic reality. Now, what do you see?
A really cool clip from a Dutch artist who created that to show what is possible today. Now imagine a couple of years from now what will be possible. And we will be able to create deepfakes that we cannot recognize as being deepfakes. And we only have to look at what happens when we can imitate an identity. To look at Twitter. When Twitter announced their blue verification subscription, which within I think in half an hour, people were imitating brands and brands lost billions of shareholder value because of that. So this is really problematic. Now of course, if we then take this a step further, we can move to manipulation. And if we have these artificially intelligent non playing characters, these AI can be used to sell us stuff. So imagine you are in a virtual experience and you see several NPCs scattered around to. Now imagine that you are looking to buy a new car. And all that information is available of course, because all your data is available to the owner of that game. And the owner of that game might then sell that information to an Advertiser who allows NPCs to talk about that particular car brand that you are interested in. And because we can know if you are listening, we can know if you are watching what they are saying, they will adapt whatever they are saying based on whether you are listening or not. And to you it just looks like characters in the game having a conversation. But in the meantime you are being manipulated to being persuaded to buy a car. Personally, I think that's a lot worse than having display ads that follow you around the web, because in this case you do not know whether you are being manipulated or not. Then the final issue that we have to look into is the issue of copyright. Getty Images, the famous big known stock company, has said that they're going to remove all AI art. Same for ArtStation. And at the same time Getty Images is suing Stability AI, the company behind Stable Diffusion, for literally scraping the entire website of images. How do they know that they did that? In some images that you generate with Stable Diffusion, the watermark of get the images is even still visible. So they were sort of red handed. So the question of how do we define copyright in an age of generative AI is really difficult to answer. We'll have to see how that will evolve in the years to come. So will we end up in a hyper personalized AI generated filter bubble? That's of course the question. I think probably yes. But it doesn't have to be that way. And we really have a chance to take control to our own lives and to ensure that we have control of our own data, of our own digital assets, over our own identity. And I believe that there are three ways to overcome these challenges. These are education, verification and regulation. So if we look at education, I think we need to educate the world on how to deal with digital technologies. We've sort of sleepwalked in, into this digital age where big tech has created amazing seamless technology, but we never thought about carefully of what this all means to us. The technology is evolving so fast that I cannot blame anyone that it's difficult to keep up. It's even for me as a futurist, it's my job to keep up to date and even for me, it's difficult. So if you're a CEO of a very large company, I cannot expect you to know everything that's going on. If you are a parent of two or three children, I can't expect you to know what the impact is of chatgpt on your lives. But we do need to understand that and we do need to educate our children, because who are going to educate our children? It's not the parents because they don't know, it's not the teachers because they don't know. So who is going to teach our children how to behave with this technology? I think that's very, very crucial if we want to ensure a thriving digital future. That's exactly what we're focusing on with my research institute. Now, if we are able to educate the general public and the enterprise organizations and the governments, then we need to move to verification because we want to know whether some content is actually being created with AI. And OpenAI is developing watermarks and there are other companies working on creating those watermarks, but it's nothing, not really. You know, it's not very accurate at the moment. At the moment I think it's like 25% accurate, which is worse than throwing a coin whether something is generated by AI or not. But moreover, if I am interacting in the metaverse with an avatar, I want to know that that avatar is being controlled by the person that I think I am dealing with. And not that I'm dealing with something totally else. I'm having a phishing act going on. So maybe we need to link our avatars to soulbound tokens. But then my wallet might get hacked, which also happens all the time. So maybe we need to link that soulbound token to. To biometrics. These are all important things to think about, how to verify our identity in an immersive world. And then, of course, the third one is regulation. We need to create regulation that will ensure that we build the next iteration of the Internet with the right rules in mind, that we don't end up in a world where big tech controls our data even more than they do today. Because, mind you, in a world of generative AI and the metaverse, we will create 100 times more data than we do today. So we need to make sure that that data belongs to us and not to a centralized identity. And I think regulation that is there to encourage innovation, but also move it into the right direction, is very, very.
