Googles Latest Breakthrough is HERE! (Unbounded AI) Generative Worlds

So Google have presented some really interesting research that regards generative AI games. On this paper you can see that it shows Google presents Unbounded, a generative infinite game of character life simulation. And this is basically the architecture that is needed to create fully AI generated games where you have an entire narrative story developed by the player who is able to control exactly what is going on in the story. So I'm going to break down exactly how this works because a lot of the stuff here does seem quite rudimentary. But with the way things are headed, I wouldn't be surprised if within the next two years we have entirely AI generated worlds and games. And trust me, when you see the things that are in this video, you're going to start to see why this is going to happen. So most of you guys understand what traditional game engines are like. You can see that with traditional game engines there are essentially these rules and of course code pieces that are governing exactly how the game works. There's engines behind the game formula. There are of course codes, there are of course physics, there are of course mechanics to how these games work. Now of course, this is something that has been traditional in gaming for almost 20 years. This is how gaming traditionally works. And essentially what Google is aiming to do here is basically set the foundations for something that completely changes this entire reality. And I wouldn't be surprised if this happens in the future because my, oh my, gaming is going to get a huge change in a variety of different ways. Ways. So essentially we've actually seen something like this before. If you guys have been part of the gaming community, you'll probably remember this trailer from Gamespot that was essentially going to be an AI generated game. Now essentially what this was is this is pretty similar to what Google are trying to do here. And you can see it says every bird, every trish, every, every ocean, every ruin, every star was procedural. And basically this was a game trailer that was trying to have a game that every single world that you went to was completely AI generated. So imagine you went to a star or a planet, and as you went to that star or the planet, it was able to design the ships, it was able to design all the elements, it was able to design the kinds of things that are lurking in the shadows for you. Everything there was generated on the fly just using LLMs to creatively generate that. Then of course, because generative AI is working in so many different fields like text, video, I mean images, sound, all of those things can be used to create that entire reality. And this is what the game tried to do 10 years ago. And essentially, of course this game, no Man's sky, didn't actually work, unfortunately at the time. But with current technology, Google are basically saying that, look, this kind of thing is of course possible. Now one of the things that you need to do is you need to of course have custom character creation and basically remember how AI generated characters are a thing. Now like you can literally make any kind of character you can think of on midjourney and you can customize that character in a variety of different ways. You can adjust their, you know, head size, you can adjust their face size. These are the kind of things that we're basically going to be able to do in the future with certain games and we're going to have a ridiculous level of character control and consistency regarding that. Now one of the things that they wanted to do with this research was they wanted to maintain their character consistency. So in this paper they talk about how they use a variety of different methods to actually maintain their that style of character consistency. And it's pretty cool because if you go on mid journey right now and you type in consistent character, and you type in consistent character, a male, a female, redhead, explorer, you will literally get yourself, you know, a few images, a bunch of different images that are all of the same character doing different poses, different animation stances. I mean, it's really, really cool to see where the field already is with regards to character consistency. So this is something that I thought was really, really cool. And, and this is essentially one major part of this because when you're generating the real time game, you actually need to ensure that your character is consistent. One of the things that you're going to be doing here is if you have a fully generated game, like not just the character is consistent, you'll actually have something that is pretty incredible. So if it's not just the character, you're going to need to ensure that if you're generating every frame with your generative AI tool, you're going to have to make sure that the character does remain consistent. And this is what they aim to do with their software. So another thing that they also do is they also make sure that the environment that gets generated is remarkably consistent. So I actually generated my own character in ChatGPT and you can see here, this is a very basic example. So if you go into ChatGPT or GPT4O right now and you prompt the AI system to do anything, like let's say, for example, you say, give me a screenshot from a video game of a certain character you can describe the character as looking like yourself, looking like a random person. I use the prompt, give me a picture of a random video game character with a hud, with health bars and stamina bars. And this is the image that came up. And what I did then is I then used some other generative AI tools to be able to do something really cool. So what I did was I took my character, I took the health bars and the statistics, and then I used that to then generate other environments. So I generated an environment that was, you can see, in some kind of, I guess you could say green land that looked really cool, looked really playful. I then generated that again and did that for, I guess you could say, some kind of tropical paradise. But once you see that look, you can have your character roaming in many different environments across many different places. You can see right here that this is going to be something that a lot of people are going to have so much fun with. I mean, imagine creating your character and you're able to literally control where it goes. You're able to walk around and say, okay, go to this environment. Okay, let's go to the tree dimension. Okay, let's go to the tropical forest dimension. Okay, let's see if you. What happens if you eat this? What happens if your character eats that? I mean, it's going to be really cool. Now, of course, remember how before I talked about, of course, how they ensured character consistency? Google has a new method because other people have tried this with other methods, and there's a little bit of problem with their method. So you can see right here, the character, the environment, and the semantic alignment, basically, environment consistency just maintains that the environment that they're in actually remains consistent with what it is that you're describing. Character consistency. This is essentially, of course, the character that you've got, you know, whatever image of or character that you generate in the first instance, that character is going to be completely maintained. And then, of course, semantic alignment basically means prompt adherence. So if you do state something really well, it makes sure that it manages to capture the entire essence of your prompt. One of the worst things that does happen is when you input a prompt into ChatGPT and it doesn't efficiently get what you're trying to communicate. So with that, they have developed their own unique system. So you can see theirs manages to capture every single essence of the AI generated world. Now, they actually made some unique advances in AI as well. So one of the things they actually spoke about was the fact that they managed to distill a large language model so the process of distillation is basically where you make a smaller LLM a lot smarter from a big LLM. So it's quite like diluting juice, but it still tastes pretty decent. So essentially what they did was they used two LLMs to play the game together. One LLM acts as a player, the other one acts as the game master. And they recorded 5,000 different gameplay scenarios from GPT4O, and basically they used that distilled LLM and they actually perform very, very well when you compare them to Big LLM. So it was trained on 5,000 recorded gameplay sessions. And of those 5,000 different gameplay scenarios, they managed to really, really drive down home exactly how well this should work into that large language model. So they used GPT4 and now the model is basically as good as GPT4, which is pretty insane now, crazily. Okay, I remember I was always thinking to myself, why are Google building out these really small models? And for those of you who don't know what I'm talking about, most people aren't up to date with Google's AI news just because they aren't like right now in terms of the hype cycle, one of the major players. But they've actually been developing this model called Gemma2 and it's a 2 bit billion parameter large language model. And this one is currently better than GPT 3.5, Llama 2 Mixtral and a bunch of other LLMs. But this one is super, super small and it's able to perform really, really effectively. And they managed to distill all of the characteristics that a larger model would get. And of course the reason why they did this is because if you actually want to interact with an AI model, you need to be able to do that with something that actually has decent latency. So can see right here that this is arguably one of the smallest models you can see compared to, you know, llama 270B. This is a 2 billion parameter large language model and you can see just how crazy the Chatbot Arena ELO is. So Google are really hammering down on these AI systems that can perform really well for these small edge use cases that allow for real time user experiences. And essentially this is how they managed to get this game working in real time. I'm going to show you something after this that's really cool that I think you guys should definitely experiment with because it does show you where we're headed. So this is basically how the environment kind of, you know, works. I guess this diagram is a little bit distracting in the sense that like, everything is just on one page, but this is basically where they manage to track the health, they manage to track the user through many different environments. And this is going to be something that's in real time. So essentially you could say to the user, okay, there's something over there. Why don't you go ahead and explore that? Why don't you find a shop here, why don't you rest, why don't you have some food, yada, yada, yada. And basically you can see right here that with all of these instructions, with all of these different things, with all of these, you know, game environment generations, these are going to be the things that we have in the future. Now, I personally wanted to test something like this. I know it's not exactly like this, but I do think it's gonna show you guys exactly what the future holds us for when we're exploring AI generated worlds. So, no, this isn't any kind of sponsorship, but if you've ever played around with a real time AI generator, you'll know that this stuff is remarkably fun. So let's say I say a man in a suit, but cartoon. Let me do cartoon character. Okay, cartoon character. And let's say he's on the sofa eating pizza. Okay? Like this is the kind of thing that you could do when you have real time, any AI generated stuff. Let's say he's eating, you know, noodles. Okay, we can make him eat some noodles instead, we can make him eat some rice instead. Uh, we can make him eat some rice. We can, can make him eat some. Let's see if this guy wants to eat some chicken instead. Okay, he's eating some chicken. I guess he kind of looks like a chicken. Okay, we could say a man on a suit. We could say a fat man in a suit. Now that he's at all of that chicken. I mean, I notice it probably isn't the best kind of, you know, way to demonstrate this, but imagine you are having this kind of game where literally the possibilities are endless. I know that, you know, some people do want to follow a storyline and a narrative of things that you're supposed to do, but. But I mean, when you think about the future of open World games, you're going to be able to do, you know, let's say a man is on the sofa. Let's say, you know, this guy, instead of fat, he becomes a fit man. Okay, on the sofa. I mean, eating chicken now. Or I could even put, maybe, maybe not fit. I could put a muscular man, but yeah, this kind of thing is absolutely crazy when you have a variety of different ways that you can explore your creativity in AI. And I think it's going to be super interesting to see how video games manage to incorporate this into their future releases. So it will be interesting to see the first game that does this. And I do think that whatever first games manage to, you know, allow you to explore these different characters.