Googles Genie 2 Just SHOCKED The Entire AI Indstry (AI Generated Videogames)

So Google just changed absolutely everything with Genie 2, their large scale AI model that can basically create AI generated video games from a single image prompt. What you're seeing on screen now is the prompt first and then you're seeing the user control the character in this AI generated game. Games are about to change on a level that we can't even fathom. And this is something that allows endless variety of action, controllable, playable 3D environments for training and evaluating embodied AI agents. And this is literally just based on a single prompt. And this can be played by a human or an AI agent using keyboard and mouse inputs. Now this is actually quite different from Google's Genie 1, which was pretty similar. It was an AI generated game, but it was unfortunately confined to a diverse array of 2D worlds. But with Genie 2, this is a significant leap forward as it allows you to generate a vast diversity of rich 3D worlds. Now, Google state that GENIE is a world model, meaning it can simulate virtual worlds, including the consequence of taking any action. For example, that means jumping, swimming, etc. And it was trained on a large scale video dataset. And like other generative models, it actually demonstrates various emergent capabilities at scale, such as object interactions, complex character animation physics, and the ability to predict the behavior of other AI agents in game. Now we're going to take a look at some more example videos of people interacting with Genie 2, where you just use a single image to actually depict what kind of game you want to play. One of the things that Google actually talk about is the fact that Genie 2 responds intelligently to actions taken by pressing keys on a keyboard, identifying the character and moving it correctly. For example, the model has to figure out the arrow keys should move the robot, not the trees or the clouds. And this is really important because the model has to understand what kind of mobject you're going to be controlling and where to move that object. Right here you can see a cute humanoid robot is in the woods and the user is able to control them in various directions. We can also see a humanoid robot in ancient Egypt. This character is being controlled and is able to look around certain environments. We can also see a first person view of a robot on a purple planet. And, and we can see the different controls for forward, backward, left and right. And of course we can see this last one here of a first person view of a robot in a loft apartment in a big city. Being able to control this robot, such as jumping, moving forward, moving backwards is really impressive. This is where we actually have something called generating counterfactuals and this is where they state that they can actually generate diverse trajectories from the same starting frame, which is where they basically mean that even if they all start with the same frame, and they can generate many different outcomes in different scenarios that start at the same exact starting point, which is really interesting because it means that these AI agents in the games can explore different worlds and different realities, all starting from the same point. So this is something that I find to be truly fascinating because it allows everyone to experience different realities, but also allows the AI to get a better world model. Now, the next thing that I'm going to discuss is probably the most crazy. This is where we actually have one of the most incredible things. This is what they call long term memory. Now the reason that this is so important for AI generated games is because the games are generated on the fly, meaning that you need to essentially generate things as they happen. And it's not an actual game engine, which means that there's no base level of code or physical renderings. Now this is a tricky problem in generative AI, but somehow Google have managed to solve this to the point where if you look one direction and then you look back in the same direction, then the same elements and objects still remain. Now this is pretty crazy because a lot of individuals have played an AI generated game before. And I'm going to show you guys exactly what I mean, what problem exists and how Google have done something absolutely incredible. So take a look at this gameplay, which is AI generated Minecraft. And I'm going to show you guys, when I play this AI generated game, what changes. So I want you guys to quickly take a look at this gameplay. This is AI generated Minecraft. And you can see that right now this gameplay is being generated step by step. But something crazy is going to happen. When I look down and I look at a different environment, when I look up again, you can see that the area has changed. This mountain spawned in out of nowhere. And if I look down again and then I look up again, you're going to see that the entire environment has once again changed. That mountain disappeared. And I can do this several different times to generate completely strange, wonderful and random environments, which is why long term memory is needed. Next, what we have that is super interesting is the fact that this model, Genie 2, actually generates plausible content on the fly and maintains a consistent world for up to a minute. So I think this one here is pretty incredible because not only in this demo are we seeing the character have different abilities and different spells depending on what kind of game this is, but we can literally see this character run around, jump and explore for up to an entire minute. Now, if you. It might not seem that impressive to people who aren't in the AI space, but for those of you who have been in this space for quite some time, you'll know that this is remarkably impressive when it comes to generating a coherent scene for this long. I mean, being able to run around in this environment is just absolutely incredible. And I think something that a lot of us in the AI community are constantly thinking at this moment in time is that if this is the worst these models will, will ever look when it comes to AI generated games, we can only imagine playing for hours and hours in AI generated worlds in the future. This kind of thing definitely opens up the creative area for those of you who want to explore different things. And for those of you who are super creative, Google actually also talk about how you also have diverse environments. They talk about how Genie 2 can create different perspectives, such as a first person view, which we've seen before, which is of course really cool. But one of the coolest things is that you can also generate third person views. Like if you're in a driving video game, for those of you who are familiar with those games, you'll know that you sit above the car, right behind the car, and you get the POV as if you're looking over the car in order to drive that car. A very thrilling and really good camera pov. But we can also see is that you can also get isometric views, which is really cool for those explore games where, where you want to explore with a character. Now they also talk about how Genie 2 learned to create complex 3D visual scenes. Here we can see that in this game, a user is running around, well, not running around, but walking around and looking at this mecha robot and then walking right underneath that robot to examine the 3D structure. Now, one of the coolest things about this environment, a lot of people would be asking is whether or not you actually can interact with this environment. And one of the things you can do is, is of course interact. So you can see right here that this character is jumping and is actually popping the balloons in the environment, which is really cool. And you can see the buttons being pressed actually showcase that jumping and of course attacking are things that the character is doing. Once again, we also managed to see another character that shoots a barrel and it actually explodes, which is, you know, if you are a gamer, you'll know that this is exactly what happens when you shoot Those red barrels. And you can see as well that this user is able to go into this place where they're literally able to open a door and enter a new environment, which is truly, truly stunning. Let me take a look at what this means for the future of gaming. The next thing they actually talk about is character animation. So you can see that Genie here is able to animate this character jumping. It's also able to animate this character here that seems to be some kind of Mecha or some kind of, you know, robot character that is walking. That would have essentially a different level of walk animation. Now you can also see that when this character manages to try and climb the ladder, you can see that this animation of them climbing the ladder is also done really well as well, which is super, super fascinating because it shows us that the model is able to generate a vast different amount of characters and also understand that different characters move in different ways. Next, what we do have is we have NPCs. So it actually talks about how you can interact with other agents in game and even have complex interactions with them. My favorite is this one where you can see this user that interacts with this pink, fluffy character and then manages to, you know, I guess you could say, shoot this character, slice the character, or do something along those lines and their character just unfortunately disappears. You can see in this other example, a user goes in the cave and then is able to walk around with another character that seemingly is in there, which is really interesting. And then we have this last one, which is like this Assassin's Creed or Red Dead Redemption style game, which is where you're running around with another character that is seemingly following you on your adventures, which is pretty interesting because these are some really interesting AI generated games. Now what we actually see next Here is how Genie 2 manages to model the physics of these games. So right here we can see a boat that is turning, drifting, and we can see that the water physics that is in the game is remarkably accurate for such a short clip. So this is something that is really interesting because a lot of the times one of the hardest problems for these generative AI engines is to manage the physics correctly. So for these physical games, it's of course going to be something that's really important as the characters are going to have to interact with the objects in the world and of course ensure that whatever it is that they're interacting with, whether it be a tree, a rock, you know, or for example, some water, it's going to have to model that physics in an accurate way. So another thing that we do take a look at when it comes to physics is the smoke simulation. So for those of you guys that do play games, you'll know that the atmosphere is really important for ensuring a great gaming experience. And right here we can see that the smoke physics is something that is quite accurately represented, even though it's in the background of the image. We also do have gravity here, which is a vital factor when determining how a game works, because if you don't have gravity, things aren't going to work properly. And you can see that with this car driving around on this cliffside edge as it manages to drive off the cliff, we can see that it instantly starts to plummet to its death where it remains, and then of course, manages to drive on. We also have lighting, which is remarkable, genuinely remarkable, because this is something that is pretty hard to do even by video game standards with current render engines. And we're seeing that this is something that currently works even in these AI generated games. I'm genuinely not sure how it does, but it is something that seems to be working. Now. What's crazy about all of this is that we have reflections actually working now. I'm not sure it's going to look like those insane RTX graphics videos, but. But the fact that we do have reflections in mirrors and reflections in the street puddles is something that is absolutely incredible. Some game engines even struggle to, you know, accurately model this kind of thing. So the fact that this AI generated game engine, Genie 2, is actually a little bit more advanced than we would have initially thought, considering just how hard it is to get accurate reflections to be done. Well, so another thing that they actually show us with Genie 2 is that this allows you to rapidly prototype new generative experiences. So you can actually prompt Genie 2 with different images generated by Google's AI image model. And then you can see how GENIE is able to navigate these different avatars. We saw the plain one. We can also see this Dragon one here, where we see that it's actually able to get the correct dragon that is flapping its wings. We could also see in other examples, for example, let's say we wanted to do a game about an eagle. We could potentially use Genie 2 to develop many different ideas and prototypes rapidly and quickly, without of course, doing the investment of making the game. Essentially what they're stating here is that in the future, when maybe you've got Genie 4 or Genie 5, you're going to be able to simply use an image, and then from that image you're going to be able to test if the experience is worthwhile exploring as a video game or as whatever it is that you want to explore. Something that I think makes sense now, you can also use images like concepts are, and then also step into those and explore those as real time games. So this is something that is absolutely incredible because this is something that means that this model has out of distribution generalization capabilities, which is completely incredible because most models don't have this, which basically just is a fancy way of stating that look, this model can perform well on tasks that it wasn't specifically trained on. We also have something interesting which is where they talk about using these agents for games. So you can see you have this image. And then they prompted it to walk through the blue door to see what it could find. And then shortly after that they also prompted their AI agent Seema to to use the controls and go through the red door, which was really cool. And then they also had some outtakes, which is where we get these outtakes, which is where this character actually prefers to parkour over snowboarding, which is pretty hilarious because sometimes these characters don't do what you want them to do. And then we also got the outtake of where while not taking any action, a random ghost appeared in the garden, which is pretty funny or pretty scary, depending on what you believe. So I think overall this is something that is remarkably fascinating and, and it's really interesting regarding the future of video games. I know many people have speculated whether or not we're going to be exploring completely AI generated worlds in the future. And I think Genie 2 shows us that there is a real path to that level of exploration. And I really can't wait for Google to cook up something next.