Googles New GameNGen SURPRISES Everyone! (GameNGen Simulates DOOM Videogames)

So, ladies and gentlemen, this news is actually news that most certainly surprised me. If you ever wondered how video games worked, or better yet, could imagine a world where video games aren't created by traditional coding methods, but are instead entirely powered by artificial intelligence. Today, we're diving into a groundbreaking concept from Google DeepMind called Game Engine, a new approach where AI takes the wheel and in generating interactive video game environments in 100% real time that humans can actually play. So let's actually talk about the basics of traditional game engines so you can grasp the understanding of why this research is so impactful and why everyone was stunned. So, games like Doom, Fortnite or Minecraft are built using game engines. Now, these engines are like software frameworks that handle everything from reading your keyboard inputs to updating the game world and then, of course, rendering that on your screen. Now, traditionally, these engines are carefully coded by developers to create an immersive experience. But instead of this, what if, instead of coding every little detail, we could use an AI to generate the game environment on the fly whilst the human plays? And this idea is what we're going to be talking about in today's video. So let's look at gamegen Engine is the first game engine that is entirely powered by a neural model, which is a type of AI that simulates how the human brain works. And what does this mean? Well, essentially, instead of relying on tons of handwritten code, Game Engine uses AI to generate the game's environment and interactions in real time based on what happens in the game. So imagine playing a game like Doom, but instead of the game being rendered through traditional programming methods, it is created in real time by an AI model. And this is essentially what Game Engine does. So you might be thinking, how does Game Engine work? So the system is built on something called a diffusion model. So you have to think of it as a very advanced predictive system that can guess what happens in a game environment by, of course, looking at previous actions. The AI watches how the game unfolds frame by frame, like watching a flipbook animation. And it learns to predict what should happen next. If a player presses a button to the left or to shoot, the AI figures out the new game state and then renders the corresponding frames. Now, Game Engine does this so efficiently that it can run 2 Doom, a classic and complex game, at over 20 frames per second using specialized hardware. And this is crucial. This speed is crucial because for a game to actually feel smooth and responsive, it needs to run high frame rates. I'm not sure if you've ever played a game before and you've suffered from something that gamers call lag. I've experienced it myself. And it's not something that works if you're actually trying to play the video. So this is rather important for, for the consistency and the coherence of humans actually enjoying the game. And usually higher frame rates respond better with gamers. And frame rates don't just exist at 30 frames per second or even 60 frames per second that you might be watching this video in. Some monitors allow for 244 frames, which results in extraordinarily smooth gameplay. This is where we're going to be talking about training Game Engine. So in order to train the AI to simulate a game environment, the researchers followed a two phase approach. Number one is to train an AI agent. So at first they created an AI agent that learns to play Doom by itself. This AI agent is like a virtual player that goes through millions and millions of game scenarios, learning different actions, outcomes and environments. Then number two is generating the game data. The game sessions played by this agent are recorded and then turned into training data. And this data is then used to teach the AI model to predict what the next game frame should look like based on the previous actions and frame. Now, the real world performance of this looks almost identical to the original game. In fact, when human testers were asked to identify whether a clip was from the original game or the AI simulation, they could only do so slightly better than if they were guessing randomly. And this shows us from these preliminary results that AI generated environments are incredibly realistic. Now, in terms of image quality, the AI achieves a level of detail compared to modern video compression techniques. And this means that the game doesn't just run smoothly, it actually looks pretty good too. Now you might be thinking, why on earth is this even important? Well, this is important because this could represent a massive shift in how video games are made. Traditionally speaking, creating a game requires thousands and thousands of hours of coding, designing, testing and iterating. But with AI driven engines like Game Engine, creating games could become way faster, way cheaper, and more accessible. So imagine being able to create a game world simply by just describing it in words or by drawing a few simple sketches. And this could theoretically open up game development to a whole new group of creators who don't necessarily know how to code, how to program, or even how to build a game. Now, these implications don't just apply to Game Engine. This technology could be potentially applied to any interactive software. Think about virtual simulations for training, education, or even entertainment, where AI generates the environment based on the User input in real time. Now, there were some limitations of this, which was solved quickly. One challenge the researchers faced was something called autoregressive drift. Simply put, the AI generates more and more frames in a row. And as these small mistakes can accumulate, this leads to unrealistic results that look nothing like the actual game. Imagine a painter who's copying a painting line by line, but with every line, the mistakes add up until the whole painting looks off. To solve this, they introduced a technique called noise augmentation, which basically means adding a controlled randomness to the training process. And this way, the AI learns to correct itself and stay more aligned with what the game should actually look like. Now you might be thinking, what, what's next? Are Google just going to dump this research? Are they going to continue going? So the researchers behind Game Engine are actually quite excited about its potential, but they also acknowledge some limitations. For instance, the AI's memory is limited to a few seconds of game history, which can sometimes lead to inaccuracies. Also, while it works well with Doom, which is a relatively older and simpler game, future versions will need to handle more complex and of course, modern games. Now, despite these challenges, Game Engine represents a big step towards a new way of thinking about interactive digital environments. And that idea of AI generated worlds is no longer science fiction. It's quickly becoming a reality. Now, one of the things I did want to do is I did want to do some experiments myself on certain websites. Using today's available tool, with current image generation techniques, we can actually provide certain environments with an image that looks rather like current modern game environments. For example, right here, you can see that I've prompted GPT4O to create a F1 game. First person gameplay style screenshot, literally in a couple of seconds managed to create me this image of an F1 car barreling down the long road at some kind of F1 virtual environment. Now, with this, of course, it isn't currently playable, but with new technology that was only released two to three months ago, we can actually animate this in a remarkable amount of time. You can see that in just a few seconds I'm able to get this footage that looks remarkably realistic. Now, the thing about this is that this is currently using runways Gen 3 Alpha Turbo and, and this was literally done within around, I think, six or seven seconds. So you can imagine if in the future it's going to be possible. With the inference speed increases with increasing levels of technology. Regarding AI, could we on the fly generate completely new video games simply from text to image prompts and then of course enter a completely generated world. I think this future is not that far off, provided that there still is more research and development into this. But this quick example showing you guys that you can make stuff like this yourself shows you the AI generated game world might not be as far off as many people have thought. If you enjoyed the video, I'd love to see you on the next one. And hats off to Google DeepMind for such amazing research.