Worlds FIRST Fully AI Generated Game Is Here (Oasis AI Minecraft Explained)

The gameplay you're currently seeing is 100% AI generated. That means that this gameplay isn't prerecorded. It's being generated as someone inputs the keyboard. Now, I want to talk to you guys about something groundbreaking that this company has recently achieved. Can you imagine a world where every frame, every interaction and every piece of the game environment is generated by AI in real time, responding instantly to what you do? Well, that world is here and it's called Oasis. So. So Oasis is the first playable real time, open world AI model game that generates an entire game, albeit in Minecraft, frame by frame, using transformer technology. It's not just a game, it's an interactive experience where everything from the physics to the four game rule and graphics come alive as you play. Now, what's notable about Oasis in Minecraft is that your keyboard and mouse inputs drive everything in real time. You can move, you can jump, you can pick up items and even break blocks, which is all, interestingly, powered by an AI model that learned just by watching the gameplay. Now, this isn't just a giant leap for gaming, but a step towards AI models that can simulate complex interactive worlds and potentially replacing our old and traditional game engines with something more adaptive, more fluid and more dynamic. Now, what's crazy about all of this is that the company is actually releasing the code model weights for local use. And of course, they released a live playable demo, which you can see the gameplay right now. This is exactly my gameplay. I know, it's not that interesting. Now, you may be wondering who on earth is behind this kind of crazy technology? Well, it's actually two companies. One of them is called Deckart. Now, dec Deckart AI is an efficient AI platform that provides a order of magnitude improvement in both training and inferencing of the largest generative models. Now, leveraging these advanced capabilities, DeskArt is able to train foundational generative interactive models and make it accessible for everyone in real time. This company has raised $21 million in seed funding from prominent investors such as Sequoia Capital and Zee Ventures, and and they aim to continue pushing the boundaries of generative AI. Now, this company collaborated with another company called Etched AI. Now, Etched AI is where things get absolutely insane, because Etched AI was founded in 2022 by Harvard dropouts Gavin Uberti and Chris Zhu. Now, this company has actually positioned itself insanely as a challenger to Nvidia in the AI hardware space by focusing on specialized chips rather than general purpose GPUs. And they've raised $120 million in funding to develop Their incredible Sohu chip, which they claim can outperform Nvidia's GPUs by a significant margin when it comes to transformer models. Now, you might be wondering, okay, they've got a Sohu chip. Sounds pretty crazy, but what is that? So the Sohu chip is an ASIC designed specifically for transformer models, which which are basically the backbone of many generative AI applications. For example, text generation, image generation, and video generation. Unlike general purpose GPUs that need to support a wide variety of architectures, SOHU is just completely optimized only for transformers. And this specialization allows it to achieve a much higher performance and efficiency by eliminating the unnecessary components and control flow logic that general purpose chips require. So they've basically created a chip that is just completely focused on what AI is doing, especially with our current technology, and that's where they view the future to be. Now, we can see this in a short clip from the CEO actually talking about why he's betting big on his chips.

Well, so who's a kind of AI chip that is only able to do one very specific thing? An accelerator from Nvidia or Google is flexible. It can be programmed to run many different kinds of AI models like convolutional networks, LSTMs, or transformers. Right. Our chips are different. They are only able to run this one very narrow class of model, which we call transformers. But when running these kinds of models, they are vastly more more performant, more than 20 times higher throughput. Well, we're able to burn the transformer algorithm into the chip. We are building our own silicon and our own servers. And this enables us to get more than 20 times higher throughput in terms of output words per second from models like chat, GPT or llama. But you're also able to get much better latency numbers if you measure the time to the first token. We can cut that by a factor of 20 as well. Well, we are taking a bet. Unlike every other company in the market, our chips can only run transformers. So if transformers change dramatically or go away, then we'll be in a bad place. But if we're right and transformers keep being the dominant way that AI models work, will be the most performant chip on the market by an order of magnitude.

So now you might be wondering, okay, how the hell does this even work? Well, the core idea behind Oasis is how the AI generates every part of the game in real time. Imagine you have a blank canvas, and each time you press a button, the AI draws the next part of the game frame by Frame as you play. And the technology behind Oasis has two main approaches. So essentially we have number one, which is the diffusion models. And diffusion models are essentially a kind of reverse noise removals. So if you start with a blurry picture full of random static and the AI gradually cleans up until you get a clear image. For Oasis, this process isn't just used to create pictures, but also to generate video frames, making the gameplay smooth and interactive. Then of course, we have Transformers and attention. Transformers are like a very attentive friend who remembers everything you just did. And in Oasis, the Transformer AI page pays attention to both what happened and before and what you're doing now. And this way it can generate realistic video frames that make sense in the context of the actual game. Like if you jump, it remembers where you landed and uses that to generate what comes next. Now, of course, since this would take a ton of computing power to generate every fame at full equality, Oasis uses something called a Vit Vae, which is essentially just a fancy name for a compression method. And they use this to shrink down the amount of data that the AI has to handle at one time. And this way it focuses on the important stuff rather than worrying about every single pixel. So the big breakthrough with Oasis is that it's actually combining all of these technologies to make an entire interactive game world come to life in real time, based entirely on what you're doing without relying on pre programmed graphics or rules. It's essentially an entirely new way to create and experience games. Now, thanks to their efforts, they can now create each frame in about 47 milliseconds. And this means that real time gaming becomes possible, although it's pretty demanding on current hardware. The Real Time Game Changer. The real game changer will be a new type of hardware called Sohu, which is being developed specially to run these kinds of AI models much faster and at a lower cost. While Oasis can run on current GPUs at 360p, hear this. The Sohu hardware could make it possible to run in full 4K, allowing for a much better experience. Plus it could make running Oasis cheaper, allowing more people to use it without high costs. So in total, it looks like this is just the beginning of what we're starting to see with real, real time generated AI games. Now, the crazy thing about all of this is that this company isn't the first company to do this. There was actually another company that most of you guys know called Google. Now, Google did this with everyone's favorite game, which is called Doom. Now, they called it Game engine, and it's an innovative game engine created by the researchers at Google that operates using a pretty similar technology. So essentially, game engine, which is quite similar to what we have today, works to essentially generate images or videos, but but adapted for interactive gameplay. And with GameGen, what they wanted to do to demonstrate the capabilities, they used it to simulate the video game Doom. Now, Doom was chosen because it's iconic and many people have done crazy different things with Doom. And of course it's an open source version that the researchers could use. Now what's crazy is that the gameplay you're seeing on screen right now is completely AI generated once again. And it was performing impressively, generating game scenes at 20 frames per second, which is fast enough for real time gameplay. And the AI, AI not only recreated the graphics of Doom, but managed the game mechanics like keeping the player's health, ammo and interactions even with the world, such as opening doors or fighting enemies, which is completely crazy. Now, they kind of trained this in a different way. They actually used an AI agent which was trained to play the game and then record its actions and resulting game states as it plays. And then the recording was basically the training data for the next stage. The second stage basically involves what we spoke about before, using the diffusion model and then of course using that to make interactive gameplay. Now this was pretty crazy, of course, as you're about to see. This is incredible because it allows for real time game. But there is one problem that these models do suffer with which I'm about to show you guys in my gameplay. So of course this is our first area where we can look and decide at where we like to explore. Now you can see we've got village output, dense forest, rugged coastline, desert expanse, and I got to be honest, it doesn't really matter which one you pick, because this game is AI generated, you can EAS easily transport around, which I will show you. So immediately you are beamed off into this place that seems like a Minecraft world. And of course, like I said before, this is completely AI generated. Now one of the first things that you'll notice is that when you move your cursor left and right, it's actually quite remarkably slow. That's of course being the fact that this game is generating these frames whilst I'm inputting them, so I don't expect it to be too fast. Now, of course you can see that the resolution is quite low as of course this is something that makes sense considering the fact that it's AI generated and getting extremely high quality outputs is going to be very, very hard. Now, of course, in the paper they speak about how you do get some rough outputs. Right now, that's exactly what you can see here. And one of the favorite things that I really do like about this model is the fact that anytime you would like to teleport around, you can simply look at the ground and then look up again and. And you'll see that you're in a different location. For example, let's say we are now in this snow biome. You can see that if I want to teleport to a different biome, if I just look down at the ground, I then look back up, you can see that the environment has changed. So, yeah, I don't really like that. Let's look down again. And then if I look up, you can see that there's some wood next to me, so it might build a structure. And now we're right beneath a sand biome. So this is a game that is basically generating the next frame based on the previous frame. So. So that's why when you look at the ground and then you look back up, you can see that the entire environment has changed. Now, if this game manages to get more memory, where it's able to take in not just the last one frame or two frames, but the last 20 or 30 or maybe the last two to three minutes of the game, then you would get a lot more character consistency. Now, surprisingly, you can actually manage to interact with blocks. You can pick a stone pickaxe, you can enter different blocks. And interestingly enough, you can see that the Minecraft sand physics just aren't there, which I find to be remarkable and kind of interesting, because when we look at how Minecraft sand is, essentially what usually happens is that the sand will drop once one of those blocks have been, you know, broken. So I'm surprised that this engine didn't manage to get that down. I guess potentially what could have happened is that in the training data, you may have not got the. That recorded. So all AI models and stuff like that, they're all based on the training data. And, well, if you're trying to be a Minecraft Pro, this is essentially sandstone, so that might be why it's not there. But essentially what I'm trying to say is that this gameplay depends on the AI generated data that you have. So if you have data that's super high quality, and you have data that is very specific to certain biomes, then that data is going to be reflected in the gameplay. So, for example, right here, what I'm going to try to do is destroy this bottom block and then this top block here is supposed to drop down. But like I said before, this is showing you that in their data, I'm guessing that they didn't account for this because the block just. Yeah, like right there, it just spawned back in. But it's really uncanny because I can, like, change my hot bar menu. I am trying to eat, but I don't have any hunger bars, so I'm guessing that's why I can't move much. But you can also place blocks, which is really cool. And so, for example, what I can do is I can jump down and I can place blocks here and I can place some that some disappear, some turn into sand, as you guys can see. And then, yeah, I'm out of blocks now because the value changed as it was being AI generated. And I'm guessing that if I continue to do this, it's going to spawn me in various different locations. I wonder how high I can get if I continue to go even higher. Now, this is pretty confusing because we can see from the bar that it looks like I'm in survival mode, but I'm not really in survival mode. So it's kind of interesting. It's kind of weird. It's. It's. It's really trippy. It's kind of like a dream world if you're dreaming about Minecraft. But yeah, it's definitely a little bit weird. But you can interact with these. You can look up if you also want to change as well. And you can see you can get into random areas and. Wow, there's like a crafting table here. I didn't. I didn't see that. So I want to see if I can interact with the crafting table. Let me do that. Can I interact? Oh, there's a bunch of different items there. Wow, that's. That's super weird. Oh, I can continue. I can end the game. I mean, I've got 36 seconds left, so I don't remember how to do that, how to get out of this. Okay, we got fences. Is this food? Okay? Nope, it's fences. So, yeah, when you're building stuff and you're destroying stuff, things do kind of mesh together. It is kind of weird, but like I said before, this is just a quick part of the gameplay that I wanted to show you guys how strange this entire thing is. But I think we'll get more of this in the future in a way that actually makes sense.