DeepMind built an AI that plays modern 3D games like No Man's Sky directly from pixels, generalizing across different games and sometimes beating specialist agents. It understands 3D space and follows instructions, but struggles with sequences longer than 10 seconds and long-term strategy.
What do I see here? This is DeepMind's new AI playing video games. But not just Atari Breakout, where they built an AI that could play it incredibly well eight years ago. And it is not even like Nvidia's AI that can play Minecraft. It played Minecraft well, but only that. You see, many of these works are tailored to one specific game. If you wish to play a different game, you need a different AI. One AI, one game. And now, this is perhaps the first time an AI can do well at a variety of modern 3D games at the same time. Yes, one AI, many games. And even though DeepMind's new AI learned to play a variety of games, the goal is not quite to maximize the points or to beat the game. No, no, the goal is something else. Here, dear fellow scholars, this is two minute papers with Dr. Zsolnai. Yes. The goal here instead is to be able to follow instructions from fellow scholars. And that is us humans. And it plays in a way that I like to call the holy grail of AIs playing video games. And that is not looking at numbers that describe the game, like coordinates, scores and other data. No, sir. It looks at the screen, reads the pixels, and uses the keyboard and mouse to try to perform these actions, much like a human would. That is incredibly difficult. An excellent paper for two minute papers episode 850. So when we ask it to drive that car in Goat Simulator, well, it drives that car, but it is not impressive car driving. However, what is impressive is that it just looks at a 2D pile of pixels. And from that, it understands the 3D world it represents and how it changes over time. That is not trivial at all. For us, this requires the human vision system. And now an AI can do that. It sees not just a pile of pixels, it sees a bit more like we humans do. That is extremely impressive, especially for these complex 3D games. This was much easier for Atari Breakout, but here, that is a huge leap in just eight years. Wow. And something from the paper really surprised me. I did not expect this at all. Now, hold on to your papers, fellow scholars, because after it has trained on more than one game, it was able to play all games better. Hmm. Isn't that the definition of intelligence? It is able to gather knowledge from somewhere and apply it to somewhere else. So can it do that? Oh my. Look at that. This is amazing. Here is the baseline performance of a specialist agent that played one game for a long time. And here is the agent that played less, but has played several different games. And it beats the specialist AI. At its own game and not by a little. Wow. Now limitations. Clearly this is not human level performance yet the success rate is quite reasonable for a first crack at the problem and we are not looking for 100% everywhere. Even humans are not doing 100% on these tests, so there is room to grow. And what do you think version two will score here? Let me know in the comments below. Also, the length of these sequences is limited to 10 seconds and it cannot do intense longer term strategic planning. For instance, you could ask it not just to jump the fence, but to find resources and build a camp in a strategy game. That is going to be the plan for the next follow up papers on this. But you remember the first law of papers. The first law of papers says that research is a process. Do not look at where we are, look at where we will be. Two more papers down the line and I think they may have broken through. I think that through incremental improvements we can get to something truly special from here. What a time to be alive. And ultimately this is not just an AI to play video games. Although I am quite sure there was lots of fun to be had at the DeepMind lab with this paper. The goal is to be able to create AI systems and agents that can understand us and and help us out in a wide range of challenging tasks in a 3D world. I will note that I had the luck of being able to access this paper a little earlier and by the time you are watching this I am likely on a plane to visit the DeepMind lab to have a look at their newest works. I don't yet know what it will be, but I can't wait to tell you more about it as soon as it will be possible to talk about them. Subscribe and hit the bell icon if you don't want to miss out on those. And thank you so much for watching me reading papers and flipping out together with me for 850 episodes. That is a huge honor. Thank you. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud computer. No commitments or negotiation required. Just sign up and launch an instance and hold on to your papers. Because with the Lambda GPU cloud you can now get on demand H100 instances and they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So the Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today.
