A Two Minute Papers video (Dr. Zsolnai) comparing OpenAI's 2019 hide-and-seek emergent strategies with a 2021 DeepMind paper that trains agents via screen-based, open-ended learning to perform well on many unseen games. The video narrates several gameplay examples (hiding, ramp-building, ball-control, king-of-the-hill), contrasts 'expert' vs 'journeyman' agents, briefly references AlphaFold, and includes a Lambda GPU Cloud sponsorship.
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to see how an AI can win a complex game that it has never seen before. 0Prior training on that game? Yes. Really? Now before that, for context, have a look at this related work from 2019 where scientists at OpenAI built a super fun hide and seek game for their AI agents to play. And boy did they do some crazy stuff. Now these agents learn from previous experiences and to the surprise of no one, for the first few million rounds, we start out with pandemonium. Everyone just running around aimlessly. Then over time, the hiders learned to lock out the seekers by blocking the doors off with these boxes and and started winning consistently. I think the coolest part about this is that the map was deliberately designed by the OpenAI scientists in a way that the hiders can only succeed through collaboration. But then something happened. Did you notice this pointy doorstop shaped object? Are you thinking what I am thinking? Well, probably. And not only that, but later the AI also discovered that it can be pushed near a wall and be used as a ramp. And ta da gotem. Then it was up to the hiders again to invent something new. So did they do that? Can this crazy strategy be defeated? Well, check this out. These resourceful little critters learned that since there is a little time at the start of the game when the seekers are frozen, apparently during this time they cannot see them. So why not just sneak out and steal the ramp and lock it away from them? Absolutely incredible. Look at those happy eyes as they are carrying that ramp. But today is not 2019, it is 2021. So I wonder what scientists at the other amazing AI lab, DeepMind have been up to. Can this paper be topped? Well, believe it or not, they have managed to create something that is perhaps even crazier than this. This new paper proposes that these AI agents look at the screen just like a human would and engage in open ended learning where the tasks are always changing. What does this mean? Well, it means that these agents are not preparing for an exam, they are preparing for life. Hopefully they learn more general concepts and as a result maybe excel at a variety of different tasks. Even better, These scientists at DeepMind claim that their AI agents not only excel at a variety of tasks, but they excel at new ones they have never seen before. Those are big words. So let's see the results. The red agent here is the hider and the blue is the seeker. They both understand their roles. The red agent is running and the blue is seeking. Look Its viewing direction is shown with this lightsaber looking line pointing at the red agent. No wonder it is running away. And look, it manages to get some distance from the seeker and finds a new, previously unexplored part of the map and hides there. Excellent. And you would think that the Star wars references end here. No, not even close. Look, in a more advanced variant of the game, this green seeker lost the two other hiders. And what does he do? Ah, yes, of course. Grabs his lightsaber and takes the high ground. Then it spots the red agent and starts chasing it. And all this without ever having played this game before. That is excellent. In this cooperative game, the agents are asked to get as close to the purple pyramid as they can. Of course, to achieve that, they need to build a ramp, which they successfully realize. Excellent. But it gets better. Now, note that we did not say that the task is to build a ramp. The task is to get as close to the purple pyramid as we can. Does that mean yes? Yes, it does. Great job bending the rules, little AI in this game, the agent is asked to stop the purple ball from touching the red floor. At first it tries its best to block the rolling of the ball with its body. And then, look, it realizes that it is much better to just push it against the wall. And it gets even better. Look, it learned that best is to just chuck the ball behind this slab. It is completely right. This needs no further energy expenditure and the ball never touches the red floor again. Great. And finally, in this King of the Hill game, the goal is to take the white floor and get the other agent out of there. As they are playing this game for the first time, they have no idea where the white floor is. As soon as the blue agent finds it, it stays there. So far, so good. But this is not a cooperative game. We have an opponent here. Look. Boom. A quite potent opponent indeed, who can take the blue agent out. And it understands that it has to camp in there and defend the region again. Awesome. So the goal here is not to be an expert in one game, but to be a journeyman in many games. And these agents are working really well at a variety of games without ever having played them. So, in summary, OpenAI's agent, expert in a narrower domain, DeepMind's agent journeyman in a broader domain, two different kinds of intelligence, both doing amazing things. Loving it. What a time to be Alive. Scientists at DeepMind have knocked it out of the park with this one. They have also published AlphaFold this year, a huge breakthrough that makes an AI predict protein structures Now I saw some of you asking why we didn't cover it. Is it not an important work? Well, quite the opposite. I am spellbound by it and I think that paper is a great gift to humanity. However, I try my best to educate myself on this topic. However, I don't feel that I am qualified to speak about it. Not yet anyway. So I think it is best to let the experts who know more about this take the stage. This is of course bad for views, but no matter. We are not maximizing views here, we are maximizing meaning this episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI, check out Lambda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000 and V100 instances and hold on to your papers because Lambda GPU Cloud can cost less than half of aws and and Azure plus they are the only cloud service with 48 gigabyte RTX 8000. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
