DeepMind trained AI soccer players by first teaching them to move like humans, then letting them play millions of simulated games in days. They learned to pass, anticipate teammates, and dive to block shots without being explicitly programmed to do any of it.
Questions this video answers
January 22, 2023.
Three days in real life.
Fifty days (simulated longer runs were run for 50 days of wall-clock time).
$1.10 per hour for 100 instances on Lambda GPU Cloud versus $4.10 per hour with AWS (noted as 73% savings).
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to see how DeepMind's incredible AI teaches these virtual characters to move, play football, soccer, and so much more. Now when reading this paper, I expected that they would learn the basics, but what happened afterwards, I did not expect at all. They also learned some absolutely amazing tricks and even more surprising things. And yes, we are going to look at all of them. But first, the humble beginnings. Now this is a physics based game. So to move, these AI agents need to use their joints and produce just the right kind of forces and torque on them to create sophisticated movements and amazing plays. So let's see. Oof. Unfortunately we are seeing none of that here. Why? Well, of course, because they haven't had enough time to learn yet. So are these just the humble beginnings or is this it? Is this the limit of their AI? Well, not quite, but first they need to learn to move. How? Well, scientists at DeepMind say by imitating motion capture data. In other words, this tries to copy how real humans move. This is not bad. At least it can now perform some basic movements. So that's great. However, that is not nearly enough. Look, it cannot control the ball yet. Not even close. But once again, let's come back five days later. And now can it pull it off? Now hold on to your papers and oh yes, yes it can. And with flying colors it can now follow a moving target, dribble, kick and so much more. Now get this. We will put these agents into a training camp for no less than five years and see how they do. We started out like this and now. Holy mother of papers. These little AIs can play. And they can do so much more than just moving around. Look, they can do these little tricks where the ball bounces back from an invisible wall. And not only that, but this player tries to pass to the other AI this way. And will they sc. Oh yes, fantastic. I have to say these are now pretty good players. Good job little AI. However, wait a second. What about the five years part? Did scientists at DeepMind have to wait for five years on this paper and hope that something good comes out of it? Well, not quite. You see, in real life, one second passes exactly in one second. That is not new information to anyone. However, if we have a quick computer in a simulation, 1 second in the game can be simulated in a matter of milliseconds in real life, thereby speeding up time itself. Not our time, of course, but the time that the AI lives in. So this five year training camp only took three Days in real life, that's not that much. Hmm. Actually, are you thinking what I am thinking? Yes. If it's so quick, let's run it for even longer, see what happens. They ran it for 50 days, and let's see the level of plays these can now do. And when I saw these results, I almost fell off the chair. They have become even better. Let's see what they have learned now. They can not only think a few steps ahead, but they are even anticipating the behavior of their teammates and position themselves accordingly. They also learn to use body to body contact to their advantage. High kicks are also performed to catch the defenders off guard. And I absolutely loved how they developed these quick turns to get away with the ball. This is incredible. I love it. And they can even play their way out of really difficult situations. Look at this one. Oh boy. The defending red player would have to clear this ball without scoring an own goal. So it would have to kick the ball in this direction. And the margin of error is razor thin so far. Good thinking. But can it pull this off? Wow. Good job, Little AI. The saves they are capable of are also outstanding. And as we don't really have a referee around, these AIs haven't exactly been encouraged to be gentle with each other. So this can happen too. Ouch. And here is one of my favorite moments. After a little altercation, the red player falls. And it almost seems like throwing a little tantrum. This is super fun. Especially that he was the one who started it. I am sure some of you fellow scholars will point out that this is not unlike some real football players. And of course, this Little AI appears so injured it cannot get up. But later, when it is really needed to be there to defend. Look, a miracle happened. And of course, this guy just cannot help it. This is one of the most fun papers I've read in a while. So cool. However, we are experienced fellow scholars over here, so we would like to know so much more than what we've seen here. So let's look under the hood and see some more about what they have really learned. And these are the curves that I am looking for. Yes, and I see that their ELO ratings increase over time. This means that as they learn more, they get better. That is very reassuring. I still haven't seen what my heart really desires yet. So let's continue. Now we're talking more detailed data. This is so cool to see. Look, over time, they can get up quicker. On average, they use their joints better to run faster and. Yes, there we go. Division of labor. Yes, yes, yes. Wow. That is what my heart desires. This is where we struck gold. This is the graph that shows that these two players learned not only to move properly, but how to work together as a team. This is also reflected in their passing frequency, which is increasing over time. But the division of labor, that is where the real magic happens. You see, this is one of those elusive qualities that is really difficult to learn properly, often even for humans. And it really shows here too how well, look, this is one of the very few cases in the paper that does not show steady linear growth at times, it even worsens over time. After one day of training, the agents know just enough to score a goal alone and thus they become quite selfish. We all know that guy, don't we? But over time they realize that as a team, they can perform even better. Wow. What an incredible piece of work. And this is the moment where I got goosebumps when reading this paper. I love moments like this and I am so happy to share it with you. And remember, this is where we started and this is where we ended up. And all this through the power of modern learning algorithms. And yes, this is two Minute Papers, the corner of the Internet where we fellow scholars look at a blue curve and make happy noises. So good. If you enjoyed this, make sure to subscribe and perhaps even even hit the bell icon if you wish to see more amazing papers like this. So what do you think? What would you use this for? Let me know in the comments below. If you're looking for inexpensive cloud GPUs for AI, Lamda now offers the best prices in the world for GPU cloud compute. No commitments or negotiation required. Just sign up and launch an instance and hold on to your papers. Because with Lambda GPU Cloud you can get on demand a 100 instances for $1.10 per hour versus $4.10 per hour with AWS. That's 73% savings. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs. Com papers to sign up for one of their amazing GPU instances today. Thanks for watching and for your generous support and I'll see you next time.
