DeepMind trained AI agents to play football in simulation, then deployed them on real robots with 20 joints each. The agents learned to work as a team, anticipate plays, control the ball, and even developed a gait that reduces knee injuries.
Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Doctor papers? Today you are going to see an insane paper. One of the most fun papers I've read in a while. Earlier, scientists at DeepMind were looking to teach a bunch of AI agents to play football soccer. Unfortunately, they started out like this. Not very impressive. But then they trained them for five years.
What?
Five years? Well, their training took what felt like five years for them, but only three days for us. Because these are not real robots. And thus on a fast computer, they can be simulated faster than real time. So what happened in that time? This happened. Whoa. Now they can not only play amazingly well, but they can even think a few steps ahead. And. And they are even anticipating the behavior of their teammates and position themselves accordingly. The saves they are capable of are also outstanding. And as we don't really have a referee around, they also beat up each other. And when I say that, I really mean it. I mean it's bad. Real bad. Also, the only thing they like more than throwing each other is. Is throwing tantrums. And it gets even worse. This lovely chap almost seems like someone who is faking an injury. But then look, a miracle happened. And Today, scientists at DeepMind brought this work to the next level. How? In a different paper, scientists at OpenAI simulated a robot hand in a video game world. Then the things it learned in this simulation could be brought into life by deploying it into the real world. So, train in a simulation and then play in the real world? Yes, that is fantastic. But why am I saying this? Oh wait. Is it possible that the same happened here too? Real robots playing football? Yes, that is exactly what happened. So with that, let the insanity begin. The new environment contains 2 real robots with 20 actuated joints for each. So. So what did they learn first? Well, first they learned to get up and also score goals. All this in a simulated robot body. That is very impressive already. But wait, because that's just the start. Now it is time for random perturbations. What does that mean? It means that forces are randomly exerted on the AI to make sure it can recover from an impact. And are you thinking what I am thinking? Oh, yes. Now let's go to the real robots and let's test that impact. Oh, that is not very nice. But then. Wow, look at that. What it had learned in a simulation now really applies to a real life robot. It can get hit and is able to get up over and over again. My goodness. But there is a problem. What is the problem? Well, their knees get injured all the time. So what did scientists at DeepMind do with that? Well, hold on to your papers, fellow scholars. Because they taught the robot to get up and move in a way that is not too hard on the knees. Yes, that's right. And thus they didn't suffer that many knee injuries. What a time to be alive. So with that, finally they can now run and score goals. But wait, playing a match and winning takes more than just running and kicking. They need to have a variety of other skills and combine them in order to win. So did they? Oh yes, yes they did. They can now control the ball, turn with it, block the ball with their bodies, shoot accurately, even in the case of a moving ball, which is even harder. And that is the miracle of research papers. Such incredible progress can happen. Just one more paper down the line and Yes, I know, I know you are asking the question, ok, did they beat up each other here too, like in the previous paper? And the answer is no. You see, they are penalized for getting too close to each other. So luckily they are. That did not happen. But this kind of improvement in just one paper. I am wondering what these little AIs will be capable of two more papers down the line. Well, what do you think? Let me know in the comments below.
Weights and Biases provides tools to track your experiments in your deep learning projects. What you see here is their Artifacts feature which speeds up the most common machine learning steps like uploading raw data, splitting it into training, validation and test sets, and of course the best part, starting to train a neural network. It is used by many prestigious labs including OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you.
Thanks for watching and for your generous support and I'll see you next time.
