A single AI controller learns to walk, dance, and do gymnastics across wildly different simulated body shapes without retraining. It adapts even when bodies change mid-motion, but struggles when shapes stray too far from what it trained on.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. This glorious paper from about seven years ago was about teaching digital creatures to walk. The numbers here showcase the process of learning over time, and it is clear that the later generations did much better than the earlier ones. These control algorithms are not only able to teach these creatures to walk, but they are quite robust against perturbations as well. Or more simply put, we can engage in one of the favorite pastimes of a computer graphics researcher, which is of course throwing boxes at a character and seeing how well it can take it. This one has done really well. Well, kind of. Now we just noted that this is a computer graphics paper, and an amazing one at that. But it does not yet use these incredible new machine learning techniques that just keep getting better year by year. You see, these agents could learn to inhabit a given body. But I am wondering what would happen if we would suddenly change their bodies on the fly? Could previous methods handle it? Unfortunately, not really. And I think it is fair to say that an intelligent agent should have the ability to adapt when something changes. Therefore, our next question is, how far have we come in these seven years? Can these new machine learning methods help us create a more general agent that could control not just one body, but a variety of different bodies? Let's have a look at today's paper. And with that, let the fun begin. This initial agent was blessed with reasonable body proportions. But of course we can't just leave it like that. Yes, much better. And look, all of these combinations can still work properly. And all of them use the same one reinforcement learning algorithm. But do not think for a second that this is where the fun ends. No, no, no. Now hold on to your papers and let's engage in these horrific asymmetric changes. There is no way that the same algorithm could be given this body and still be able to walk. Goodness, look at that. It is indeed still able to walk. If you have been holding on to your papers. Good. Now squeeze that paper because after adjusting the height, it could still not only walk, but even dance. But it goes further. Do you remember the crazy asymmetric experiment for the legs? Let's do something like that, with thickness. And as a result they can still not only walk, but even perform gymnastic moves. Woohoo. Now it's great that one algorithm can adapt to all of these body shapes, but it would be reasonable to ask how much do we have to wait for it to adapt? Have a look here. Are you seeing what I am seeing? We can make changes to the body on the fly and the AI adapts to it immediately. No retraining or parameter tuning is required. And that is the point where I fell off the chair when I read this paper. What a time to be alive. And now scholars bring in the boxes. It can also inhabit dogs and fish and we can also have some fun with them as we grab a controller and control them in real time. The technique is also very efficient as it requires very little memory and computation. Not to mention that we only have to train one controller for for many body types instead of always retraining after each change to the body. However, of course this algorithm isn't perfect. One of its key limitations is that it will not do well if the body shapes we are producing strays too far away from the ones contained in the training set. But let's leave some space for the next follow up paper too. And just one more thing that didn't quite fit into this story. Every now and then I get these heartwarming messages from you fellow scholars noting that you've been watching the series for a while and decided to turn your lives around and go back to study more and improve. Good work Mr. Monad. That is absolutely amazing and reading these messages are a true delight to me. Please keep them coming. What you see here is an instrumentation for a previous paper that we covered in this series which was made by Weights and Biases. I think organizing these experiments really showcases the usability of their system. Weights and Biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money and it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub and more. And the best part is that if you have an open source, academic or personal project, you can use their tools for free. It really is as good as it gets. Make sure to visit them through wnb.com papers or click the link in the video description to start tracking your experiments in five minutes. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
