Meet Your Virtual AI Stuntman! 💪🤖

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to look at a paper from three years ago, and not any kind of paper, but my kind of paper, which is in the intersection of machine learning, computer graphics and physics simulations. This work zooms in on reproducing reference motions, but with a twist and adds lots of additional features. So what does all this mean? You see, we are given this virtual character, a reference motion that we wish to teach it. And here additionally, we are given a task that needs to be done. So when the reference motion is specified, we place our AI into a physics simulation where it tries to reproduce these motions. That is a good thing because if it would try to learn to run by itself alone, it would look something like this. And if we ask it to mimic the reference motion. Oh yes, much better. Now that we have built up confidence in this technique, let's think bigger and perform a backflip. Uh oh, well that didn't quite work. Why is that? We just established that we can give it a reference motion and it can learn it by itself. Well, this chap failed to learn a backflip because it explored many motions during training, most of which resulted in failure. So it didn't find a good solution and settled for a mediocre solution instead. A proposed technique by the name Reference State Initialization RSI, in short, remedies this issue by letting the agent explore better during the training phase. Got it. So we add this RSI and now all is well, right? Let's see. Ouch. Not so much. It appears to fall on the ground and tries to continue the motion from there. A for effort, little AI. But unfortunately that's not what we are looking for. So what is the issue here? The issue is that the agent has hit the ground and after that it still tries to score some additional points by continuing to mimic the reference motion. Again, A for effort, but this should not give the agent additional scores. This method we just described is called early termination. Let's try it. Now we add the early termination and RSI together and let's see if this will do the trick. And yes, finally, with these two additions, it can now perform that sweet, sweet backflip rolls and much, much more with flying colors. So now the agent has the basics down and can even perform explosive dynamic motions as well. So it is time. Now hold on to your papers as now comes the coolest part. We can perform different kinds of retargeting as well. What is that? Well, one kind is retargeting the environment. This Means that we can teach the AI a landing motion in an idealized case and then ask it to perform the same, but now off of a tall ledge. Or we can teach it to run and then drop it into a computer game level and see if it performs well there. And it really does. Amazing. This part is very important because in any reasonable industry use, these characters have to perform in a variety of environments that are different from the training. Environment 2 is retargeting not the environment, but the body type. We can have different types of characters learn the same motions. This is pretty nice for the Atlas robot, which has a drastically different weight distribution. And you can also see that the technique is robust against perturbations. Yes, this means one of the favorite pastimes of a computer graphics researcher, which is throwing boxes at virtual characters and seeing how well it can take it. Might as well make sure of the fact that in a simulated world we make up all the rules. This one is doing really well. Oh, note that the Atlas robot is indeed different than the previous model and these motions can be retargeted to it. However, this is also a humanoid. Can we ask for non humanoids as well perhaps? Oh yes. This technique supports retargeting to T. Rexes, dragons, lions, you name can even get used to the gravity of different virtual planets that we dream up. Bravo. So the value proposition of this paper is completely out of this world. Reference state initialization, early termination, retargeting to different body types, environments. Oh my. To have digital applications like computer games use this would already be amazing. And just imagine what we could do if we could deploy these to real world robots. And don't forget, these research works just keep on improving every year. The first law of papers says that research is a process. Do not look at where we are, look at where we will be. Two more papers down the line. Now, fortunately we can do that right now. Why is that? It is because this paper is from 2018, which means that follow up papers already exist. What's more, we even discussed one that teaches these agents to not only reproduce these reference motions, but but to do those with style. And style there meant that the agent is allowed to make creative deviations from the reference motion, thus developing its own way of doing it. An amazing improvement. And I wonder what researchers will come up with in the near future. If you have some ideas, let me know in the comments below. What a time to be alive.

This episode has been supported by Lamda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI CH. Check out Lambda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000 and V100 instances and hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Plus they are the only cloud service with 48 gigabyte RTX 8000. Join researchers at organizations like Apple, MIT and Caltech Emergency in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their long standing support and for helping us make.

Better videos for you. Thanks for watching and for your generous support and I'll see you next time.