Dr. Zsolnai breaks down a SIGGRAPH paper that uses imitation learning, GANs, and reinforcement learning to animate realistic quadrupeds. The system fixes motion capture problems like foot sliding and missing transitions, then proves it handles perturbations and pathfinding without breaking.
Dear fellow scholars, this is two minute papers with Dr. Zsolnai. If we have an animation movie or a computer game with quadrupeds, and we are yearning for really high quality lifelike animations, motion capture is often the go to tool for the job. Motion capture means that we put an actor, in our case a dog, in the studio, and we ask it to perform sitting, trotting, pacing and jumping, record its motion and transfer it onto our virtual character. There are two key challenges with this approach. One, we have to try to weave together all of these motions because we cannot record all the possible transitions between sitting and pacing, jumping and trotting, and so on. We need some filler animations to make these transitions work. This was addressed by this neural network based technique here. The other one is trying to reduce these unnatural foot sliding motions. Both of these have been addressed by learning based algorithms in the previous works that you see here. Later, bipeds were also taught to maneuver through complex geometry and sit in not one kind of chair, but any chair, regardless of geometry. This already sounds like science fiction. So are we done or can these amazing techniques be further improved? Well, we are talking about research, so the answer is of course yes. Here you see a technique that reacts to its environment in a believable manner. It can accidentally step on the ball, stagger a little bit, and then flounder on this slippery surface and it doesn't fall and it can do much, much more. The goal is that we would be able to do all this without explicitly programming all of these behaviors by hand. But unfortunately there is a problem. If we write an agent that behaves according to physics, it will be difficult to control properly. And this is where this new technique shines. It gives us physically appealing motion and we can grab a controller and play with the character like in a video game. The first step we need to perform is called imitation learning. This means looking at real reference movement data and trying to reproduce it. This is going to be motion that looks great, is very natural. However, we are nowhere near done because we still don't have any control over this agent. Can we improve this somehow? Well, let's try something and see if it works. This paper proposes that in step number two, we try an architecture by the name generative adversarial network. Here we have a neural network that generates motion and the discriminator that looks at these motions and tries to tell what is real and what is fake. However, to accomplish this, we need lots of real and fake data that we then use to train the discriminator to Be able to tell which one is which. So how do we do that? Well, let's try to label the movement that came from the user controller inputs as fake and the reference movement data from before and as real. Remember that this makes sense as we concluded that the reference motion looked natural. If we do this over time, we will have a discriminator network that is able to look at a piece of animation data and tell whether it is real or fake. So after doing all this work, how does this perform? Does this work? Well, sort of, but it does not react well if we try to control the simulation, if we let it run undisturbed, it works beautifully. And now when we try to stop it with the controller, well, this needs some more work, doesn't it? So how do we adapt this architecture to the animation problem that we have here? And here comes one of the key ideas of the paper. In step number three, we can rewire this whole thing to originate from the controller and introduce a deep reinforcement learning based fine tuning stage. This was the amazing technique that DeepMind used to defeat Atari Breakout. So what good does all this for us? Well, hold on to your papers. Because it enables true user control while synthesizing motion that is very robust against tough, previously unseen scenarios. And if you have been watching this series for a while, you, you know what is coming, of course, throwing blocks at it and see how well it can take the punishment. As you see, the AI is taking it like a champ. We can also add pathfinding to the agent and of course, being computer graphics researchers, throw some blocks into the mix for good measure. It performs beautifully. This is so realistic. We can also add sensors to the agent to allow them to navigate in this virtual world in a realistic manner. Just a note on how remarkable this is. So this quadruped behaves according to physics, lets us control it with the controller, which is already somewhat of a contradiction. And it is robust against these perturbations at the same time. This is absolute witchcraft. And no doubt it has earned to be accepted to siggraph, which is perhaps the most prestigious research venue in computer graphics. Congratulations. What you see here is an instrumentation for a previous paper that we covered in this series, which was made by weights and biases. I think organizing these experiments really showcases the usability of their system. Weights and Biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money. And it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub and more. And the best part is that if you're an academic or have an open source project, you can use their tools for free. It really is as good as it gets. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to WeightsAndBiases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
