Terrain Traversal with Reinforcement Learning | Two Minute Papers #26

Dear fellow scholars, this is two minute papers with Reinforcement learning is a technique that can learn how to play computer games or any kind of activity that requires a sequence of actions. We are not interested in figuring out what we see on an image, because the answer is one thing. We are always interested in a sequence of actions. The input for reinforcement learning is a state that describes where we are and how the world looks around us. And the algorithm outputs the optimal next action to take. In this case, we would like a digital dog to run and leap over and onto obstacles. By choosing the optimal next action, it is quite difficult as there are a lot of body parts to control in harmony. The algorithm has to be able to decide how to control leg forces, spine curvature, angles for the shoulder, elbow, hip and knees. And what is really amazing is that if it has learned everything properly, it will come up with exactly the same movements as we'd expect animals to do in real life. So this is how reinforcement learning works. If you do well, you get a reward, and if you don't, you get some kind of punishment. These rewards and punishments are usually encoded in the score. If your score is increasing, you know you've done something right and you try to self reflect and analyze the last few actions to find out which of them were responsible for this positive change. The score would be, for instance, how far the dog could run on the map without falling. And at the same time, it also makes sense to minimize the amount of effort to make it happen. So reinforcement learning, in a nutshell, it is very similar to how a real world animal, or even a human would learn. If you are not doing well, try something new. And if you are succeeding, remember what you did that led to your success and keep doing that. In this technique, dogs were used to demonstrate the concept, but it's worth noting that it also works with bipeds. Reinforcement learning is typically used in many control situations that are extremely difficult to solve otherwise, like controlling a quadrocopter properly. It's quite delightful to see such a cool work, especially given that there are not so many uses of reinforcement learning in computer graphics yet. I wonder why that is. Is it that not so many graphical tasks require a sequence of actions? Or maybe we just need to shift our mindset and get used to the idea of formalizing problems in a different way so so we can use such powerful techniques to solve them. It is definitely worth the effort. Thanks for watching and for your generous support and I'll see you next time.