Researchers at Seoul National University trained a simulated dragon to perform aerobatic maneuvers using reinforcement learning combined with self-regulated learning. The agent breaks hard tasks into smaller subgoals like orientation and position, then cycles through generating alternatives, evaluating them, and learning from outcomes.
Dear fellow scholars, this is two minute papers with zsolnai. Scientists at the Seoul National University in South Korea wrote a great paper on teaching an imaginary dragon all kinds of really cool aerobatic maneuvers like sharp turning, rapid winding, rolling, soaring and diving. This is all done by a reinforcement learning variant, where the problem formulation is that the AI has to continuously choose a character's actions to maximize a reward. Here, this reward function is related to a trajectory which we can draw in advance. These are the lines that the dragon seems to follow quite well. However, what you see here is the finished product. Curious to see how the dragon falters as it learns to maneuver properly. Well, we are in luck. Buckle up. You see the ideal trajectory here with black and Initially, the dragon was too clumsy to navigate in a way that even resembles this path. Then later, it learned to start the first turn properly. But as you see here, it was unable to avoid the obstacle and likely needs to fly to the emergency room. But it would probably miss that building too. Of course, after more learning, it was able to finish the first loop, but was still too inaccurate to perform the second. And finally, at last, it became adept at performing this difficult maneuver. Applause. One of the main difficulties of this problem is the fact that the dragon is always in motion and has a lot of momentum. And anything we do always has an effect later. And we not only have to find one good action, but whole sequences of actions that will lead us to victory. This is quite difficult. So how do we do that to accomplish this? This work not only uses a reinforcement learning variant, but also adds something called self regulated learning to it, where we don't present the AI with a fixed curriculum, but we put the learner in charge of its own learning. This also means that it is able to take a big complex goal and subdivide it into new smaller goals. In this case, the big goal is following the trajectory with some more additional constraints, which by itself turned out to be too difficult to learn with these traditional techniques. Instead, the agent realizes that if it tracks its own progress on a set of separate but smaller sub goals, such as tracking its own orientation, positions and rotations against the desired target states separately, it can finally learn to perform these amazing stunts. That sounds great, but how is this done exactly? This is done through a series of three steps, where step one is generation, where the learner creates a few alternative solutions for itself and proceeds to the second step, evaluation, where it has to judge these individual alternatives and find the best ones, and third, learning, which means looking back and recording whether these judgments indeed put the learner in a better position. By iterating these three steps, this virtual dragon learned to fly properly. Isn't this amazing? I mentioned earlier that this kind of problem formulation is intractable without self regulated learning, and you can see here how a previous work fares on following these trajectories. There is indeed a world of a difference between the two. So there you go. In case you enter a virtual world where you need to train your own dragon, you'll know what to do. But just in case, also read the paper in the video description. If you enjoyed this episode and you wish to watch our other videos in early access or get your name immortalized in the video description, please consider supporting us on patreon through patreon.com Twominutepapers the link is available in the video description and this way we can make better videos for you. We also support cryptocurrencies. The addresses are also available in the video description. Thanks for watching and for your generous support and I'll see you next time.
