Károly Zsolnai-Fehér shows AI that designs its own body shape and movement at once. An ant redesigns its legs for better walking, another grows springs for skipping, and one learns to climb stairs asymmetrically. The system minimizes material while maximizing what the creature can do.
Dear fellow scholars, this is two minute papers with Reinforcement learning is a class of learning algorithms that chooses a set of actions in an environment to maximize a score. Typical use cases of this include writing an AI to master video games, or avoiding obstacles with a drone, and many more cool applications. What ties most of these ideas together is that whenever we talk about reinforcement learning, we typically mean teaching an agent how to navigate in an environment. A few years ago, a really fun online app surfaced that used a genetic algorithm to evolve the morphology of a simple 2D car with the goal of having it roll as far away from a starting point as possible. It used a genetic algorithm that is quite primitive compared to modern machine learning techniques in and yet it still does well on this. So how about tasking a proper reinforcement learner to optimize the body of the agent? What's more, what if we could jointly learn both the body and the navigation at the same time? Okay, so what does this mean in practice? Let's have a look at an example. Here we have an ant that is supported by four legs, each consisting of three parts that are controlled by two motor joints. With the classical problem formulation, we can teach this ant to use these joints to learn to walk. But in the new formulation, not only the movement, but the body morphology is also subject to change. As a result, this ant learned that the body can also be carried by longer, thinner legs and adjusted itself accordingly. As a plus, it also learned how to walk with these new legs, and this way it was able to outclass the original agent. In this other example, the agent learns to more efficiently navigate a flat terrain by redesigning its legs that are now reminiscent of small springs and uses them to skip its way forward. Of course, if we change the terrain, the design of an effective agent also changes accordingly. And the super interesting part here is that it came up with an asymmetric design that is able to climb stairs and travel uphill efficiently. Loving it. We can also task this technique to minimize the amount of building materials used to solve a task. And subsequently it builds an adorable little agent with tiny legs that is still able to efficiently traverse this flat terrain. This principle can also be applied to the more difficult version of this terrain, which results in a lean insect like solution that can still finish this level that uses about 75% less materials than the original solution. And again, remember that not only the design, but the movement is learned here at the same time. While we look at these really fun bloopers, I'd like to let you know that we have an opening at our institute at the Vienna University of Technology for one postdoctoral researcher. The link is available in the video description. Read it carefully to make sure you qualify. And if you apply through the specified email address, make sure to mention two minute papers in your message. This is an excellent opportunity to read and write amazing papers and work with some of the sweetest people. This is not standard practice in all countries, so I will note that you can check the salary right in the call. It is a well paid position in my opinion, and you get to live in Vienna. Also, your salary is paid not 12, but 14 times a year. That's Austria for you. It doesn't get any better than that. Deadline is end of January. Happy holidays to all of you. Thanks for watching and for your generous support and I'll see you early January.
