Researchers trained a legged robot in simulation using only touch and position sensors, then transferred the controller to the real world. The robot learned to climb hills, stairs, mud, and vegetation, and carried a 10 kg payload without seeing anything.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. What you see here is one of my favorite kind of paper where we train a robot in a software simulation and then we try to bring it into the real world and see if it can still navigate there. The simulation is the textbook and the real world is the exam, if you will. And it's a really hard one. So let's see if this one passes or fails. Now, the real world is messy and full of obstacles. So to even have a fighting chance in training a robot to deal with that, the simulation better has those properties too. So let's add some hills, steps and stairs, and let's see if it can overcome these. Well, not at first. As you see, the agent is very clumsy and can barely walk through a simple terrain. But as time passes, it grows to be a little more confident, and with that, the terrain also progressively becomes more and more difficult in order to maximize learning. That is a great life lesson right there. Now, while we look at these terrain experiments, you can start holding on to your papers. Because this robot knows absolutely nothing about the outside world. It has no traditional cameras, no radar, no lidar, no depth sensors, no, no, no, none of that. Only proprioceptive sensors are allowed, which means that the only thing that the robot senses is its own internal state. And that's it. Whoa. For instance, it knows about its orientation and twist of the base unit, a little joint information like positions and velocities, and really not that much more. All of which is proprioceptive information. Yep, that's it. And bravo. It is doing quite well in the simulation. However, reality is never quite the same as the simulations. So I wonder if what was learned here can be used there. Let's see. Wow, look at how well it traverses through this rocky mountain stream. And not even this nightmary. Slowly descent gives it too much trouble. It works even if it cannot get a proper foothold. And it is slipping all the time. And it also learned to engage in this adorable jumpy behavior when stuck in vegetation. And it learned all this by itself. Absolute witchcraft. When looking at this table, we now understand that it still has reasonable speeds through moss, and why it is slower in vegetation than in mud. Really cool. If you have been holding onto your paper so far, now squeeze that paper. Because if all that wasn't hard enough, let's add an additional 10 kilogram or 22 pound payload and see if it can shoulder it. And let's be honest, this should not happen. Wow, look at that. It can not only shoulder it but it also adjusts its movement to the changes to its own internal physics. To accomplish this, it uses a concept that is called learning by cheating or a teacher student architecture, when the agent that learned in the simulator will be the teacher. Now, after we escaped the simulator into the real world, we cannot lean on the simulator anymore to try to see what works. So then what do we do? Well, here the teacher takes the place of the simulator. It distills its knowledge down to train the student further. If both of them do their jobs really well, the student will successfully pass the exam with flying colors. As you see, it is exactly the case here. This is an outrageously good paper. What a time to be alive. This episode has been supported by Weights and Biases. In this post they show you how to use their reports to explain how your model works, show plots of how model versions improved, discuss bugs, and demonstrate progress towards milestones. If you work with learning algorithms on a regular basis, make sure to check out weights and biases. Their system is designed to help you organize your experiments and it is so good it could shave off weeks or even months of work from your projects and is completely free for all individuals, academics and open source projects. This really is as good as it gets and it is hardly a surprise that they are now used by over 200 companies and research institutions. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
