Video (2 Minute Papers) by Dr. Zsolnai summarizing a DeepMind paper that trains agents to learn on-the-job from human demonstrations without pre-collected data. The presenter outlines four experimental phases—pandemonium, following, memorization, independence—shows the agent eventually acting independently (including switching neural activity), and reports tests of generalization (obstacles, larger worlds). The clip includes a sponsor read for Lambda GPU Cloud.
Chapters
Dear fellow scholars, this is 2 Minute Papers with Dr. Zsolnai. Today we will see how and what DeepMind's AI is able to learn from a human, but with a twist. And the twist is that we are going to remove the human teacher from the game and see what happens. Is it just memorizing what the human does? Or is this AI capable of independent thought? Their previous technique did something like that. Like guiding us to the bathroom, start a band together, or even find out the limitations of the physics within this virtual world. However, there is a big difference. This previous technique had time to study, this new one, not so much. This new one has to do the learning on the job. Oh yes, in this project we will see an AI that has no pre collected human data. It really has to learn everything on the job. So can it? Well, let's see together. Phase one, Pandemonium. Here the AI says well, here we do things. I am not so sure. Occasionally it gets a point. But now look, uh oh, the red teacher is gone. And oh boy, it gets very confused. It does not have much of an idea what is going on. But a bit of learning happens. And later, phase two following. It is still not sure what is going on, but when it follows this red chapter, it realized that it is getting a much higher score. Before it only got two. And now look at it. It is learning something new here. And look, he is gone again. But it knows what to do. Well, kind of. It still probably wonders why its score is decreasing. So a bit later. Phase three, memorization. First the usual dance with the demonstrator. But now when he is gone, it knows exactly what to do and just keeps improving its score. And then comes the crown jewel. Phase four, independence. No demonstrator anywhere to be seen. This is the final exam. It has to be able to independently solve the problem. But here comes the twist. For the previous phase we said memorization, and for this we say independence. Why is this different? Well look, do you see the difference? Hold on to your papers because we have switched up the colors. So the previous strategy is suddenly useless. Oh yes. If it walks the same path as before, it will not get a good score. And initially that's exactly what it's trying to do. But over time it is now able to learn independently and indeed find the correct order by itself. And what I absolutely loved here is. Look, over time the charts verify that indeed, as soon as we take away the teacher, it starts using different neurons. Right after becoming an independent entity. I love it. What an incredible chart. And all this is excellent news. So if it really has an intelligence of sorts, it has to be able to deal with previously unseen conditions and problems. That sounds super fun. Let's explore that some more. For instance, let's give it a horizontal obstacle. Good. Not a problem. Vertical.
Also.
Fine. Now let's make the world larger and it is still doing well. Awesome. So I absolutely love this paper. DeepMind demonstrated that they can build an AI that learns on the job, one that is even capable of independent thought, and even better, one that can deal with unforeseen situations too. What a time to be alive. So what do you think? What else could this be useful for? What do you expect to happen? A couple more papers down the line? Please let me know in the comments below. I'd love to hear your thoughts.
This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI, check out Lambda GPU Cloud. Get this they've recently launched an Nvidia RTX A6000 with 48 GB of memory. And hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Our thanks to Lamda for their long standing support and for helping us make.
Better videos for you. Thanks for watching and for your generous support and I'll see you next time.
