DeepMind trained AI agents using human feedback and demonstrations, then tested them on 150 tasks like building towers and cleaning rooms. The agents matched or surpassed human performance, with particular strength in following instructions.
Dear fellow scholars, this is Two Minute.
Papers with Dr. Zsolnai. Today we are going to have a peek into DeepMind's journey into creating an AI that behaves like a human. How? Well, by dropping it into a virtual playhouse, asking humans to give it tasks, judging it rather harshly, and hoping that it learns something from it. But not so fast. First we have to start with an AI agent, a little assistant that already knows a few things. Just dropping a randomly acting agent into this game would do no good. So how do we do that? Well, scientists at DeepMind say that first we should ask it to look at humans interacting with previous AI agents and have it copy them. Let's call this behavioral cloning. So how good is behavioral cloning? Well, the answer is not good at all.
Let's have a look together.
This agent has looked at humans building towers from books. And now it's your turn, little ei. And it does not too well, I am afraid. Oh my. That is not good. Look, it has learned a bit from humans. It is trying so hard, but unfortunately it has no idea about how physics works and does not have the finance required to build this tower. So is that it? We can't even write an AI that clones what humans can already perform in virtual worlds. Well, do not despair, because here comes the second best part of the paper. In this work, this agent is used just as a starting point. And now we can ask it to do a bunch of tasks and have humans give it a score. For instance, here you see it trying to arrange these fruits in a row. And whenever it progresses with this task, it gets a nice spike in the score. And whenever it does not do well, we can tell it that this is not the way. Alright, but what does this process do for us? Well, it creates an agent that can learn not the behavior of humans, but the reward model itself. What does that mean? Well, it tries to learn what the human teacher would think about what the AI is doing right now. And is that useful? Well, have a look at it yourself. Here is the agent after it started to learn the reward model. And let's build that tower again. First try is not too bad, but the second one. Now look at that. That is a proper tower of books right there. Very proud of you, little AI. So how well did it do? Here is the score for a human player. And here it is for the not too great first try with the behavioral cloning AI. And now hold on to your papers. And look at that. This is the new agent. It does almost as well as a real human does.
Wow.
This is an amazing chart that shows that these agents can learn tasks that require walking around, understanding, a bit of physics, and a bit of fineness as well. I love it. And it can learn not just building towers, but. But it also learned to build triangles of objects. And of course, everyone's favorite. What good would an AI be if we could not teach it to clean our room? So can it do that? Oh yes. And with flying colors. Loving it. But wait, I said that this was the second best part of the paper. What is the best part? It is this. Remember, first we start out from the behavioral cloning agent and then start learning the reward model. This does nearly as well as a human does. So that is absolutely amazing. We can teach an AI to behave like a human does in these tasks. That is incredible. But that's not it. Get this. If we repeat this process and let it learn for a bit longer, we get this. Whoa. Are you seeing what I am seeing? Yes, that's right. If we let it train for a bit longer, this happens. It can even surpass humans building that tower. An AI that learns from humans and over time even surpasses humans. And wait, it gets better. It surpasses humans not just on this one particular task, but on a set of over 150 standardized tasks. Here they are. Generally it greatly outperforms in instruction following type tasks. While it is a bit worse than humans at answering questions, but it still does really well. Wow. Perhaps this is one more step towards general intelligence. And also, welcome to Two Minute Papers and land of the fellow scholars who look at bar charts and make happy noises. Yeah, what a time to be alive.
If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud compute. No commitments or negotiation required. Just sign up and launch an instance. And hold on to your papers because with Lambda GPU Cloud you can get on demand a 100 instances for $1.10 per hour versus $4.10 per hour with AWS. That's 73% savings. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today.
Thanks for watching and for your generous support and I'll see you next time.
