A Two Minute Papers video by Dr. Zsolnai showcasing DeepMind’s new agent, Ada, which rapidly learns to solve several simulated object-manipulation tasks that previously required long simulated training. The narrator describes level rules (often hidden from the agent), stringent constraints (e.g., a 22-second time limit and no intermediate rewards), and demonstrates how Ada discovers causal object interactions, adapts across rounds, and even develops cooperative and optimized behaviors within seconds.
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to torment DeepMind's new AI called Ada with tasks that are impossible. And then we will see if it is smart enough to solve them. Now you see, we talked about previous AI techniques that were able to learn over time. For instance, Nvidia's little knights were able to learn to fight and by themselves. But this took 10 years of training. Now, not 10 years in our lives. 10 years in their lives as they live inside a simulation which with a quick computer will only take a few days to simulate in a different paper. AI agents started out like this and over time they learned to play football and some really advanced techniques. And there was no referee, so they also learned to be not too kind to each other. Ouch. So how long did this take? Well, these folks also trained for years in simulation time that is. And now with DeepMind's new AI, this agent will hopefully be able to learn cool new things. And hopefully it will not take years. Yes, we are going to build a virtual playhouse and play a little game throughout the game. This little window to the world is what the AI sees only we see the whole level. So now, little AI, you have one job and that is to hold the black cube. But there is a problem. What is the problem? Well, of course the fact that there is no black cube. Not one in sight. But we have a secret. The secret is that if we touch the black pyramid with the yellow sphere, out comes a black cube. However. Psst. This is a secret. The AI does not know about this rule and has to find out by itself. And you know what? Let's make it even worse. If it touches the yellow sphere to the purple pyramid, both get destroyed, making this task impossible to finish. And you know what? Let's make it even worse. Give it a strict 22nd time limit. So good luck with that little AI. I can't wait to see this. This is going to be a lot of fun. So let's see, round one. It starts exploring, likely to find the black cube, picks up the yellow sphere, dashes away and oh boy, bad news, real bad. It proceeds to touch the purple pyramid. You know what that means, right? Oh yes. Both of them get destroyed. Little does the AI know the task is now impossible. It still tries to combine some objects together. Maybe something good happens. But we already know nothing good happens here. And then the time runs out. And now comes the interesting part. Did it learn anything from this? I am so excited. Let's start again. Hmm, it says let's not do what we did earlier. That's a good start. It takes the black pyramid. And there we go. Good job. We got the black cube. And now it is time for a victory dance. Fantastic. So is there any point in running round three? Oh yes. Yes there is. The goal is that we wish to see what it learned from the previous success. Did it do it just by chance or does it understand what just happened? Let's see. Oh boy. It is going straight for the correct answer. It really knows what just happened. It truly uncovered the rules of the game. And now it is busy optimizing its route to solve it even quicker. I love it. And when running a similar task for two of these ADA agents, they learn the rules of the game independently. But that is expected. However, what is not expected is. Look at that. Wow. They learned to throw to get this task done even quicker. And later on they learn to work together to be even more efficient. So now, yes, it is time to hold on to your papers, fellow scholars, because what you are seeing here is learning happening not in years, but in a matter of seconds. Learning is so quick. Here it is happening right before our eyes. I can't believe it. So if we can believe this, what do we do? Of course we make the task even more difficult. Remember, the previous task could only be solved by lifting things. So how about creating a level where lifting things make you lose immediately? That sounds fun, right? This new level can only be solved by pushing. Of course the AI does not know that. So in round one, it it starts out lifting with predictable results. This was not a success. So let's see what has it learned. Round two. Look. It starts pushing instead. And when pushing the two cubes together. Got em. Good job. That was super quick learning. And once again, the learning happened right before our eyes. So what do we do now? Of course we make it even harder. Let's add a bunch of unnecessary rules and a ton of objects to distract the AI from finding the yellow pyramid, which by the way, doesn't even exist yet. See if it can see through our little tricks. First some exploration happens and then transmutation also happens. A lot of it. Something touches something and some other something appears. This doesn't even seem to make any sense. Note once again that only we see the rules and the whole map. The AI does not know anything about the rules and is playing this game for the first time and only sees this tiny window to the world. And later. My goodness, look at that. With a stylish move, it chucks away the yellow box. That is in the way. Fabulous. Then it holds the purple box. So what happens then? Look at the rules. Yes, it makes the highly coveted yellow pyramid appear. Great. Then it goes straight for the goal. Learning is happening here too. And super quickly. And in a different cooperative level. Each player has to touch their corresponding sphere to get their pyramids. And when these new pyramids touch, we are finished. For the first try. They try to explore and eventually succeed. But do they really know how they succeeded? Do they know which actions were the one responsible for their success? Wow. For only the second try, they absolutely smashed it. That is just about the quickest and most effective solution that I can imagine. Holy mother of papers. This little AI is learning incredibly quickly. And I have to be honest, when reading the paper I was a little worried if it could learn at all. Why? Well, because it is not getting any intermediate rewards. What does that mean? It means that this game is a cruel teacher that does not tell the AI during the game how well it is doing. Only when it won the level does it tell the AI. So before that, no information is given to the AI whether it is doing well or poorly. This is especially difficult if we need to perform a a chain of actions to win the level, like you see here. And my goodness, I am absolutely stunned that the AI can still do it at all, let alone this quickly. So in just one paper we went from learning in a matter of years to a matter of seconds. Wow. This truly feels like seeing history in the making in artificial intelligence. What a time to be alive. So what do you think? What would you use this for? Let me know in the comments below.
Weights and Biases provides tools to track your experiments in your deep learning projects. What you see here is their amazing sweeps feature which helps you find and reproduce your best runs. And even better, what made this particular run the best. It is used by many prestigious labs including OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you.
Thanks for watching and for your generous support and I'll see you next time.
