Dr. Zsolnai explains how AI systems now learn 1000x faster using simulations. The video covers large language models playing games, writing code, and achieving superhuman performance in tasks requiring physical dexterity.
Here is an incredible paper that teaches virtual humans to run in the most fabulous ways. So good. So what is this? How did we get here? These are large language models, text based AI systems that can be our smart assistant, someone who can draw images for us. We all know that. However, fewer fellow scholars know that they are excellent at reading and writing computer code and can thus even learn to play Minecraft. That is incredible. Just think about it. This is a text based AI. How could it possibly control a graphical game like this? Really amazing. So at this point we know what these large language models can do. The age of surprises is is over. Or so I thought. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Previous AI based techniques can play Atari games and other games at a superhuman level. They typically do it through reinforcement learning, which means that they get a controller, look at the screen and play like a human would. DeepMind's techniques are excellent at this and thus they already do extremely well on these games. However, there is a huge problem. What is the problem? Well look, scores. Games have scores and these scores can be used as feedback for the AI to understand whether it is doing well or not. High score, doing well, lower score, not so much. And whatever other task we wish it to perform. There needs to be a score, there needs to be feedback. And therein lies the problem. Each game has a different scoring mechanism and thus each task needs a different scoring mechanism. If we are aiming to create an intelligence of sorts, any kind of intelligence requires the generality. So it has to be able to perform tasks it hasn't seen before. But how? Well, scientists at Nvidia had a crazy idea. Let's use these ChatGPT ISH large language models to write code to calculate the score for these tasks by itself. Wow, I love it. But does it work? I think that cannot possibly work. There are just too many kinds of tasks out there. Well, let's have a look together. Now little AI write code for the score for this humanoid to run and then train it. Wow, it works well. Look, no one said it has to be beautiful, we said it has to run. Running is making progress, moving forward, it recognized that. And this already works. And. And the whole thing was written by an AI. Now get this, let's try some other way. First, little AI design a task where forward movement is necessary. Wow, that is a fabulous way of moving forward. Now let's give it some feedback. This looks like squat jumps make the movement resemble running a little more. So now the AI says ok, got it. So you mean a duck walk? Well, not quite. No feedback. The torso has to be a little higher. Ok, good. But it is using mostly one leg to hop on. Please use both legs. Goodness. Both legs are now being used. But look. Oh boy, back to square one. Now the torso is too low again. Let's ask it to penalize that. And there we go. Finally, something that resembles running. It's not an easy task at all, it turns out. And this human feedback driven solution I think is way better than the AI on its own. This intelligence of sorts is wonderful, but human intelligence is where it's at. What a fantastic paper. But wait, it gets better. In fact, here comes the best part. This concept generalizes to so many tasks. From passing balls to balancing them, teaching robots to move, and even the crowd favorite spinning pens. Very impressive. But how does this train so well and so quickly? The key is that this learns within a computer simulation that has two one, it can't really harm itself. And two, my favorite. In the real world, one second means one second. But in a virtual world, one second can be simulated much quicker. Given a powerful computer, a little game can be simulated quicker. How much quicker? Well, in this case, this little AI is learning a thousand times quicker in a simulation than in the real world. So cool. And it gets even better. Hold on to your papers, fellow scholars. Because it can not only match the level of humans playing these games, but the evolution based variant can even showcase superhuman performance. Wow. It matches or exceeds the level of humans on 75% of dexterity based tasks. That is insanity. Finally, an AI that can do these tasks and have some sort of generality. What we are seeing here is perhaps the early days of true intelligence being born. And all this within a little piece of silicon and lots and lots of human ingenuity. What a time to be alive. However, it is a research work and it is not perfect. Not even close. For instance, please don't ask it to close your doors. Ouch. If you are looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud computer. No commitments or negotiation required. Just sign up and launch an instance and hold on to your papers. Because with the Lambda GPU cloud you can now get on demand H100 instances for just 1,99 per hour. Yes, 1,99. And they are one of the first cloud providers to offer publicly available on demand H100XS. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda cloud instances. Workstations or servers make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today.
