NVIDIA trained a virtual humanoid agent for 5 billion steps on a single GPU using a three-stage pipeline: imitation learning from motion-capture data, then tuning for precision and control. The result moves more naturally than prior work while staying easier to command. The team released the code.
Questions this video answers
5,000,000,000 training steps.
ASE (a previous work with long in-game training).
10 years in the simulated world, which the speaker says corresponded to about 10 days of real time on a fast machine.
1) Basic training (imitation from unlabeled motion-capture), 2) Precision training (controllability to player inputs), 3) Intuitive commands (higher-level actions like strike/run).
Yes — the presenter thanks the scientists for making the source code publicly available.
Chapters
Today we will see how Nvidia's new AI sends these little virtual soldiers to camp and teaches them how to fight. This is a follow up work to an absolutely amazing paper called ASE. This is a previous work with little AI soldiers that trained for 10 years. 10 years? How? Well, 10 years of their in game world, that is who. Which is fortunately, when using a fast machine, is only 10 days in our time. And they learned a great deal. They went from a bunch of sorry little recruits to battle ready soldiers. Except this chap. This chap goes like Sir, I've been training for two years, I've had enough. And now I shall leave in style. And in this new work, these little soldiers trained for 5 billion training steps on a single graphics card. It's not a cheap one, but still a single graphics card. So what do we get for these 5 billion training steps? Can this new one do anything better than the previous one? Well, first it can learn from unlabeled motion capture data. What is that? Well, a bunch of dots moving that represent movements of real humans that have been recorded. And now step number one, basic training. Here the moving dots go in and a neural network is asked to create motions that are similar to it. So, are we done? Well, not quite. You see, we wish to use this in video games. And this is not controllable yet. Imagine a video game where you press a button on the controller and nothing happens. So now comes step number two, precision training. Here, it now needs to perform the movements, but also listen to our controls, especially when we use the stick to steer it in different directions. And step number three, here we not only give it directions, but intuitive commands as well, such as striking or running. All that is great, but what can it do now? Well, let's see. Together, for instance, it can interpolate between two movement types. This means that it works like a gymnast who has perfected a difficult routine. It can't jump straight from one move to another. It needs to transition smoothly to maintain balance and perform the routine successfully. Thus, it first starts running and then, look, it gradually dreams up a smooth transition from sprinting to crouching. That is a really nice and believable transition. Love it. And now comes the best part. Hold on to your papers, fellow scholars, because now you'll see how it can do these types of transitions on demand. That means as a reaction to our button meshing on the controller. So good. However, not even this technique is perfect. It still has some issues to be worked out. For instance, check this out. Now, little AI move towards the target Attack. Great. And now celebrate. Whoa. Careful. Moments like this show that this work is still experimental. I don't think you will see this in a new game tomorrow. But this is what research is about. Taking something highly unstructured and using learning based techniques to create something useful from it. Now wait a second. In the intro I also said that this is a follow up to the ASE paper. So how does it compare to that one? Oh my. Are you seeing what I am seeing? Because I am seeing two excellent news here. Now I hear you asking. OK doctor, what are the excellent news here? Well, one, let's start with the most obvious one. The controllability of the simulation has improved leaps and bounds. That is incredible. And second, what is even more incredible, this controllability did not come at the expense of the diversity of the movesets. That is perhaps the best part of the paper. You see, this is in a class of neural network based techniques that often suffer from a phenomenon called mood collapse. What is mood collapse? Well, imagine asking a gymnast to perform incredible acrobatics. And this gymnast can do a variety of things, but unfortunately just keeps doing the same thing over and over again. Cartwheels all day, every day. Imagine looking at an Olympic gold medal athlete who just keeps doing those cartwheels. And it is not the case here. This is an athlete that can do everything. I absolutely love this. Bravo. What a time to be alive. And finally, some fantastic news for the end. The source code of this project is available. I would like to send a big thank you to the scientists on this project, making it available for all of us free of charge. And for now, let the experiments begin. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud compute. No commitments or negotiation required. Just sign up and launch an instance. And hold on to your papers. Because with the Lambda GPU cloud, you can now get on demand H100 instances for just 1,99 per hour. Yes, 1,99. And they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, mit, HR and Caltech in using Lambda Cloud instances. Workstations or servers. Make sure to go to lambdalabs. Com papers to sign up for one of their amazing GPU instances today. Thanks for watching and for your generous support and I'll see you next time.
