A Two Minute Papers video (hosted by Dr. Zsolnai) summarizing an NVIDIA research paper that trains virtual humanoid agents in the Isaac simulator for the equivalent of 10 years of experience—accelerated in simulation so 10 years of learning can be produced in roughly 10 days of wall-clock time on powerful hardware. The video walks through demo clips showing skill progression over simulated training time and highlights four technical contributions: use of latent spaces for motion switching and blending, robustness/recovery from perturbations (e.g., thrown boxes), fine-grained directional and pose control, and adversarial motion synthesis (generator vs. discriminator). The video also references a Weights & Biases report and includes a sponsor mention.
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today you will see an absolute banger paper. This is about how amazingly Nvidia's virtual characters can move around after they have trained for 10 years. 10 years? We don't have 10 years for a project. Well, luckily we don't have to wait for 10 years. Why is that? I will tell you exactly why in a moment. But believe me, these folks are not natural born warriors. They are AI agents that have to train for a long, long time to become so good. So how does that work? Well, first our candidates are fed a bunch of basic motions and then are dropped into Nvidia's Isaac, which is a virtual gym where they can hone their skills. But unfortunately, they have none. After a week of training, I expected that they would showcase some amazingly athletic warrior moves, but instead we got this. Oh my goodness. Well, let's be optimistic and say that they are practicing judo, where the first lesson is learning how to fall. Yes, let's say that then after two months. Oh, we can witness some improvement. Well, now they are not falling and they can do some basic movement, but they still look like constipated warriors. After two years, we are starting to see something that resembles true fight moves. These are not there yet, but they have improved a great deal. Except this chap. This chap goes like. Sir, I've been training for two years. I've had enough. And now I shall leave in style. I wonder what these will look like in eight more years of training. Well, hold on to your papers and let's see together. Oh my. That is absolutely amazing. Now that's what I call a bunch of real fighters. See, time is the answer. It even made our stylish chap take his training seriously. So which one is your favorite? From here, did you find some interesting movements? Let me know in the comments below. Now, I promise that we will talk about the ten year thing. So, did scientists at Nvidia start this paper in 2012? Well, not quite. This is 10 years of training, but in a virtual world. However, a real world computer simulates this virtual world and it can do it much quicker than that. How much quicker? Well, a powerful machine can simulate these 10 years. Not in 10 years, but in 10 days. Oh yes, now that sounds much better. And we are not done yet. Not even close. When reading this paper, I was so happy to find out that this new technique also has four more amazing features. One, it works with latent spaces. What is that? A latent space is a made up place where similar Kinds of data are laid out to be close to each other. In our earlier paper, we used such a space to create beautiful virtual materials for virtual worlds. Nvidia here uses a latent space to switch between the motion types that the character now knows. And not only that, but their AI also learned how to weave these motions together, even if they were not combined together in the training data. That is incredible. 2. This is my favorite. It has to be. They not only learn to fall, but in those 10 years, they also had plenty of opportunity to learn to get up. Do you know what this means? Of course. This means the favorite pastime of the computer graphics researcher, and that is throwing boxes at virtual characters. We like to say that we are testing whether the character can recover from random perturbations. That sounds a little more scientific. And these AI agents are passing with flying colors, or flying boxes, if you will. Wow. 3. Also, the controls are excellent. Look. This really has some amazing potential to be used in virtual worlds, because we can even have the character face one way and move into a different direction at the same time. More detailed poses can also be specified. And what's more, with this, we can even enter a virtual environment and strike down these evil pillars with precision. Loving it. 4. These motions are synthesized adversarially. This means that we have a generator neural network creating these new kinds of motions. But we connect it to another neural network called the discriminator that watches it and ensures that the generated motions are similar to the ones in the dataset and seem real too. And as they battle each other, they also improve together. And in the end, we take only the motion types that are good enough to fool the discriminator. Hopefully, these are good enough to fool the human eye too. And as you see, the results speak for themselves. If we wouldn't be doing it this way, here is what we would get if we trained these agents from scratch. And yes, while we are talking about training, this did not start out well at all. Imagine if scientists at Nvidia quit after just one week of training, which is about 30 minutes in real time. These results are not too promising, are they? But they still kept going. And the result was. That is excellent life advice right there. And also an excellent opportunity for us to invoke the third law of papers. Not the first, the third one. This says that a bad researcher fails 100% of the time, while a good one only fails 99% of the time. Hence, what you see here is always just 1% of the work that was done. And all this is done by Nvidia. So I am sure that we will see this deployed in real world projects where these amazing agents will get democratized by putting it into the hands of all of us. What a time to be alive. So does this get your mind going? What would you use this for? Let me know in the comments below. What you see here is a report of this exact paper we have talked about with was made by Weights and Biases. I put a link to it in the description. Make sure to have a look. I think it helps you understand this paper better. Weights and Biases provides tools to track your experiments in your deep learning projects. Using their system you can create beautiful reports like this one to explain your findings to your colleagues better. It is used by many prestigious labs including OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
