In this Two Minute Papers episode, Dr. Károly Zsolnai‑Fehér presents a recent NVIDIA research paper that teaches a virtual human to perform parkour by simultaneously learning to imitate human motion and to solve goal‑directed obstacle courses. The method places a single controller in two “classrooms”: one where it copies a tiny dataset of 19 parkour clips (about 30 seconds total) to generate human‑like movements, and another where it learns to navigate arbitrary obstacle layouts. An adversarial judge receives the character’s body pose, obstacle configuration, and destination, and is trained to distinguish real human motions from the AI’s output while also being conditioned on the context. The controller is optimized to both fool the judge (producing human‑like motion) and satisfy the task objectives, with an additional stability term to keep the judge’s decisions smooth. The video showcases the AI’s ability to compose and adapt skills across a variety of seen and unseen levels, achieving impressive qualitative results, though success rates on longer levels drop to about 40% and occasional unnatural recovery motions appear. The presenter highlights the remarkable data efficiency of the approach, the trade‑off between tracking error and success rate, and notes that the paper and code are freely available. He also promotes Lambda’s GPU cloud platform, which he uses to reproduce the experiments quickly.
Chapters
This is one of the most fun papers I've read in a while, you will love this one. So, what if you knew every athletic move perfectly, but had no idea when to use which? A perfect vault is useless if the table moves away. This is the strange problem of teaching parkour to a virtual human. Systems that copy us are beautiful, but brittle. Repetitive. Systems that chase goals adapt, but often stop moving like us. But this new AI work says enough is enough, let's do both. It promises to remember exactly how humans move, yet is able to break the script when needed. And get this, here's the crazy part. The training data. They say that all you have is only 30 seconds of internet parkour to pull this off? That, Fellow Scholars, Sounds impossible. You see, previous techniques were not up to the task. This guy moves like a drunkard. Not to worry, try again. And— hehe. Yup, get some rest. Next. Now this guy looks a bit more promising. Wait a second, that's cheating! You are supposed to get through the level, not avoid it, mister. That is cheating. Get out of here! Previous technique number 3. Ooh, now we're talking. Look at that! Oof, what an insane research field. This is so much fun. Okay, so can this new technique solve this? Because the previous ones couldn't. Let's see if it can solve the level. I don't think so. Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Now hold on to your papers, Fellow Scholars, and let's see! That's a strong start. Very athletic. Ooh, look at that! Solving the level like a ninja. It really adapts and composes new skills to get through the level. Okay, so we are experienced Fellow Scholars here, so we ask: was this the only level it can finish, or is this the real deal? Can it solve longer levels? Let's see. Oh my, we are off to a good start. Still going? Yes! Incredible! Good job! Ouch. Okay, get well soon, little AI. Now, after taking a trip to the AI hospital, let's try a variety of levels at the same time. Ooh, this is amazing. Love it. Now, let's make it harder. A level with unseen obstacle arrangements. And, oh my, still doing amazingly well. And the final jump. Now that is a sight to behold. And it gets better, because I will show you my favorite move, and it's kind of the best. Okay, so how on Earth did they do that? Well, they put one character into two classrooms. In the first, it learns to copy human movement. This produces beautiful human-like motions, but it is limited by the training data. It cannot go beyond what it has seen. And remember, that is only 19 clips containing 30 seconds of parkour. Very little data. In the second classroom, it learns to solve new obstacle courses. And from this point on, the same AI controller learns in both classrooms simultaneously. Do not forget about the part where both classrooms give it body, obstacle, and destination information. Super important. And here comes the most beautiful part. Hmm, it's quite the formula, but we'll break it down. I'll try my best. My best way to imagine this is that now we train a judge to minimize 3 kinds of error. Here are real human movements, together with the obstacles around them. The judge should call these real. Then, here, the movements are produced by the AI. The judge should call these fake. Now, we use this judge to train our AI athlete. If it moves like a clumsy AI, it gets a poor score. Get outta here! But as its movements become more human-like and appropriate for the obstacle, it gets a treat—a good score. And the judge and athlete keep improving together. The athlete gets better at fooling the judge, while the judge gets better at spotting artificial movements. This is absolutely beautiful. Finally, this last term keeps the judge stable, so tiny changes in a movement do not completely change his decision. And this is what leads to these absolutely stunning results. But it is not perfect. It offers lower tracking error than its predecessors, but at a cost of a bit of a hit to the success rate. This tradeoff is among the weaker spots, I think. So I would chalk this up as a limitation. Longer levels still only have about 40% success. The other limitation is that unnatural recovery motions are also possible. Now, as promised, here is my favorite move in the sequence. Wow! This is incredible. What a time to be alive! But here is the most important thing. The research paper is freely available. Free knowledge for all of us, and potentially code later too. So, all of this knowledge is out there for all of us for free. And nobody is talking about it. This is driving me mad! This is why I keep making these videos. This is what I live for. Every research paper is a gift to humanity, and I want to make sure that humanity receives this gift. I use Lambda to reproduce AI research papers, often in minutes. It's also great to train your own models, or fine-tune an existing one. Run inference, or text to image or video? Easy peasy. Running a Deepseek chatbot or agent? Super fast, super reliable. Lambda gives you powerful NVIDIA GPUs to run your own experiments. I test ideas from the papers I cover, and moments later, results. Love it. Seriously, try it out now at lambda.ai/papers.
