YouTube explainer by Károly Zsolnai-Fehér (Two Minute Papers) showcasing an NVIDIA AI that learned character motion from 4,000,000 frames (~1,000 minutes). The speaker compares the new diffusion-like motion synthesis technique to previous methods, explains that low-level noise appears as twitching, and demos seven experiments: improved natural turns and reduced foot sliding; path-following with styles; stochastic transitions between poses; improvisation between pose 'molds'; following dynamic targets; joystick-based game control; and long-horizon coherent planning. The video concludes with a Lambda Labs sponsorship call-to-action.
Chapters
Today we are going to see how we can teach an AI to create beautiful, elegant and realistic motions for these characters. It had the opportunity to learn from 4 million frames. That is about a thousand minutes of motion. So what does it have to show for it? Ooh, alright, that's not a good start. In fact, it looks completely hopeless. Look how it is twitching here now. Now let's have a previous technique. Try it. Ok, no twitching. That's good. But oh, also not good. And oof. It gets even worse. And then even worse. It is almost like an invisible hand just started yanking the character around. So this is doing crazy stuff. And the new technique twitches. But the question is, why is it twitching? Well, these animations are synthesized a bit like image generator AI techniques. Those start out from a noisy image and then over time clear up that noise to get you a beautiful image. And get this here, the noise in the animation shows up as twitching. But in images, this noise can slowly be removed over time. And yes, depending on how much of this denoising we do, the better things get, the more diverse these motions become. Up until a point, of course, this looks pretty good. So now let's test it and see if it really is better than previous techniques. Nice. I see lots of different kinds of motions. Great variety too. Also no invisible hand. Maybe just a little. So that is a fantastic baseline. But now let's see how far we can take it through seven amazing experiments. Experiment 1. When comparing to a previous technique, I love how much more natural this feels. Turns are much more realistic than the previous method, which also has. Look at that unnatural foot sliding for the previous one. I see very little of that with the new one. Great. Experiment two. Now try to follow a path. And it not only does that, but it can also do it in different styles. So cool. This is great if you want to avoid an obstacle and have a specific path that it should take. Not a problem. So cool. Experiment three. Now little AI cook up transitions between any two poses and take into consideration the motions specified by a user. And here comes the best part. Look, every time it does it, it can do it a bit differently. And hold on to your papers, fellow scholars, because all this runs in real time. Next level, video games, here we come. So motion generation check mark. Now let's look at control four. Let's put down these molds. And like in a funny TV show, you have to take these poses, but between them, improvise. So good. Five, follow these targets. Now we just said that this is real time. So are you thinking what I Am thinking yup, we can specify new ones on the fly and it adapts to the new situation quickly. 6. Now I wonder, this is all well and good, but can it really be used in a video game? Is it possible to get that degree of control over an AI technique? And the answer is yes. And it is not nearly as complex as it sounds. Quite the opposite in fact. You can just grab a joystick and give it commands. With that. Of course the motion will not be instantaneous. It takes into consideration how long it takes to turn around. Super realistic. And now 7 if we are not looking for too much control and would like to observe it doing its thing, that is also completely fine. Remember this mold experiment required that the AI would plan between the molds. And look here. This is crazy. It can plan so long into the future. We can just keep watching and watching and it still moves coherently. This is not trivial at all. Why is that? Well, this particular figure learned from a database with recordings of lots of humans moving around. And these are lots and lots of short snippets. So the fact that it can weave these short motions together is an absolute miracle. Loving it. AI research papers are incredible. We just learned so much from this and the fact that we could see this kind of AI based technique in the video games and virtual worlds of the future. Sign me up right now. What a time to be alive. And if you feel inspired and you wish to run your own experiments on Nvidia GPUs, check out Lambda. Lambda is a cloud service for large scale AI projects and it is my absolute favorite. Seriously. Try it out now@lambdalabs.com papers or click the link in the description. If you use Lambda, you can run your own experiments on super powerful hardware and you also support two minute papers.
