Dr. Zsolnai covers AI methods that animate creatures by learning motion across species. A system trained on some animals can animate ones it never saw, like cats and Komodo dragons, and can fill in missing animation frames. The authors released the code.
May 15, 2025
Dr. Zsolnai (Two Minute Papers / LARRUZ)
Cat and Komodo dragon
$20 of credit for free
You can "pay up to 10 times less" than elsewhere
Alright, two top tier. Absolutely amazing AI papers. Today this AI animates creatures it has never seen before. And here, even a dinosaur can learn something from a mystery creature. And this one makes a poorly made drawing into a fully fledged 3D animation. Both solve problems that I thought were impossible. Oh my. So what is this? People talk a great deal about breathing life into virtual characters. That is computer animation. But whenever they talk about computer animation, they are talking about moving virtual humanoids around. Humans mostly. But what about these guys? Can we still animate them? Well, the problem is the you are not an animator, so you give an AI the shape of the character that they call a skeleton, and out comes how it would move. And then of course, these skeletons are just the underlying structure. And the final appearance of the character is of course on top of that. Well, people tried to do that with automated techniques, but the results were unsatisfactory to say the least. But with this new method. Surprise. These are really believable motions on a wide variety of icky creatures. That would already be good. But this is nothing compared to what is coming now. You see, these animals don't just move around. They can learn from each other. For instance, this dinosaur is doing something interesting standing on one leg. Where did it learn this from? Oh yes, it learned it from a flamingo. Imagine that. And note that the insane part is that this is not just copying it. It has learned the motion and has adapted it to its own body. That is absolutely incredible. And with this system, a chicken can also learn how to jump from an ostrich as well. Loving it. So adaptation is possible, but how far can we push it? Well, let's give it a try Here. Scientists have excluded the cat and the Komodo dragon from the training set, so the AI has never seen them. And if we do that with the previous technique, unfortunately it seems completely clueless. It's like a confused pile of digital limbs having an existential crisis. No wonder, because it hasn't seen this kind of animal before. Now let's see what the new one can do. Ooo, I can't believe it. Motions generated for unseen animals. This is insanity. So how on earth does it do that? Well, it has looked at a bunch of animals moving around, and there are parts in different animals that are semantically similar parts that have a similar function. For instance, arms and legs are a general concept. And if you look at a new animal, having seen a lot of different kinds of arms and leg morphologies in action, you may be able to infer how it is used. You Fellow scholars can do that? Of course, because you are all brilliant man. I love my audience. But the incredible part is that here a machine is doing that. Generalizing knowledge to unseen examples. And to me, that sounds like intelligence. Wow. And you'll love this one. Look. You can even peer into what it is thinking about these motions. It understands when this animal is relaxed and when it is attacking. And to demonstrate trait that it really understands this, it shows similar motions in completely differently shaped animals. Yes, it has learned the concept of attacking. Same vibe, different body. So cool. And it also understands when other different kinds of motions, like falling or growling start, and when they end, and how they look in different animals. But it gets even better. Now hold on to your papers, fellow scholars. Because it can even take an input motion that is incomplete. And when it switches to orange, you see how the AI synthesized the rest of the motion. Goodness. This is like image inpainting, filling out missing parts of an image, only for animation. Brilliant. And now look. Are you kidding me? Here they took an animation of somebody parts and left out the rest. And the orange part, for instance. The upper body motion is then simulated by the AI and the source code is freely available for all of us to experiment with. Thank you so much. Now imagine combining it with a technique that could take a character, in this case humanoids. Then you just draw a line to indicate what it should be doing. And it would perform the motion accordingly. That is of course possible. And that technique is called becoming an artist who is excellent at their job. But doing this automatically with an AI and not a chance. Why? Dear fellow scholars, this is two minute papers with Dr. Zsolnai. I'll show you why. Let's punch something this way and see if that is possible. Here is the line. Now little AI, you do the rest. Well, it is loading up, but then. Hey, nothing happens. Where's the punch? This other one. Look carefully. Why did it do that? Well, here's the problem. We humans understand the world in 3D, even if shown just a piece of drawing on a 2D paper. But this technique doesn't. It does not understand that we meant that the line is forward, not sideways. But of course, once again, how on earth would it understand it? If we wanted to move backwards, we would use the same line. So it would kind of have to understand the intention behind the drawing as well, which is of course impossible. So even when we would try a new method to do that. Wait a minute. It understood. This is exactly what we wanted. Wow. So yes, with this other technique, we can create a simple storyboard without any artistic training and it works incredibly well already. One more problem that used to be impossible which is now possible. It is truly stunning what it is capable of and once again. Yup, look at that. These papers might as well be some secret super projects because almost nobody knows about them. This is why I am making two minute papers and I hope you appreciate it too. This is arcane knowledge of the impossible. I don't think you hear about these anywhere else. Fellow scholars, check this out. This is Deep Infra, an AI inference cloud where you can get access to the best OpenAI models like Deep Seq Llama and pay up to 10 times less than elsewhere. Or use my favorite the amazing Flux text to image AI, generate speech from text and more. It is super easy to use. Try it out today@deepinfra.com papers or click the link in the description and you get $20 of credit for free.
