A neural network trained on 20 hours of motion-capture data generates realistic fighting moves for game characters in real time. The AI invents novel combinations of attacks and footwork, responds to opponent actions with blocking and evasion, and runs on a standard game controller.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to see if a virtual AI character can learn or perhaps even invent these amazing signature moves. And this is a paper that was written by Sebastian Starke and his colleagues. He is a recurring scientist on this series. For instance, earlier he wrote this magnificent paper about dribbling AI characters. Look, the key challenge here was that we were given only three hours of unstructured motion capture data. That is next to nothing. And from this next to nothing, it not only learned these motions really well, but it could weave them together even when a specific movement combination was not present in this training data. But as these motions are created by human animators, they may show at least three. One, the training data may contain poses that don't quite adhere to the physics of a real human character. Two, it is possible that the upper body does something that makes sense. The lower body also does something that makes sense, but the whole thing put together makes. Does not make too much sense anymore. Or three, we may have these foot sliding artifacts that you see here. These are more common than you might first think. Here is an example of it from a previous work. And look, nearly all of the previous methods struggle with it. Now, this new work uses 20 hours of unstructured training data. Remember, the previous one only used three. So we rightfully expect that by using more information, it can also learn more. But the previous work was already amazing. So what more can we really expect this new one to do? Well, it can not only learn these motions, weave together these motions like previous works, but hold on to your papers, because it can now also come up with novel moves as well.
Wow.
This includes new attacking sequences and combining already existing attacks with novel footwork patterns. And it does all this spectacularly well. For instance, if we show it how to have its guard up and how to throw a punch, what will it learn? Get this. It will keep its guard up while throwing that punch. And it not only does that in a realistic fluid movement pattern, but it also found out about something that has strategic value. Same with evading an attack with some head movement and counter attacking. Loving it. But how easy is it to use this? Do we need to be an AI scientist to be able to invoke these amazing motions? Well, if you have been holding on to your paper so far, now squeeze that paper and look here. Wow. You don't need to be an AI scientist to play with this. Not at all. All you need is a controller to invoke these beautiful motions. And all this runs in real time. My goodness. For instance, you can crouch down and evade a potential attack by controlling the right stick and launch a punch in the meantime. And remember, not only both halves have to make sense separately, the body motion has to make sense as a whole. And it really does. Look at that. And here comes the best part. You can even assemble your own signature attacks. For instance, perform that surprise spinning back fist, an amazing spin kick. And yes, you can even go full Karate Kid with that crane kick. And as a cherry on top, the characters can also react to the strikes, clinch or even try a takedown. So with that, there we go, we are again one step closer to having access to super realistic motion techniques for virtual characters. And all we need for this is a controller. And remember, all this already runs in real time. Another amazing Siggraph paper from Sebastian Staake. And get this, he is currently a 4th year PhD student and already made profound contributions to the industry. Huge congratulations on this amazing achievement. What a time to be alive.
Perceptilabs is a Visual API for TensorFlow, carefully designed to make machine learning as intuitive as possible. This gives you a faster way to build out models with more transparency into.
How your model is architected, how it.
Performs, and how to debug it.
Look, it lets you toggle between the.
Visual modeler and the code editor. It even generates visualizations for all the model variables and gives you recommendations both during modeling and training.
And does all this automatically. I only wish I had a tool.
Like this when I was working on my neural networks during my PhD years. Visit perceptilabs.com papers to easily install the free local version of their system today. Our thanks to Perceptilabs for their support and for helping us make better videos for you.
Thanks for watching and for your generous support and I'll see you next time.
