EA’s New AI: Next Level Gaming Animations!

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to have a look at this new AI that is able to look at a bunch of unstructured motion data like this. Then place a character in a video game and see all the amazing things that it can learn from it. Walking, running, dancing, you name it. And look at that. Yes, this works for bipeds and quadrupeds at the same time. Now that is quite a challenge. So let's see what is going on here first, why is this a challenging problem? Why do we not just copy the movements from the training data? These were recorded by real humans and dogs after all. Well, that will not cut it here. You see, in video games we get to control these characters, which means that we can stop any movement at any time and start a new one. Aha. So this requires neural networks to look at a big soup of motion data. And how big is this soup? I will tell you in a moment and I think you will be very surprised. And wait. Even learning the essence of these movements is not enough. It needs to learn about transitions too, by itself. Oh yes, transitions are key.

Why?

Well, look at this. Here is a previous method. And whenever we change direction, look, these transitions are quite unnatural. And this is not parkour. We are still talking about just running around. A much simpler task to animate. And it is still not working too well. Not good. And oh my. Are you seeing what I am seeing? What is that? Oh yes, that's right. This is foot sliding, the bane of our existence. More on that in a moment. But now let's see how the new method solves this problem. Wow. Now that is fantastic. So much better. And what about the doggies? Well, previous methods are not too bad here, but the new one is so much more fluid. The movement of the body is now better. But if you don't find that too noticeable, also check the movement of the tail too. Previous methods seem a lot more tentative, while the new one is so much more lifelike. This new technique can also perform more advanced actions. I loved how it performs the dribbling here. And note that the ball is controlled by the physics engine, so the character has to react to the ball's movement quickly and convincingly. And the dance moves it has are really cool too. It can not only dance, but it can dance so much more convincingly than previous methods. Loving it. But if we wish to get even more crazy, we can even combine different movements for the lower and upper body. And I know from previous papers that we can't Just copy paste two motions together to get this. These are so much more challenging. And the new method does it with flying colors. So good. One of the many key insights in.

The paper is that they propose using.

This diagram, which is the phase space. And as we pick a point and move it around there, our character starts moving. However, not all spaces are this intuitive lock. With previous techniques, good movements required this crazy pass and were thus more difficult to achieve. We have two more incredible insights about this paper now. One, hold on to your papers, because this is where I fell off the chair when reading this paper. As promised, let's have a look at how much training data this technique required to learn all these beautiful fluid movements. What are you seeing? What I am seeing? That is impossible, right? Look, it learned to animate quadrupeds by just seeing 17 minutes of footage. And perhaps even better, dancing just nine minutes. That is absolutely amazing. And the second insight is about foot sliding. This new technique shows less foot sliding in most cases than previous methods. And even in the worst case it is comparable. Look, this was a huge problem for previous techniques, but really, whatever metric we use to measure the new one against previous techniques, it performs better in pretty much all of them. Incredible. And huge respect to Sebastian Staake, first author of this work, who just keeps publishing these incredible papers one after another, and almost always with the source code as well. And. And yes, this also means that the source code for this technique is also available. So from now on we can soon expect much more lifelike character movements in our games and virtual worlds. What a time to be alive.

This video has been supported by Weights and Biases. Check out their recent offering Fully Connected, a place where they bring machine learning practitioners together to share and discuss their ideas, learn from industry leaders and even collaborate on projects together. You see, I get messages from you fellow scholars telling me that you have been inspired by the series, but don't really know where to start. And here it is. Fully Connected is a great way to learn about the fundamentals, how to reproduce experiments, get your papers accepted to a conference and more. Make sure to visit them through WNB Me papers or just click the link in the video description. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you.

Thanks for watching and for your generous support and I'll see you next time.