Dr. Zsolnai explains how embodied avatars let players control in-game characters using their own body movements. The technology uses AI to map real motion to digital characters, opening new possibilities for how games work.
Now this paper is next level for gaming. It is about embodied avatars. What is that? Well, it is you, not with a controller, but actually you yourself with your own body within a video game. You can move around in an intergalactic something. Use your lightsabers to defeat these evil bunny boxes, or if aim and shoot in a video game. But once again, not with a controller, but by moving your body. Absolutely amazing. So is this really coming? Is this even possible? I'll give you a this should be absolutely impossible. Why? Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Well, previous techniques were not so good at this. They make your virtual character move in ways that you did not move in, suffer from a bit of a delay, or they simply don't understand the kinds of movements for a practical game environment. After all, this task is about mapping the sensor readings with a headset and a couple of controllers into real human motion. And of course, all this in real time. Now wait, wait, wait. We got to talk about this. Look, this also means that we don't have any access to lower body information. So all this should be impossible. How does it know what the lower body is doing here? Well, first it guesses from your motion. Now what your motion might look like in the next few moments. This makes it more responsive. And. And then it also guesses from the upper body motion what the lower body should be doing. Then fuse the lower and upper body motion together. Wow, that is absolute insanity. And I did not think that was possible at all. And yet, look at that footage. If you look closely, you see that as the hand moves down here, we can actually guess what the knees are supposed to be doing to achieve this upper body motion. If you look here, you see that there is some uncertainty. It is not always perfect. This also works in games really well. The lower body here does not look exactly the same as in reality, but there is some correlation between the two. One can be guessed from the other with a reasonable chance of success. Beautiful. But it gets even more difficult. Previous techniques are also not that great at following a prescribed path. The movements were a little too wooden. And let's have a look at this new technique. And oh my, that is so much more natural. Loving it. The new method also generalizes to different body types, which is not trivial at all. And we can also get crazy with the movements, sit down, play baseball, and so much more. And get this, if we don't have too much space around us, or let's be honest, we are a bit lazy. And this AI technique is also ready to give us direct control of the lower body through a joystick. And that means. Oh yes. Comfy Gaming. Loving it. Now hold on to your papers, fellow scholars, because it can even make sure that our fingers are moving correctly in the game and that they don't collide. Yes, now you can even pet a bunny in a virtual world or be fabulous like this. And we only need one more ingredient to make all this happen. So what is the only ingredient that remains? Well, three hours of motion capture data. Yes, the AI looks at three hours of human movement and if you add this algorithm it is able to perform this absolute miracle. An absolute slam dunk. Paper. Wow. Make sure to check it out in the video. Description, source code and a playable demo is also included. So good. Loving it. I cannot wait to see this in the games of the future. What a time to be alive. Also, what do you fellow scholars think? What would you use this for? Let me know in the comments below. If you're looking for an Nvidia GPU cloud service for large scale AI projects, check out One Click Clusters by Lambda, featuring up to 512 Nvidia H100 GPUs with InfiniBand Networking. And hold on to your papers because with One Click Clusters you can access all these GPUs on demand with reservations starting at just two weeks. Try it out at LambdaLabs.com paper or click the link in the description.
