This AI Makes You A Virtual Stuntman! 💪

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we will try to create a copy of ourselves and place it into a virtual world. Now this will be quite a challenge. Normally to do this we have to ask an artist to create a digital copy of us, which takes a lot of time and effort. But there may be a way out. Look, with this earlier AI based technique, we can take a piece of target geometry and have an algorithm try to rebuild it to be used within a virtual world. The process is truly a sight to behold. Look at how beautifully it sculpts this piece of geometry until it looks like our target shape. This is wonderful. But wait a second. If we wish to create a copy of ourselves, we probably want it to move around too. This is however, a stationary piece of geometry. No movement is allowed here. So what do we do? What about movement? Well, have a look at this new technique where getting a piece of geometry with movement cannot get any simpler than this. Just do your thing, record it with a camera and give it to the AI. And I have to say I am a little skeptical. Look, this is what the previous technique could get us. This is not too close to what we are looking for. So let's see what the new method can do with this data. And. Oh, this is not great. So is this. It is the geometry cloning dream dead. Well, don't despair quite yet. This issue happens because our starting position and orientation is not known to the algorithm. But it can be remedied.

How?

Well, by adding additional data to the AI to learn from. And now hold on to your papers and let's see what it can do now. And. Oh my goodness. Are you seeing what I am seeing? Our movement is now replicated in a virtual world almost perfectly. Look at that beautiful animation. Absolutely incredible. And if even this is not good enough, look at this result too. So good. Loving it. And believe it or not, it has even more coolness up the sleeve. If you have been holding onto your paper so far, now squeeze that paper because here comes my favorite part of this work. And that is a step that the authors call scene fitting. What is that? Essentially what happens is that the AI reimagines us through a video game character and sees our movement, but does not have an idea as to what our surroundings look like. What it does is that from this video data, it tries to reconstruct our environment, essentially recreating it as a video game level. And that is quite a challenge. Look at first it is not close at all, but over time it learns what the first obstacle should look like, but still the rest of the level, not so much. Can this still be improved? Let's have a look together as we give it some more time and our character a few more concussions, it starts to get a better feel of the level and it really works for a variety of difficult dynamic motion types, cartwheels, backflips, parkour jumps, dance moves, you name it. It is a robust technique that can do it all. So cool. And note that the authors of the paper gave us not just the blueprints for the technique in the form of a research paper, but they also provide the source code of this technique to all of us free of charge. Thank you so much. I am sure this will be a huge help in democratizing the creation of video games and all kinds of virtual characters. And if we add up all of these together, we get this. This truly is a sight to behold. Look. So much improvement. Just one more paper down the line and just imagine what we will be able to do a couple more papers down the line. Well, what do you think? Let me know in the comments below.

This video has been supported by Weights and Biases. Check out their recent offering Fully Connected, a place where they bring machine learning practitioners together to share and discuss their ideas, learn from industry leaders and even collaborate on projects together. You see, I get messages from you fellow scholars telling me that you have been inspired by the series and but don't really know where to start. And here it is. Fully Connected is a great way to learn about the fundamentals, how to reproduce experiments, get your papers accepted to a conference and more. Make sure to visit them through WNB Me papers or just click the link in the video description. Our thanks to WeightsAndBiases for their long standing support and for helping us make better videos for you.

Thanks for watching and for your generous support and I'll see you next time.