This content was created by LARRUZ. This content was uploaded by LARRUZ. This is a YouTube video titled "This New AI Makes DeepFakes... For Animation Movies! 🧑‍🎨" created on April 12, 2022. It features Dr. Zsolnai discussing a new AI technique for reimagining videos and movies by adding elements like smiles or age changes, with a focus on temporal coherence. The video includes examples of altering real and animated characters and highlights the potential of AI in video synthesis.
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to use this new AI film director technique to reimagine videos and even movies just by saying what we want to add to it. Yes, really. And you will see that the results are absolute insanity. Now, previous techniques are already able to take a photo of someone, make them young or old, but this work, this is something else. Why this can do the same for not only one still image, but for an entire video. Just imagine how cool it would be to create a more smiling version of an actor without actually having to re record the scene. Hold on to your papers and look at how we can take the original video of an Obama speech and make it a little happier by adding a smile to his face or even make him younger or older. And these are absolutely amazing. I love the gray hair. I love the young man hair. I also love how the gestures and head turns are going through. This looks like something straight out of a science fiction movie. Loving it. Now, what is the key here? The key here is that this AI knows about temporal coherence. What does that mean? Well, it means that it does this in a way that when it synthesizes the second image, it remembers what it did to the first image and takes it into consideration. That is not trivial at all. And if you feel that this does not sound like a huge deal, let's have a look at an earlier technique that doesn't remember what it did just a moment ago. Prepare your eyes for a fair bit of flickering. This is an AI based technique from about six years ago that performs style transfer. And when creating image two, it completely disregards what it did to image one. And as you see, this really isn't the way to go. And with all this newfound knowledge, let's look at the footage of the new AI and look at the temporal coherence together. Just look at that. That is absolutely incredible. I am seeing close to perfect temporal coherence. My goodness. Now not even this technique is perfect. Look, when trying it here, Mark Zuckerberg's hair seems to come to a life of its own. I also find the mouth movements a bit weaker than with the other results. And now, if you allow me, let me also enter the fray. Well, this is not me, but how the AI reimagines me as a young man. Good job, little man. Keep those papers coming. This is incredible. I love it. And this is what I actually look like. Okay, just kidding. This is old man Caroj as imagined by the AI who is vigorously telling you that Papers were way better back in his day. This is the original and this is my sister Carolina filling in when I am busy. I would say that the temporal coherence is weaker here, but it is understandable because the AI has to do a ton of heavy lifting by synthesizing so much more than just a smile. And even with all these amazing results so far, you have seen nothing yet. No sir, it can do even more if you have been holding on to your paper so far. Now squeeze that paper because it works not only on real humans, but animated characters too. And just look at that. We can order these characters to smile more, be angrier or happier, and we can even add lipstick to a virtual character without having to ask any of the animators and modelers of this movie. How cool is that? And remember, we were lucky to be able to do this to one still image just a couple papers ago. And now all this with temporal coherence. And I hope you are enjoying these results, many of which you can only see here on two minute papers, nowhere else. And of course I would like to send a huge thank you to the authors who took time off their busy day to generate these just for us. Now even the animated movie synthesis has some weaker results in my opinion. This is one of them, but still very impressive. And don't even get me started about the rest. These are so good and just imagine what we will be able to do just a couple more papers down the line. My goodness, what a time to be alive. So what would you use this for? What do you expect to happen a couple more papers down the line? Please let me know in the comments below. I'd love to hear your thoughts. Weights and Biases provides tools to track your experiments in your deep learning projects. Using their system you can create beautiful reports like this one to explain your findings to your colleagues better. It is used by many prestigious labs including OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
