Dr. Károly Zsolnai-Fehér shows how StyleGAN3 ages and de-ages celebrity faces by walking through latent space. The catch: the AI can't perfectly embed real photos, so the aged version looks like the person but isn't an exact match. You can try it yourself.
Dear fellow scholars, this is 2 Minute Papers with Dr. Today we are going to take a bunch of celebrities and imagine what they look like as tiny little babies. And then we will also make them old. And at the end of this video, I'll also step up to the plate and become a baby myself. So what is this black magic here? Well, what you see here is a bunch of synthetic humans created by a learning based technique called Stylegan3 which appeared this year in June 2021. It is a neural network based learning algorithm that is capable of synthesizing these eye poppingly detailed images of human beings that don't exist and even animate them. Now how does it do all this black magic? Well, it takes walks in a latent space. What is that? A latent space is a made up place where we are trying to organize data in a way that similar things are close to each other. In our earlier work we were looking to generate hundreds of variants of a material model to populate this scene. In this latent space, we can concoct all of these really cool digital material models. A link to this work is available in the video description. Stargan uses walks in a similar latent space to create these human faces and animate them. And now hold on to your papers because a latent space can represent not only materials or the head movement and smiles for people, but even better, age too. You remember these amazing transformations from the intro. So how does it do that? Well, similarly to the material example, we can embed the source image into a latent space and take a path therein. It looks like this. Please remember this embedding step because we are going to refer to it in a moment. And now comes the twist. The latent space for this new method is built such that when we take these walks, it disentangles age from other attributes. This means that only the age changes and nothing else changes. This is very challenging to pull off because normally when we change our location in the latent space, not just one thing changes, everything changes. This was the case with the materials, but not with this method which can take photos of well known celebrities and make them look younger or older. I kind of want to do this to myself too. So you know what? Now it's my turn. This is what Baby Karoj might look like after reading baby papers. And this is old man Karoj complaining that papers were way better back in his day. And this is supposedly baby Karoj from a talk at a NATO conference. Look, apparently they let anybody in these days. Now this is all well and good, but there is a price to be paid for this. So what is the price? Let's find out together what that is. Here is the reference image of me, and here is how the transformations came out. Did you find the issue? Well, the issue is that I don't really look like this. Not only because the beard was synthesized onto my face by an earlier AI, but really, I can't really find my exact image in this. Take another look. This is what the input image looked like. Can you find it in the output somewhere? Not really. Same with the conference image. This is the actual original image of me and this is the output of the AI. So I can't find myself. Why is that? Now, you remember I mentioned earlier that we embed the source image into the latent space and this step is unfortunately imperfect. We start out from not exactly the same image, but only something similar to it. This is the price to be paid for these amazing results. And with that, please remember to invoke the first law of papers which says do not look at where we are, look at where we will be. Two more papers down the line. Now, even better news as of the writing of this episode, you can try it yourself. Now, be warned that our club of fellow scholars is growing rapidly and you all are always so curious that we usually go over and crash these websites upon the publishing of these episodes. If that happens, please be patient. Otherwise, if you tried it, please let me know in the comments how it went or just tweet at me. I'd love to see some more baby scholars. What a time to be alive. This video has been supported by Weights and Biases. Check out their recent offering Fully Connected, a place where they bring machine learning practitioners together to share and discuss their ideas, learn from industry leaders, and even collaborate on projects together. You see, I get messages from you fellow scholars telling me that you have been inspired by the series but don't really know where to start. And here it is. Fully Connected is a great way to learn about the fundamentals, how to reproduce experiments, get your papers accepted to a conference and more. Make sure to visit them through WNB Me papers or just click the link in the video description. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
