Dr. Zsolnai reviews StyleFlow, an AI that edits faces with surgical precision. The tool can age someone up, change their expression, add facial hair, and adjust lighting in a single image. A demo shows it giving Elon Musk a majestic beard, though the background sometimes gets damaged in the process.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Here you see people that don't exist. They don't exist because these images were created with a neural network based learning method by the name Stylegan2, which can not only create eye poppingly detailed looking images, but it can also fuse these people together or generate cars, churches, horses, and of course, cats. This is quite convincing. So is there anything else to do in human face generation? Are we done? Well, this footage from a new paper may give some of it away. If you have been watching this series for a while, you know that of course researchers always find a way to make these techniques better. We always say two more papers down the line and it will be improved a great deal. And now we are one more paper down the line. So let's see together if there has been any improvement. This new technique is based on Stargan 2 and is called Style Flow and it can take an input photo of a test subject and edit a number of meaningful parameters. Age, expression, lighting, pose, you name it. Now note that there were other techniques that could pull this off, but the key improvement here is that one, we can perform many sequential changes. And two, it does all this while remaining faithful to the original photo. And hold on to your papers because three, it can also help Elon Musk to grow a majestic beard. And believe it or not, you will also see a run of this algorithm on me as well at the end of the video. First, let's adjust the lighting a little bit. And now it's time for that beard. Loving it. Now a little aging. And one more beard please. It seems to me that this beard is different from the young man beard, which is nice attention to detail. And note that we have strayed very far from the input image. But if you look at the intermediate states, you see that the essence of the test subject is indeed quite similar. This Elon is still Elon. Note that these results are not publicly available and were made specifically for us. So you can only see this here on two minute papers. That is quite an honor. So thank you very much to Ramin Abdaal, the first author of the paper, for being so kind. Now, another key improvement in this work is that we can change one of these parameters with little effect on anything else. Have a look at this workflow and see how well we can perform these sequential edits. These are the source images and the labels showcase one variable change for each subsequent image and you can see how amazingly surgical the changes are. Witchcraft. If it says that we changed the facial hair, that's the only change I see in the output. And just think about the fact that these starting images are real photos, but the outputs of the algorithm are made up people that don't exist. And notice that the background is also mostly left untouched. Hmm, why does that matter? You will see in the next example. So far so good, but this method can do way more than this. Now let's edit not people, but cars. I love how well the color and pose variations work. Now if you look here, you see that there is quite a bit of collateral damage as not only the cars but the background is also changing, opening up the door for a potentially more surgical follow up paper. Make sure to subscribe and hit the bell icon to get notified when we cover this. One more paper down the line. And now here's what you have been waiting for. Will get hands on experience with this technique where I shall be the subject of the next experiment. First, the algorithm is looking for high resolution frontal images. So for instance, this would not work at all. We would have to look for a different image. No matter. Here's another one where I got locked up for reading too many papers. This could be used as an input for the algorithm. And look, this image looks a little different. Why is that? It is because Stargan 2 runs an embedding operation on the photo before starting its work. This is an interesting detail that we only see if we start using the technique ourselves. And now come on, give me that beard. Ho ho ho. Loving it. What do you think? Whose beard is better? Elon's or mine? Let me know in the comments below. And now please meet old man Caroj. He will tell you that papers were way better back in his day and the usual transformations. But also note that as a limitation we had quite a bit of collateral damage for the background. This was quite an experience. Thank you so much. And we are not done yet because this paper just keeps on giving. It can also perform attribute transfer. What this means is that we have two input photos. This will be the source image and we can choose a bunch of parameters that we would like to extract from it. For instance, the lighting and pose can be extracted through attribute transfer and it seems to even estimate the age of the target subject and change the source to match it better. Loving it. The source code for this technique is also available. And make sure to have a look at the paper. It is very thoroughly evaluated. Evaluated. The authors went the extra mile there and it really shows. And I hope that you now agree, even if there is a technique that appears quite mature, researchers always find a way to further improve it. And who knew it took a competent AI to get me to grow a beard. Totally worth it. What a time to be alive. This episode has been supported by Weights and Biases. In this post they show you how to how to use sweeps to automate hyperparameter optimization and explore the space of possible models and find the best one. During my PhD studies I trained a ton of neural networks which were used in our experiments. However, over time there was just too much data in our repositories and what I am looking for is not data but insight. And that's exactly how Weights and Biases helps you by organizing your experiments. It is used by more than 200 companies and research institutions including OpenAI, Toyota Research, GitHub and more. And get this Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnba.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
