An AI Made All of These Faces! 🕵️‍♀️

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today I am going to try to tell you the glorious tale of AI based human face generation and showcase an absolutely unbelievable new paper in this area. Early in this series we covered a stunning paper that showcased a system that could not only classify an image, but write a proper sentence on what is going on and could cover even highly non trivial cases. You may be surprised, but this thing is not recent at all. This is four year old news. Insanity. Later, researchers turned this whole problem around and performed something that was previously thought to be impossible. They started using these networks to generate photorealistic images from a written text description. We could create new bird species by specifying that it should have orange legs and and a short yellow bill. Then researchers at Nvidia recognized and addressed two shortcomings. One was that the images were not that detailed. And two, even though we could input text, we couldn't exert too much artistic control over the results. In came Stylegan to the rescue, which was then able to perform both of these difficult tasks really well. Furthermore, there are some features that are highly localized as we exert control over these images. And you can see how this part of the teeth and eyes were pinned to a particular location and the algorithm just refuses to let it go, sometimes to the detriment of its surroundings. A follow up work titled Stylegan2 addresses all of these problems in one go. So Stylegan2 was able to perform near perfect synthesis of human faces. And remember, none of these people that you see here really exist. Quite remarkable. So how can we improve this magnificent technique? Well, this new work can do so many things, I don't even know where to start. First and most important, we now have much more intuitive artistic control over the output images. We can add or remove a beard, make the subject younger or older, change their hairstyle, make their hairline recede, put a smile on their face, or even make their nose pointier. Absolute witchcraft. So why can we do all this with this new method? The key idea is that it is not using a generative adversarial network. Again, in short, again means two competing neural networks where one is trained to generate new images and the other one is used to tell whether the generated images are real or fake. Gans dominated this field for a long while because of their powerful generation capabilities. But on the other hand, they are quite difficult to train and we only have limited control over its output. Among other changes, this work disassembles the generator network into F and G and the discriminator network into E and D or in other words, adds an encoder and decoder network here. Why? The key idea is that the encoder compresses the image data down into a representation that we can edit more easily. This is the land of beards and smiles. Or in other words, all of these intuitive features that we can edit exist here and when we are done, we can decompress the output with a decoder network and produce these beautiful images. This is already incredible, but what else can we do with this new architecture? A lot more. For instance too, if we add a source and destination subjects, their coarse, middle or fine styles can also be mixed. What does that mean exactly? The coarse part means that high level attributes like pose, hairstyle and face shape will resemble the source subject. In other words, the child will remain a child and inherit some of the properties of the destination people. However, as we transition to the find from source part, the effect of the destination subject will be stronger and the source will only be used to change the color scheme and microstructure of this image. Interestingly, it also changes the background of the subject. 3 It can also perform image interpolation. This means that we have these four images as starting points and it can compute intermediate images between them. You see here that as we slowly become Bill Gates, somewhere along the way, glasses appear. Now note that interpolating between images is not difficult in the slightest and has been possible for a long, long time. All we need to do is just compute average results between these images. So what makes a good interpolation process? Well, we are talking about good interpolation when each of the intermediate images makes sense and can stand on their own. I think this technique does amazingly well at that. I'll stop the process at different places. You can see for yourself and let me know in the comments if you agree or not. I also kindly thank the authors for creating more footage just for us to showcase in this series. That is a huge honor. Thank you so much. And note that Stylegan 2 appeared around December of 2019 and now this paper by the name Adversarial Latent Autoencoders appeared only four months later. Four months later. My goodness, this is so much progress in so little time. It truly makes my head spin. What a time to be alive. What you see here is an instrumentation of this exact paper we have talked about which was made by weights and biases. I think organizing these experiments really showcases the usability of their system. Weights and Biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money and it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub, and more. And the best part is that if you have an open source, academic or personal project, you can use their tools for free. It really is as good as it gets. Make sure to Visit them through wnb.com papers or just click the link in the video description to start tracking your experiments in five minutes. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.