Dr. Zsolnai walks through NVIDIA's StyleGAN research, which generates photorealistic human faces and transforms sketches into images with five editing modes: sketch-to-image, style transfer, material swaps, attribute editing like adding beards, and interpolation between faces. The tool is research for now, not a product, but shows how AI image generation is becoming easier to use.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. We are back and today we are going to marvel at the capabilities of Nvidia's incredible image generation AI. And we will see together that we can all become witchers. Previously, they were able to create an incredible AI that can take a piece of text from us and create an appropriate image. Or it could also give us a little more artistic control, as it could even transform a rough sketch into a beautiful output image. A variant generation was also possible. Or if we didn't feel like drawing, it could also take segmentation maps which specify which region should be what the output was. Again, a beautiful photorealistic image. So these were mostly landscapes. And what about human faces? Can we generate human faces too? And I wonder if we could also edit them. If so, how? Well, Nvidia already has incredible works for generating human faces. Look, this is their Stylegan 3AI. And as you see, my goodness, these are incredible high quality faces. And we even have a tiny bit of control over the images. Not a great deal, but a little control is there. It works on art pieces too, by the way. And this new technique finally gives us a little more creative freedom in editing these images in five different ways. How? Well, check this out. Oh yes. One, we start sketching and out comes an image of this quality. Wow. This truly is a testament to the power of AI these days. It doesn't just generate images out of thin air, but here we can really harness the power of this AI. And the best part is that we don't even need to be an expert at drawing. So cool. Two, if we don't feel like drawing, but we would really like to see some well known people as cubist paintings, this can do that too. Three, if we have an image with snow and we feel like chocolate, vanilla or cherry ice cream would be so much better for snowboarding. Also not a problem. And here comes my favorite. 4. We can make people into witchers. I think I might make a good one. Look, this is Obi Wan Karoi, a synthetic image of me made by an AI with an added beard. And look, you don't want to mess with witcher Karoj, do you? Hmm. So who else would make a great witcher? Well, here are some of my favorites. For instance, Grimes, Beardyman, Skrillex, Robin Williams, Obama and the Rock. All excellent witchers. And five, it can not only produce these images, but even interpolate between them. What does that mean? This means that not only the final image, but but intermediate images can also be generated. Look, we are going from a photo to a sketch to a Mona Lisa style painting. While we are looking at these wonderful results, I would like to send a huge thank you to the authors and they were very patient and took quite a bit of time their busy day to delight you fellow scholars with these animations. As a result, some of these animations you can only see here on 2 Minute Papers. That is a huge honor. Thank you so much. So how easy is it to use? Well, so easy that it only takes a minute to produce such excellent results. And it likely means that this will be an excellent tool in democratizing artistic image editing and giving it into the hands of everyone. Now make no mistake, this is a research paper, not a product yet. But Nvidia has an excellent record of transferring these works into real products. For instance, they published a similar audio to face paper in 2017 and now just a few years later, it is out there for everyone to use. How cool is that? I really hope that this work will also have a similar fate. What a time to be alive. So does this get your mind going? What would you use this for? Let me know in the comments below.
What you see here is a report of this exact paper we have talked about which was made by Weights and Biases. I put a link to it in the description. Make sure to have a look. I think it helps you understand this paper better. Weights and Biases provides tools to track your experiments in your deep learning projects. Using their system you can create beautiful reports like this one to explain your findings to your colleagues better. It is used by many prestigious labs including OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get.
A free demo today.
Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you.
Thanks for watching and for your generous support and I'll see you next time.
