A Two Minute Papers YouTube video (narrated by Károly Zsolnai-Fehér) that summarizes NVIDIA’s StyleGAN2 advances over StyleGAN1. The presenter explains prior progress in generative neural networks (text→image, progressive growing), the limitations of StyleGAN1 (artifacts, localized feature sticking), and highlights StyleGAN2’s architectural simplifications and improved image quality across faces, cars, churches, horses, and cats. The video notes that the paper includes further results, the code runs in-browser, and that Weights & Biases sponsors the episode.
Chapters
Dear fellow scholars, this is two minute papers with neural network based learning algorithms are on the rise these days and even though it is common knowledge that they are capable of image classification, or in other words, looking at an image and saying whether it depicts a dog or a cat, nowadays they can do much, much more. In this series we covered a stunning paper that showcased the system by that could not only classify an image, but write a proper sentence on what is going on and could cover even highly non trivial cases. You may be surprised, but this thing is not recent at all. This is four year old news. Insanity. Later, researchers turned this whole problem around and performed something that was previously thought to be impossible. They started using these networks to generate photorealistic images from a written text description. We could create new bird species by specifying that it should have orange legs and a short yellow bill. Later, researchers at Nvidia recognized and addressed two shortcomings. One was that the images were not that detailed and two, even though we could input text, we couldn't exert too much artistic control over the results. In came Stylegan to the rescue, which was able to perform both of these difficult tasks really well. These images were progressively grown, which means that we started out with a coarse image and go over it over and over again, adding new details. This is what the results look like and we can marvel at the fact that none of these people are real. However, some of these images were still contaminated by unwanted artifacts. Furthermore, there are some features that are highly localized as we exert control over these images. You can see how this part of the teeth and eyes are pinned to a particular position and the algorithm just refuses to let it go, sometimes to the detriment of its surroundings. This new work is titled Stylegan2 and it addresses all of these problems in one go. Perhaps this is the only place on the Internet where we can say that finally teeth and eyes are now allowed to float around freely and mean it with a positive sentiment. Here you see a few hand picked examples from the best ones and I have to say these are eye poppingly detailed and correct looking images. My goodness. The mixing examples you see here are also outstanding. Way better than the previous version. Also note that as there are plenty of training images out there for many other things beyond human faces, it can also generate cars, churches, horses, and of course cats. Now that the original Stylegan 1 work has been out for a while, we have a little more clarity and understanding as to how it does what it does and the redundant parts of the architecture have been revised and simplified. This clarity comes with additional advantages beyond faster and higher quality training and and image generation, for instance. Interestingly, despite the fact that the quality has improved significantly, images made with the new method can be detected more easily. Note that the paper does much, much more than this, so make sure to have a look in the video Description in this series we always say that two more papers down the line and this technique will be leaps and bounds beyond the first iteration. Well, here we are not two, but but only one more paper down the line. What a time to be alive. The source code of this project is also available. What's more, it even runs in your browser. This episode has been supported by Weights and Biases. Weights and Biases provides tools to track your experiments in your deep learning projects. It can save you a ton of time and money in these projects and is being used by OpenAI, Toyota Research, Stanford and and Berkeley. Here you see a beautiful final report on one of their projects on classifying parts of street images and see how these learning algorithms evolve over time. Make sure to Visit them through wendb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
