Dr. Zsolnai explains how researchers built a detector that identifies images made by neural generators like StyleGAN and BigGAN. Trained only on ProGAN fakes versus real photos, it generalizes to catch outputs from many other generators, even after compression.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we have an abundance of neural network based image generation techniques. Every image that you see here and throughout this video is generated by one of these learning based methods. These can offer high fidelity synthesis, and not only that, but we can even exert artistic control over the outputs. We can truly do so much with these. And if you're wondering, there is a reason why we will be talking about an exact set of techniques and you will see that in a moment. So the first one is a very capable technique by the name Cyclegan. This was great at image translation, or in other words, transforming apples into oranges, zebras into horses, and more. It was called Cyclegan because it introduced a cycle consistency loss function. This means that if we convert a summer image to a winter image and then back to a summer image, we should get the same input image back. If our learning system obeys to this principle, the output quality of the translation is going to be significantly better. Later, a technique by the name Biggan appeared, which was able to create reasonably high quality images. And not only that, but it also gave us a little artistic control over the outputs. After that, Stylegan and even its second version appeared, which, among many other crazy good features, opened up the possibility to lock in several aspects of these images. For instance, age, pose some facial features and more. And then we could mix them with other images to our liking, while retaining these locked in aspects. And of course, deepfake creation provides fertile grounds for research works for so much so that at this point it seems to be a subfield of its own, where the rate of progress is just stunning. Now that we can generate arbitrarily many beautiful images with these learning algorithms, they will inevitably appear in many corners of the Internet. So an important new question. Can we detect if an image was made by these methods? This new paper argues that the answer is a resounding yes. You see a bunch of synthetic images above and real images below here, and if you look carefully for the labels, you'll see many names that ring a bell to our scholarly Cyclegan, Biggan. Stylegan. Nice. And now you know that this is exactly why we briefly went through what these techniques do at the start of the video. So all of these can be detected by this new method. And now hold on to your papers, because I kind of expected that. But what I didn't expect is that this detector was trained on only one of these techniques, and leaning on that knowledge, it was able to catch all the others. Now that's incredible. This means that there are foundational elements that bind together all of these techniques. Our seasoned fellow scholars know that this similarity is none other than than the fact that they are all built on convolutional neural networks. They are vastly different, but they use very similar building blocks. Imagine the convolutional layers as Lego pieces and think of the techniques themselves to be the objects that we build. Using them we can build anything. But what binds these all together is that they are all but a collection of Lego pieces. So this detector was only trained on real images and synthetic ones created by the Progan technique. And you see with the blue bars that the detection ratio is quite close to perfect for a number of techniques, save for these two. The AP label means average precision. If you look at the paper in the description you will get a lot more insights as to how robust it is against compression artifacts, a little frequency analysis of the different synthesis techniques, and more. Let's send a huge thank you to the authors of the paper who who also provide the source code and training data for this technique. For now we can all breathe a sigh of relief that there are proper detection tools that we can train ourselves at home. In fact, you will see such an example in a second. What a time to be alive. Also good news, we now have an unofficial Discord server where all of you fellow scholars are welcome to discuss ideas and learn together in a kind and respectful environment. Look, some connections and discussions are already being made. Thank you so much for our volunteering fellow scholars for making this happen. The link is available in the video description. It is completely free and if you have joined make sure to leave a short introduction. Meanwhile, what you see here is an instrumentation of this exact paper we have talked about which was made by Weights and Biases. Weights and Biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money and it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub and more. And the best part is that if you are an academic or have an open source project, you can use their tools for free. It is really as good as it gets. Make sure to Visit them through wnb.com papers or just click the link in the video description and and you can get a free demo today. Our thanks to WeightsAndBiases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
