Károly Zsolnai-Fehér explains how AI detects deepfakes by training neural networks on paired original and manipulated videos. The detector can pinpoint tampered facial regions with heatmaps and sometimes even improve forgeries, though video compression makes detection harder for both humans and machines.
Over a thousand videos.
Dear fellow scholars, this is 2 Minute Papers. With the recent ascendancy of several new AI based techniques for human facial reenactment, we are now able to create videos where we transfer our gestures onto famous actors or politicians and impersonate them. Clearly, as this only needs a few minutes of video as training data from the target, this could be super useful for animating photorealistic characters for video games and movies, reviving legendary actors who are not with us anymore, and much more. And understandably, some are worried about the social implications of such a powerful tool. In other words, if there are tools to create forgery, there should be tools to detect forgery, right? If we can train an AI to impersonate, why not train another AI to detect impersonation? This has to be an arms race. However, this is no easy task to say the least. As an example, look here. Some of these faces are real, some are fake. What do you think? Which is which? I will have to admit my guesses weren't all that great. But what about you? Let me know in the comment section. Compression is also an issue. Since all videos you see here on YouTube are compressed in some way to reduce file size. Some of the artifacts that appear may easily throw off not only an AI, but a human as well. I bet there will be many completely authentic videos that will be thought of as fakes by humans in the near future. So how do we solve these problems? First, to obtain a neural network based solution, we need a large data set to train it on. This paper contains a useful dataset with over a thousand videos that we can use to train such a neural network. These records contain pairs of original and manipulated videos along with the input footage of the gestures that were transferred. After the training step, the algorithm will be able to pick up on the smallest changes around the face and tell a forged footage from a real one. Even in cases where we humans are unable to do that. This is really amazing. These green to red colors showcase regions that the AI thinks were tampered with. And it is correct. Interestingly, this can not only identify regions that are forgeries, but it can also improve these forgeries too. I wonder if it can detect footage that it has improved itself. What do you think? Thanks for watching and for your generous support, and I'll see you next time.
