A two-stream neural network detects image tampering by analyzing color and noise patterns to spot splicing, copy-paste, and inpainting. The algorithm resists common compression tricks used to hide forgeries, though better detectors could inadvertently help train more convincing fakes.
Dear fellow scholars, this is two minute papers with AS facial reenactment videos are improving at a rapid pace. It is getting easier and easier to create video impersonations of other people by transferring our gestures onto their faces. We have recently discussed a technique that is able to localize the modified regions within these videos. However, this technique was limited to human facial reenactment. That is great, but what about the more general case with manipulated photos? Well, do not worry for a second. Because this new learning based algorithm can look at any image and highlight the regions that were tampered with, it can detect image splicing, which means that we take a part of a different image and add it to this one. Or copying an object and pasting it to the image elsewhere. Or removing an object from a photo and filling in the hole with meaningful information harvested from the image. This we also refer to as image inpainting. And this is something that we also use often to edit our thumbnail images that you see here on YouTube. Believe it or not, it can detect all of these cases and it uses a two stream convolutional neural network to accomplish this. So what does this mean exactly? This means a learning algorithm that looks at one the color data of the image to try to find unnatural contrast changes along edges and silhouettes, and 2 the noise information within the image as well and see how they relate to each other. Typically, if the image has been tampered with, either the noise or the color data is disturbed, or it may be that they look good one by one, but the relation of the two has changed. The algorithm is able to detect these anomalies too, as many of the images we see on the Internet are either resized or compressed or both. It is of utmost importance that the algorithm does not look at compression artifacts and thinks that the image has been tampered with. This is something that even humans struggle with on a regular basis, and this is luckily not the case with this algorithm. This is great because smart attackers may try to conceal their mistakes by recompressing an image and thereby adding more artifacts to it. It's not going to fool this algorithm. However, as you fellow scholars pointed out in the comments of a previous episode, if we have a neural network that is able to distinguish forged images with a little modification, we can perhaps turn it around and use it as a discriminator to help training a neural network that produces better forgeries. Hmm. What do you think about that? It is of utmost importance that we inform the public that these tools exist. If you wish to hear more about this topic. And you think that a bunch of videos like this a month is worth a dollar? Please consider supporting us on Patreon. You know the drill. A dollar a month is almost nothing. But but it keeps the papers coming. Also, for the price of a coffee, you get exclusive early access to every new episode we release. And there are even more perks on Our Patreon page patreon.com TwoMinutePapers we also support cryptocurrencies like Bitcoin, Ethereum, and Litecoin. The addresses are available in the video description. With your help, we can make better videos in the future. Thanks for watching and for your generous support, and I'll see you next time.
