NVIDIA's Image Restoration AI: Almost Perfect!

Dear fellow scholars, this is two minute papers with image denoising is an area where we have a noisy image as an input and we wish to get a clear noise free image. Neural network based solutions are amazing at this because we can feed them a large amount of training data with noisy inputs and clear outputs. And if we do that during the training process, the neural network will be able to learn the concept of noise and when presented with a new previously unseen noisy image, it will be able to clear it up. However, with light transport simulations, creating a noisy image means following the path of millions and millions of light rays, which can take up to hours per training sample and we need thousands or potentially hundreds of thousands of these. There are also other cases where creating the clean images for the training set is not just expensive, but but flat out impossible. Low light photography, astronomical imaging or magnetic resonance imaging, MRI in short are great examples of this. In these cases we cannot use our neural networks simply because we cannot build such a training set as we don't have access to the clear images. In this collaboration between Nvidia Aalto University and mit, scientists came up with an insane idea. Lets try to train a neural network without clear images and use only noisy data. Normally we would say that this is clearly impossible and end this research project. However, they show that under a suitable set of constraints, for instance, one reasonable assumption about the distribution of the noise opens up the possibility of restoring noisy signals without seeing clean ones. This is an insane idea that actually works and can help us restore images with significant outlier content. Not only that, but it is also shown that this technique can do close to or just as well as other previously known techniques that have access to clean images. You can look at these images, many of which have many different kinds of noise like camera noise, noise from light transport simulations, MRI imaging and images severely corrupted with a ton of random text. The usual limitations apply. In short, it of course cannot possibly recover content if we cut out a bigger region from our images. This severely hamstrung training process can be compared to a regular neural denoiser that has access to the clean images and the differences are negligible most of the time. So how about that we can teach a neural network to denoise without ever showing it. The concept of denoising just the thought of this boggles my mind so much it keeps me up at night. This is such a remarkable concept. I hope there will soon be follow up papers that extend this idea to other problems as well. As if you enjoyed this episode and you feel that about eight of these videos a month is worth a dollar. Please consider supporting us on Patreon. We use these funds to make better videos for you, and a small portion is also used to fund research conferences. You can find us at patreon.com twominutepapers and there is also a link to it in the video description. You know the drill. $1 is almost nothing, but it keeps the papers coming. Thanks for watching and for your generous support and I'll see you next time.