Researchers at ETH Zurich built an AI that transforms photos from cheap phone cameras into DSLR-quality images. It learns from paired photos to do heavy lifting: super-resolution, inpainting, deblurring, denoising, and color recovery. They released the code, trained models, and a demo anyone can try.
Dear fellow scholars, this is two minute Papers with zsolnai. Some time ago smartphone cameras were trying to outpace each other by adding more and more megapixels to their specification sheet. The difference between a half megapixel image and a 4 megapixel image was night and day. However, nowadays we have entered into diminishing returns as most newer mobile cameras support eight or more megapixels. At this point, a further resolution increase doesn't lead to significantly more convincing photos. And here is where the processing software takes the spotlight. This paper is about an AI based technique that takes a poor quality photo and automatically enhances it. Here you can already see what a difference software can make to these photos. Many of these photos were taken with an 8 year old mobile camera and were enhanced by the AI. This is insanity. Now before anyone thinks that by enhancement I'm referring to the classic workflow of adjusting white balance, color levels and hues. No, no no. By enhancement I mean the big heavy hitters like recreating lost details via super resolution and image inpainting, image deblurring, denoising and recovering colors that were not even recorded by the camera. The idea is the first we shoot a lot of photos from the same viewpoint with a bunch of cameras ranging from a relatively dated iPhone3GS, other mid tier mobile cameras and a state of the art DSLR camera. Then we hand over this huge bunch of data to a neural network that learns the typical features that are preserved by the better cameras and lost by the worst ones. The network does the same with relating the noise patterns and color profiles to each other. And then we we use this network to recover these lost features and pump up the quality of our lower tier camera to be as close as possible to a much more expensive model. Super smart idea. Loving it. And you know what is even more brilliant? The validation of this work can take place in a scientific manner because we don't need to take a group of photographers who will twirl their mustaches and judge these photos. Though I'll note that this was also done for good measure. But since we have the photos from the high quality DSLR camera, we can take the bad photos, enhance them with the AI and compare this output to the real DSLR's output. Absolutely brilliant. The source code and pre trained networks and an online demo is also available. So let the experiments begin and make sure to leave a comment with your findings. What do you think about the outputs shown in the website? Did you try your own photo? Let me know in the comments section a high quality validation section, lots of results, candid discussion of the limitations in the paper, published source code, pre trained network and online demos that everyone can try free of charge. Scientists at ETH Zurich maxed this paper out. This is as good as it gets. If you have enjoyed this episode and would like to help us make better videos in the future, please consider supporting us on Patreon by clicking the letter P at the end screen of this video in a moment or just have a look at the video description. Thanks for watching and for your generous support and I'll see you next time.
