A Two Minute Papers YouTube episode (Nov 12, 2021) by Dr. Károly Zsolnai-Fehér presenting a Google Brain super-resolution method that iteratively refines very low-resolution inputs (examples down to 64×64 px) to synthesize convincing high-resolution faces. The video compares the new approach to prior regression-based methods, highlights user-study results (prior methods ~33% fooled; the new method ~50%), notes limitations at higher output resolutions, and includes a Lambda GPU Cloud sponsorship listing Quadro RTX 6000, RTX 8000, V100 instances and a 48 GB RTX 8000 offering.
Questions this video answers
64×64 pixels.
About 33%.
About 50% (near the 50% mark).
Quadro RTX 6000, RTX 8000, and V100 instances.
48 gigabyte RTX 8000.
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to grow people out of noise of all things. So I hear you asking what is going on here? Well, what this work performs is something that we call super resolution. What is that simple de enhanced thing? Have a look at this technique from last year. In goes a coarse image or video and this AI based method is tasked with this. Yes, this is not science fiction, this is super resolution which means that the AI synthesized crisp details onto the image. Now fast forward a year later and let's see what this new paper from scientists at Google Brain is capable of. First, a hallmark of a good technique is when we can give it a really coarse input and it can still do something with it. In this case, this image will be 64 by 64 pixels, which is almost nothing I'm afraid. And let's see how it fares. This will not be easy and well, the initial results are not good. But don't put too much of a stake in the initial results because this work iteratively refines this noise, which means that you should hold on to your papers and oh yes, it means that it improves over time. It's getting there. Whoa, still going. And wow, I can hardly believe what has happened here. In each case in goes a really coarse input image where we get so little information. Look, the eye color is often given by only a couple pixels and we get a really crisp and believable output. What's more, it can even deal with glasses too. Now of course this is not the first paper on super resolution. What's more, it is not even the hundredth paper performing super resolution. So comparing to previous works is vital. Here we will compare this to previous methods in two different ways. One, of course we are going to look previous regression based methods perform reasonably well. However, if we take a closer look, we see that the images are a little blurry, high frequency details are missing. And now let's see if the new method can do any better. Well, this looks great, but we are fellow scholars here. We know that we can only evaluate this result in the presence of the true image. Now let's see. Nice. We would have to zoom in real close to find out that the two images are not the same. Fantastic. Now while we are looking at these very convincing high resolution outputs, please note that we are only really scratching the surface here. The heart and soul of a good super resolution paper is proper evaluation and user studies. And the paper contains a ton more details on that. For instance, this part of the study shows how likely people were to confuse the synthesized images with real ones. Previous methods, especially Pause, which is an amazing technique, reached about 33%, which means that most of the time people found out the trick. But whoa, look here, the new method is almost at the 50% mark. This is the very first time that I see a super resolution technique where people can barely tell that these images are synthetic. We are getting one step closer to this technique getting deployed in real world products. It could improve the quality of your zoom meetings, video games, online images and much, much more. Now note that not even this one is perfect. Look, as we increase the resolution of the output of the image, the users are more likely to find out that these are synthetic images. But still, for now, this is an amazing leap forward in just one paper. I can hardly believe that we can take this image and make it into this image using a learning based method. What a time to be alive.
This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI, check out Lambda GPU Cloud. They recently launched Quadro RTX 6000, RTX 8000 and V100 instances and hold onto your papers. Because Lambda GPU Cloud can cost less than half of AWS and Azure plus they are the only cloud service with 48 gigabyte RTX 8000. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Our thanks to Lamda for their long standing support and for helping us make.
