This content was created by LARRUZ. This content was uploaded by LARRUZ. This is a YouTube video titled "Google’s New AI: Blurry Photos No More!" (Two Minute Papers with Dr. Zsolnai) published July 8, 2023. It is a single-speaker explanatory video in which Dr. Zsolnai describes and demonstrates a new Google AI technique for refocusing and deblurring smartphone photos using multiple lenses, contrasts it with light fields, focus stacking, and prior work (e.g., DRB Net), praises its device-agnostic advantages and use of underused ultra-wide lenses, and notes sponsorship by Weights & Biases.
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to end the rule of ugly blurry photos with a new AI technique developed by Google. You see, most smartphones today have multiple cameras and can take these beautiful portrait mode photos where the subject is in focus and the rest of the image is blurred out. This is all well and good if we have shot the image perfectly and we have in focus exactly the things that we want before pressing the button. If we missed our chance, unfortunately, that's it. Refocusing these images would be amazing if we could do it. So can we? Well, unfortunately we can. Now I hear you asking doc, why do you say unfortunately we can? Well, we have to use light fields to properly refocus an image. This unfortunately requires an expensive and inconvenient array of cameras to capture all the required information in and out of focus. Or we can try focus stacking, that is capturing a series of images with different focus points, focusing on different things in different photos. This is not expensive, but unfortunately it is time consuming and not trivial to perform. But wait a minute, I just said at the start of the video that most smartphone brands today have multiple cameras. That typically means a wide and an ultra wide lens. If we take a photo with both of these lenses at the same time, we get two images with different parts of the real scene in focus. If you think that sounds reassuring, it gets worse. Look, the two lenses give quite different results. The colors are different, the focus is different. Hmm. Perhaps a learning based algorithm could stitch these two together into an image that we can refocus. But it looks quite difficult. And today, fellow scholars, fortunately that is a possibility. This is the result from a work called DRB Net from just one year ago. This is supposed to deblur the whole image and give us a sharp result. Well, it could have been a lot better. Look here. Not too great. But now hold on to your papers, fellow scholars, and have a look at the new technique. So much better. And when we try the old technique for refocusing an image, we we get this. But with the new technique we get this instead. That is a huge difference. Just one more paper down the line. Especially that the previous work was published just a year ago and with this new work we can now refocus all of these beautiful images. And even though none of them are as perfect as having a real camera focusing at all these objects, I find these results to be absolutely incredible. Not perfect by any means, but remember, the two lenses are really different. This technique is like a classical music conductor having two different orchestras playing together in harmony. Neither of the orchestras are perfect by themselves because they are both missing some instruments, but together, if they're acted well, they can play the perfect symphony. You can now take in this symphony not with your ears, but with your eyes. Now two more things that I absolutely loved about this paper. One is that it describes a method that is not specific to a particular device and 2 these ultra wide lenses are often of lower quality than the other lenses in smartphone cameras and are typically underused and overlooked. But not here. I love it. So from now on, unusable blurry images of important family moments shall be banished forever. What a time to be alive. This video has been supported by weights and biases. They have an amazing podcast by the name Gradient Descent where they interview machine learning experts who discuss how they use learning based algorithms to solve real world problems. They've discussed biology, teaching robots, machine learning in outer space, and a whole lot more. Perfect for a fellow scholar with an open mind. Make sure to visit them through WNB MEGD or just click the link in the video description. Our thanks to WeightsAndBiases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
