Dr. Zsolnai covers neural networks that age or de-age faces in photos using semantic diffusion. The technique handles fine-grained edits like adding glasses or changing expressions, and can morph between images while keeping every frame realistic, like turning a black-and-white Einstein into color.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are living the advent of neural network based image generation algorithms. What you see here is some super high quality results from a technique developed by scientists at Nvidia called Stylegan 2. Right. All of these were generated by a learning algorithm. And while generating images of this quality is a great achievement, but if we have an artistic vision, we wonder can we bend these images to our will? Can we control them? Well, kind of. And one of the methods that enables us to do that is called image interpolation. This means that we have a reference image for style and the target image. And with this we can morph one human face into another. This is sufficient for some use cases. However it if we are looking for more elaborate edits, we hit a wall. Now it's good that we already know what Stylegan is because this new work builds on top of that and shows exceptional image editing and interpolation abilities. Let's start with the image editing part. With this new work we can give anyone glasses and a smile or even better, transform them into a variant of the Mona Lisa. Beautiful. The authors of the paper call this process semantic diffusion. Now let's have a closer look at the expression and post change possibilities. I really like that we have fine grained control over these parameters. And what's even better, we don't just have a start and end point, but all the intermediate images make sense and can stand on their own. This is great for pose and expression because we can control how big of a smile we are looking for. Or even better, we can adjust the age of the test subject with remarkable granularity. Let's go all out. I like how Mr. Cumberbatch looks nearly the same as a baby. We might have a new mathematical definition for baby face right there. And apparently Mr. DiCaprio scores a bit lower on that. And I would say that both results are quite credible. Very cool. And now onto image interpolation. What does this new work bring to the table in this area? Previous techniques are also pretty good at morphing. Until we take a closer look at them, let's continue our journey with three interpolation examples with increasing difficulty. Lets see the easy one first. I was looking for a morphing example with long hair. You will see why right away. This is how the older method did. Uh oh, one more time. Do you see what I see? If I stop the process here, you see that this is an intermediate image that doesn't make sense. The hair over the forehead just suddenly vanishes. Into the ether. Now let's see how the new method deals with this issue. Wow, much cleaner. And I can stop nearly anywhere and leave the process with a usable image. Easy example checkmark. Now let's see an intermediate level example. Let's go from an old black and white Einstein photo to a recent picture with colors and stop the process at different points. And yes, I prefer the picture created with the new technique close to every single time. Do you agree? Let me know in the comments below. Intermediate example check mark. And now onwards to the hardest nastiest example. This is going to sound impossible, but we are going to transform the Eiffel Tower into the Tower Bridge. Yes, that sounds pretty much impossible. So let's see how the conventional interpolation technique did here. Well, that's not good. I would argue that nearly none of the images showcased here would be believable if we stopped the process and took them out. And let's see the new method. Hmm, that makes sense. We start with one tower, then two towers grow from the ground and look. Wow. The bridge slowly appears between them. That was incredible. While we look at some more results, what really happened here? At the risk of simplifying the contribution of this new paper, we can say that during interpolation it ensures that we remain within the same domain for the intermediate images intuitively. As a result, we get less nonsense in the outputs and can pull off morphing not not only between human faces, but even go from a black and white photo to a colored one. And what's more, it can even deal with completely different building types or you know, just transform people into Mona Lisa variants. Absolutely amazing. What a time to be alive. What you see here is a report of this exact paper we have talked.
About which was made by weights and biases.
I put a link to it in the description. Make sure to have a look. I think it helps you understand this paper better.
Weights and Biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money and it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub and more. And the best part is that if you have an open source, academic or personal project you you can use their tools for free. It really is as good as it gets. Make sure to Visit them through wnb.com.
Papers or click the link in the.
Video description to start tracking your experiments in five minutes. Our thanks to WeightsAndBiases for their long standing support and for helping us make.
Better videos for you. Thanks for watching and for your generous support, and I'll see you next time.
