Dr. Zsolnai walks through a new AI super-resolution system trained on 20 million images with 100,000 negative prompts. You can upscale photos and control details with text prompts, like adding a mustache or changing materials. The results look sharp across faces, animals, and landscapes, though eyes and teeth sometimes still look off.
Questions this video answers
20 million images.
100,000 negative prompts.
February 13, 2024.
Dr. Zsolnai.
Chapters
And this new AI system has looked at 20 million images. So what has it learned? It learned to do something meaningful with these unusable super pixelated images. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. So the goal here would be that a pixelated image goes in and out comes what? Well, out comes an image with more details. This is what we call super resolution. And this is what the previous technique was capable of. Hmm. The result is fair. It is a little less blurry, but I don't feel that it has added too much detail to the image. But a newer technique from 2023, from less than a year ago. Wow. Now we're talking. Tons of detail just appeared. The feathers look great, but the eye. Oooh, my goodness. That eye is a bit creepy. Not a fan. And now let's see if the new technique can do even better. Oh my, look at that. That looks fantastic. If you thought the feathers were good before, look at them now. And finally, the eyes are completely different and, and much more realistic. And if we run the same for a person, previous techniques aren't even able to recognize that this should be a person. But the new technique. Oh, that is incredible. So what other incredible things is it capable of? Well, let's see together through six examples. One, have a look at this car. We have an unusably pixelated image here of a car and we know it's good. So it's going to get the car, right? I am sure. But are you thinking what I am thinking? Because I am thinking license plate. Can it do anything with the license plate? Not a chance. Right? Let's see. And whoa, look, we got something. Now I will immediately say that there is so little information in the input image that this new data might very well be completely made up. If not, bravo. But even if it is, I love the fact that the AI knows that this is roughly the kind of information that belongs there. Really cool. Two, how about trying to remix old video game graphics? That works too. And if we could do that in real time, we might not even need to store all of these detailed textures, just small coarse ones. If we know that, we can super resolve it to something way better in real time. 3. Landscapes. It not only gives us the clouds up there, but also the trees in the background in considerable detail. And if it can create so many new details, I am thinking about just drawing up a really crude draft of an image and have the super resolution AI do the heavy lifting. That would be so good. Also not far away at all for animals. Just look at that, in the area here we get a really sharp image. We can see the fur almost down to the individual strands. And in the background we even have a subtle depth of field effect as it goes out of focus. Fantastic. Five human faces. Now this one puts up a clinic on how to do this. Wow. Here it is. Even better. We get a result down to the individual level of hair strands. Some of them in the foreground look a little plasticky. And the teeth are not great either. But everything else, unbelievable. Just think about having the Internet using images and videos that are compressed down to take a hundred times less space. Because every device will be capable of performing this kind of super resolution soon. And six vintage photos and movies. This one has tons of those signature compression artifacts. And the image is also probably zoomed in, almost unusable. I don't think I would want to watch this video. And now hold on to your papers, fellow scholars. And look at this. Holy mother of papers. I am just looking and looking and. And I can't believe that this is possible. Interestingly, the paper discusses that the AI has also learned from 100,000 negative prompts. What does that mean? It means that it was given counterexamples, badly done images to also teach it not only what to do, but also what not to do. And this is a good paper. So they also tested how much of a difference this makes. So let's see. Goodness, it makes a huge difference. But we are not done yet because we haven't even talked about my favorite feature and that is text prompts. Wait, what can text prompts possibly do here? We already have an image. Just repaint the image, right? Why do we need text? You see, this is great. If we invest an additional minute to describe what is supposed to be in the scene and it creates these absolutely incredible results. We can tell it that. Look, this should be a bicycle. And now it is. But it gets better. Two, we can even use the text prompt to bend reality itself. Yes, in this case, we can even specify that the woman's hat should be made of suede or denim. That's real good. And I wonder if we can push this even further. Oh yes, yes we can. We can easily add a mustache. And I cannot tell that this has been added. I don't think anyone can. Or we can choose if our subject should be resolved to a young or an old man bending reality. What a time to be alive. And good news. An online demo of it is either coming or or will be already available by the time this video appears. So hopefully let the experiments begin. When I was a PhD student, my experiments created lots of data for my deep learning projects. But I was not looking for more data, I was looking for insights. This is what weights and biases does and it is the best everyone is using it. Try it out now at WNB Me Papers or click the link in the description below.
