Stable Diffusion Is Getting Outrageously Good! 🤯

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today you will see the power of human ingenuity supercharged by an AI. As we are leaving the advent of AI based image generation, we now have several tools that are so easy to use. We just enter a piece of text and out comes a beautiful image. Now you are asking. Okay Caroj, they are easy to use, but for whom? Well, good news. We now have a new solution called Stable diffusion where the model weights and the full source code are available for free for everyone. Finally, we talked a bit about this before and once again I cannot overstate how amazing this is. I am completely spellbound by how the community has worked together to bring this project to life. And you fellow scholars just kept on improving it and it harnesses the greatest asset of humanity and that is the power of the community working together. And I cannot believe how much stable diffusion has improved in just the last few weeks. Don't believe it. Well, let's have a look together at 10amazing examples of of how the community is already using it. 1. Today the image generation works so inexpensively that we don't even need to necessarily generate our own. We can even look at this amazing repository where we entered the prompt and can find thousands and thousands of generated images for that concept. Yes, even for Napoleon cats, we have thousands of hits. So good. Now additionally, we can also add a twist to it by photographing something in real life, obtaining a text prompt for it, and bam. It finds similar images that were synthesized by stable diffusion. This is variant generation of sorts, but piggybacking on images that have been synthesized already. Therefore, we can choose from a large gallery of these works. 2. By using a little trickery and the image inpainting feature, we can now create these amazing infinite zoom images. So good. 3. Whenever we build something really cool with Legos, we can now ask Stable Diffusion to reimagine what it would look like if it were a real object. The results are by no means perfect, but based on what it comes up with, it really seems to understand what is being built here and what its real counterpart would look like. I love it. 4. After generating a flat 2D image with stable diffusion with other techniques, we can obtain a depth map which describes how far different objects are from the camera. Now that is something that we've seen before. However, now in Adobe's After Effects look, we can create this little video with a parallax effect. Absolutely incredible. 5. Have a look at this Cat Knight. I love the eyes and all of these gorgeous details. On the armor. This image really tells a story. But what is even better is that not only the prompt is available, but also stable diffusion is a free and open source model. So we can pop the hood, reuse the same parameters as the author and and get a reproduction of the very same image. And it is also much easier to edit it this way if we wish to see anything changed. 6. If we are not the most skilled artist, we can draw a really rudimentary owl, hand it to the AI and it will draw the rest of this fine owl. 7. And if you think the drawing to image example was amazing, now hold on to your papers for this one. This fellow scholar had a crazy idea. Look, these screenshots of old Sierra video games were given to the algorithm and there is no way, right? Well, let's see. Oh wow, look at that. The results are absolutely incredible. I love how closely it follows the framing and the mood of the original photos. I have to be honest, some of these feel good to go as they are. What a time to be alive. 8. With these new web apps, variant generation is now way easier and faster than before. It is now as simple as dropping in an image. By the way, a link to each of these is available in the video description and their source code is available as well. 9. In an earlier episode we had a look at how artists are already using Dall E2 in the industry to make a photo of something and miraculously extend it almost infinitely. This is called texture synthesis and no seams anywhere to be seen. And now dear fellow scholars, seamless texture generation is now possible in in stable diffusion too. Not too many years ago, we needed not only a proper handcrafted computer graphics algorithm to even have a fighting chance to create something like this, but implementing a bunch of these techniques was also required because different algorithms did well on different examples. And now just one tool that can do it all. How cool is that? And 10 stable diffusion itself is also being improved. Oh yes, this new version adds super resolution to the mix which enables us to synthesize even more details and even higher resolution images with it. This thing is improving so quickly we can barely keep up with it. So which one was your favorite? Let me know in the comments below. And once again, this is my favorite type of work which is free and open for everyone. So I would like you fellow scholars to also take out your digital wrenches and create something new and amazing. Let the experiments begin. This episode is brought to you by Anyscale, the company behind Ray, the fastest growing open source framework for scalable AI and scalable power Python. Thousands of organizations use Ray, including OpenAI, Uber, Amazon, Spotify, Netflix and more. Ray lets developers iterate faster by providing common infrastructure for scaling, data ingest and pre processing, machine learning, training, deep learning, hyper parameter tuning, model serving, and more, all while integrating seamlessly with the rest of the machine learning ecosystem. Anyscale is a fully managed RAY platform that allows teams to bring products to market faster by eliminating the need to manage infrastructure and by enabling new AI capabilities. RAY and Anyscale can do recommendation systems, time series, forecasting, document understanding, image processing, industrial automation, and more. Go to anyscale.com papers and try it out today. Our thanks to Anyscale for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.