Dr. Zsolnai walks through 10 features of Stable Diffusion v2, the free text-to-image model. It now handles higher resolution natively, generates photorealistic humans, creates depth maps from photos, and runs on consumer GPUs. You can try it online or locally.
Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Oh my. Here is stable diffusion version two. So what is this? Stable diffusion is a free and open source text to image AI, which means that we write a piece of text and it creates exactly that image for us. All this for everyone for free. So good. And now here are 10 things you should know about version 2. 1. It can now generate images with higher resolution. More details for free. That sounds amazing, but it doesn't stop there because two, it can also perform super resolution better. Super resolution means that in goes a coarse image and out comes a beautiful image with a lot more detail. And when I say a lot, I mean a lot. Just look at how much better this is. So good. Three, it can also go from depth plus text to image. What does that mean? Well, we can give it an input image and then it fires up this paper. Oh goodness, that is a powerful paper. It can not only estimate depth maps for photos really well, but get this, it can even imagine if a drawing were a real photo and estimate the depth of that. Incredible. So it creates the depth map for this image and using our prompt fills it with information that makes sense. This is excellent if we wish to just specify the pose of our hero and it will generate as many similar variations for it as we can imagine. Or we can also do this. Lots of amazing variants for the same concept. The limit is only our imagination loving it. 4. Image inpainting is now easier. This means that we can say that we wish to retain this part of the image and the AI should just fill in the rest according to our instructions. This is super fun and it will help with one of the most important things we could all ask for. And that is quick turnaround times for our ideas. 5. It can now generate more convincing images of refractive objects. You know that I am a light transport researcher by trade and this makes me very, very happy. Human eyes, water, underwater images, not a problem. And some of these images are extremely convincing. These are especially impressive because they came about from pure prompting. No guide or any input Images were used. 6. Now hold on to your papers for photorealistic humans.
Whoa.
These are shockingly good. Especially that we are also humans, most of us anyway, and we really have a keen eye for other humans. And if even the smallest things are off, we notice immediately. And the details in this version are finally at the point where it can create believable virtual characters. But if photorealism is not your cup of tea and you like virtual worlds better, do not despair. 7. It is also excellent at creating cyberpunk book covers. 8. We can also generate incredible interiors with it. Now, not all of them are perfect. There are some flaws here, but the pace of progress in AI research is nothing short of amazing. Not so long ago we needed proper light, simulation programs and hours and hours of work to create an interior like this. And now just a text prompt. Just a few seconds and just imagine what this will look like just two more papers down the line. What a time to be alive. Note that I put a link to the authors of these images in the video description, so make sure to check them out if you wish to see more of these. 9. The textures are also incredible. Here you don't see super fancy images, just simple concepts that are executed really well. I just kept on looking and looking at each of these images and I can barely find any flaws. 10. Of course, I can only imagine how excited you fellow scholars are to try it. So can you. Yes, and it gets better because you can try it in two different ways or one. If you are patient, you can write a prompt here. You might have to wait for a bit, but as of the making of this video, it works. Now what happens when you fellow scholars get over there? Who really knows? We have crashed plenty of websites before with our scholarly stampede. But if you don't want to wait or wish to run some more advanced experiments, you can run the model yourself at home or on a consumer graphics card. Loving it. And don't forget, stable diffusion is free and open source and thus it has captured the imagination of you fellow scholars. With this you can pop the hood, take out your virtual wrenches and let the experiments begin. And with this, AI based image generation is only getting cheaper and more democratized. From here on out, a little open source competition for OpenAI and Google. Power to the people and for free. Double thumbs up. What a time to be alive. By the way, Google already has an AI that creates not only images but videos from your prompts and if everything goes well, I may or may not have exclusive access to it. If it comes to fruition, there will be a video on it with my own scholarly prompts. So? So consider subscribing and hitting the bell icon. You really don't want to miss it if it comes.
This episode is brought to you by anyscale, the company behind Ray, the fastest growing open source framework for scalable AI and scalable Python. Thousands of organizations use Ray, including OpenAI, Uber, Amazon, Spotify, Netflix and more. Ray lets developers iterate faster by providing common infrastructure for scaling data, ingest and pre processing, machine learning training, deep learning, hyper parameter tuning, model serving, and more, all while integrating seamlessly with the rest of the machine learning ecosystem. Anyscale is a fully managed RAY platform that allows teams to bring products to market faster by eliminating the need to manage infrastructure and by enabling new AI capabilities.
Hooray.
And Anyscale can do recommendation systems, time series, forecasting, document understanding, image processing, industrial automation, and more. Go to anyscale.com papers and try it out today. Our thanks to Anyscale for helping us make better videos for you.
Thanks for watching and for your generous support and I'll see you next time.
