YouTube explainer by Dr. Caro Zsolnai (Two Minute Papers) introducing and reacting to the first results of Stable Diffusion 3 — a free, open-source text-to-image model. The speaker highlights three main improvements (better text rendering within images; improved understanding of complex prompt structure; enhanced creative scene synthesis), compares SD3 to DALL·E 3 and prior Stable Diffusion releases, and notes model size estimates and likely device/runtime implications. The video also mentions related tools (Stability API, StableLM) and upcoming coverage of other models and cloud GPU options.
Chapters
Here we always talk about these amazing results of recent AI techniques like this. This is Sora, but it is currently unreleased. That means that we can marvel at the results, but we cannot try them yet. However. Oh my. The first results of stable diffusion 3 are now available for us to look at. What is that? Stable Diffusion is a free and open source, open model text to image AI that we can all use for free. And interestingly, I also hear that version three builds on Sora's architecture. I'd love to see that. Previously we talked about a version called Stable Diffusion XL Turbo and it was extremely fast. So fast that we don't even measure it in frames per second. No sir, Cats per second is where it's at. And this could generate a hundred cats per second per second. That is fantastic. However, the quality of the CATS was not as good as what I saw in other systems like Dall? E3. So can we finally get a free and open system that creates super high quality images? Well, let's have a look together. Dear fellow scholars, this is Two Minute Papers with Dr. Caro Zsolnai. Well, first, the quality and the amount of detail in these images is absolutely incredible. But it gets better in three different ways. 1. Text. Remember the times when we told Dolly that we need a sign that says deep learning and we got this. Well, those days are still not over. We are still not out of the water. Current systems do much better on text, but they can only do short rudimentary prompts and. And we often have to run it 10 or more times to get something meaningful. This is Dall E3 trying the same and we are still not there. But here, would you look at that. We get some text on the chalkboard. Or look at this. This is not just text slapped on top of an image. It is an integral part of the image itself. It also knows styles quite well. This could easily be a desktop background for many. And graffiti styles are also appearing now. Not all the text on this image seems perfect and we don't know how much cherry picking was necessary to get these. But we will soon be able to try it ourselves and then we will know 2. Understanding prompt structure. This is going to be really tough. The prompt is three transparent glass bottles on a wooden table. The one on the left has a red liquid and the number one. The middle has a blue liquid and the number two. And the one on the right has a green liquid and the number three. And there we go. Now wait, I also ran this 10 times in Dall? E 3 and I was really strict with it and it still did extremely well. It was able to do it eight times out of 10. These were the good cases and these were the failure cases. Even these are not so bad. It just switched up the colors or added some extra text. Why is this interesting? Well, stable diffusion3 can also do this, but it is an open system that is free for all of us and three Creativity. I love how it is able to imagine new scenes that we've likely never seen before. It can use its knowledge about existing things and extend that knowledge into new situations. Loving it. If everything goes well, the paper will appear in the next few days and I am also hoping to get access to the models soon. You know, images of fellow scholars holding onto their papers need to be done. Subscribe and hit the Bell icon if you're interested in a deeper look when it arrives. However, we already know some details. For instance, the earlier stable diffusion 1.5 has about 1 billion parameters, SDXL is 3.5 billion, and this new one is 0.8 billion to 8 billion. So even the heavier version of this will still likely generate images in a matter of seconds. And the lighter version will, I think, easily run on your phone in your pocket. And to have this capability right in your pocket, my goodness, what a time to be alive. And in the meantime you can do a heck of a lot more with already existing tools. For instance, the Stability API can now help you with a great deal more than just text to image. You can get it to reimagine parts of the scene as well. And don't forget Stablelm also exists. That's free too. If everything goes well, we will talk about how you can run these free large language models privately at home soon. And we will talk about more amazing models. DeepMind's Gemini Pro 1.5 and get this, a smaller free version of it that is called JAMA that you can run at home for free. That video is coming soon too. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU Cloud compute. No commitments or negotiation required. Just sign up and launch an instance and hold on to your papers. Because with the Lambda GPU cloud you can now get on demand H100 instances and they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today.
