Dr. Zsolnai tests Stable Diffusion XL and finds it handles hands, spatial layouts, and text better than before. Prompts need fewer words for good results, and ControlNet conditioning is coming soon. SDXL matches Midjourney's fidelity to artistic style in his local tests.
August 11, 2023
"199 per hour" (said aloud as "one99")
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to see some amazing and some less amazing results From Stable Diffusion XL, the popular text to image AI's new version that you can run for free online or even at home. So what's new here? Well one it offers higher resolution images and is better at challenging concepts that previous text to image AIs faltered at. And that is human hands and very specific spatial arrangements where we say that a woman should be in the background chasing a dog in the foreground. These now work better. However, I'll immediately say that this does not mean perfect. Here are some of the results I generated locally. Obviously it cannot be anything else but images of scholars holding onto their papers or at least studying a huge magical tome. It can do that. However, look, hands still seem to be an issue and if you have a favorite artist and you love the style but would like to imagine what if he explored painting different subjects, you can now do it at home for free. I think this is an amazing tool to explore new artistic ideas and and I have to say it is incredibly fun too. And when comparing to midjourney I think the quality of the results are better there. However, SDXL seems to be more true to the original style of the artist. Loving it. I could also not resist trying Daniel Baskin's amazing drink prompts which are also working quite well. Yummy. Now they say that users generally prefer the new techniques results to previous versions of Stable Diffuse, although I haven't seen this user study being linked to a peer reviewed paper. So a wise fellow scholar like you are does not take these results for granted. We are going to try some more ourselves. 2. Simpler prompting if you have played with previous versions of Stable Diffusion, you probably noticed that it requires a very detailed description of an image to create something decent. But here. Here it is easier to create something with just a few words. My experiments have mostly matched this. When asking for an illustration of a small modern house in Osaka, I got quite usable images that I liked. This layered cake in the style of a landscape was also just a couple words and the results are incredible. This is so much fun. I hope you fellow scholars will also come up with some crazy ideas using this. 3. It finally supports better text generation now writing text is typically quite difficult for text to image AIs. When I asked for the full 2 minute papers in writing, I did get better results than with most previous techniques I use, but most of the time this was a little too much to ask for. After many tries I had some success with it kind of. However I found that when asking for just sdxl. Even then it took a bit of trial and error, but I finally started seeing some more success. We are not there yet, but this is just the 1.0 version of stable Diffusion Excel, so I am excited to see how it improves over time. Also, do you remember ControlNet? ControlNet is a neural network structure that helps us give it additional inputs beyond just text to image and it is absolutely amazing. For instance, we can provide just the edges of an input image, a rough sketch or edges extracted from a real photo and we get a beautiful image with exactly that creature and framing. Other kinds of inputs also work with ControlNet and as the preliminary results show, this feature is coming soon to Stable Diffusion XL which will bump up its usability significantly and we get all of these for free forever. That is excellent. It is all so new and so fresh. There aren't too many results out there yet. And don't forget there are many ways to improve this base model through checkpoints and magical things called loras. So specialized versions of SDXL will come out as soon as weeks or perhaps the days from now. It will be so much better. What a time to be alive. And if you wish to try it in your browser or run it locally, I got you covered with some links in the video description. So for now, let the experiments begin. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud computer. No commitments or negotiation required. Just sign up and launch an instance and hold on to your papers because with the Lambda GPU cloud you can now get on demand H100 instances for just 199 per hour. Yes, one99 and they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Thanks for watching and for your generous support and I'll see you next time.
