Eight Stable Diffusion workflows: ControlNet video transforms, QR codes that become art and still scan, upscaling rough sketches, period-style portraits, RPG characters, natural-language image editing, and smoother zoom videos.
Dear fellow scholars, this is two minute papers with Dr. Today I will show you some papers and results with Stable Diffusion, a free and open source image generator AI And I cannot believe how good these results are. First, we had an earlier example with ControlNet and dancing. This means that in goes something, for instance, a video of us in and out comes a hand drawn anime version of it. The advantage of this is that we don't need to look perfect and we don't even need to be at this exact place in the background because all of this can be synthesized. And today someone took this idea to the next level. Look. And now if you think this is a blue haired woman converted into a more painterly blue haired woman, well, have a look at this. Oh yes. We don't even need to be the character we want to be. We just need to kind of look like the character we want to be. The word kind of is doing a lot of heavy lifting here. This is going to be an amazing tool for the digital artists of the future. And remember, this needs a tiny bit of time and expertise, but it is something that you can run for free on your own machine. And some of the other examples, samples even in your pocket on your smartphone. What a time to be alive. 2. We can even use Stable Diffusion's ControlNet to provide it input images for it to redraw. For instance, we can give it this childhood drawing and we can ask this little AI to create a super high quality version of it. My goodness, this is incredibly good. 3. Now here is a crazy idea. Why not give it a QR code and see what happens? Well, these look great. So this was a good thought experiment. But of course these won't work as QR codes, just as a piece of art. Wait a second. Hold on to your papers, fellow scholars. Do they actually work as QR codes? And are they readable from a smartphone camera? Yes, they are. Make sure to play with the distance a bit, perhaps increase it a little. But every single one of these work for me. I can't believe it. Artificial creativity. Wow. 4. And these video to video workflows are becoming so incredibly good that. Are you seeing what I am seeing? Can that really be? These can now be generated almost immediately. If we wish to create a smoother video, we would have to wait for a bit. But not in the order of hours, just in the order of seconds. And super quick previews can be done like this. This would not have been possible just a few months ago. Unbelievable. 5. With the right prompts, Stable Diffusion can also create an 1800s version of our favorite rock stars Elton John, Freddie Mercury, Dave Grohl, Hendrix or Jagger, you name it. I love how creative some of these solutions are and how they retain their looks and personalities while putting them into an 1800s setting. 6. We can also create custom portraits for our Dungeons and Dragons characters. Just give the AI the description and there we go. The prompts and the workflow for some of these images are available, which means that you can likely reproduce and tweak them yourself at home. So good. 7. In the meantime, Google just published a really cool paper on how to guide these image generator AIs with natural language. First we can ask for an image of a giant macaron and croissant in front of the Eiffel Tower. Well, yes, I hear you saying doc with today's papers that is not surprising. But what is surprising is that without generating a new image we can swap these objects, then enlarge the macaron and if we don't like it, replace it with a different one. Or even better, use our own photo and ask it to recapture its appearance, which it had done really well as the lighting now also has an effect on these objects as well. Not just the background is different, but everything is different. So interacting with text to image AIs via natural language. How cool is that? 8. Creating Zoom videos has also improved a great deal. Now we can choose a few images and stitch them together to create these zoom videos and they are now really smooth. Previous ones had some flickering issues, but this one is now so much better. And all this improvement in just a few months. Unbelievable. I put a link to all these works in the video description and for now let the experiment begin. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU Cloud compute. No commitment or negotiation required. Just sign up for and launch an instance and hold on to your papers. Because with the Lambda GPU cloud you can now get on demand H100 instances for just one 99 per hour. Yes, one 99 and they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances. Workstations or servers make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Thanks for watching and for your generous support and I'll see you next time.
