This YouTube tutorial (hosted by Matt Wolfe) demonstrates two ways to control pose and composition in Stable Diffusion generations using recent ControlNet extensions. The first tool, Scribble Diffusion, converts crude mouse-drawn sketches into images that follow the drawn composition (examples: mountains with sun, a penguin, a palm tree on a beach). The second uses a ControlNet pose model (via Replicate) to transfer a photographed pose onto new subjects (examples: astronaut, green alien giving thumbs up, metal robot in a thinking pose). The video includes links to the tools (scribblediffusion.com and a Replicate ControlNet pose page), a shout-out to a Discord user (Apex proxy), and an outro noting the creator passed 100,000 subscribers.
Questions this video answers
scribblediffusion.com; a Replicate ControlNet pose page (referenced as "Replicate.com Jagili Controlnet pose").
Matt Wolfe.
"I just passed 100,000 subscribers." (spoken in the outro).
Chapters
In this video, I'm going to show you how to get the exact poses and compositions that you want out of your AI generated images. Let's go. Now, one of the issues you might run into when you are trying to create AI generated images is getting the exact pose and positioning of the image that you're looking for. Well, there's some new tools that are available now that actually help you get the exact positioning from what you're trying to do with your stable diffusion prompts. Both of them are due to this new thing called Control. Net, which offer a few extra add ons that you could do with stable diffusion. And some really smart folks out there have actually made them freely available and easy to access by anyone. So the first one I want to show you is this one called scribble diffusion here, which you can find@scribbledifusion.com now this is basically just stable diffusion behind the scenes. But whatever you scribble onto the picture and then you give it a prompt, it tries to match the layout and the composition that you drew in your scribbles. So, so the example that it defaulted to when I landed on this page here was the sun setting behind the mountains. And you can see it's got three mountains and it's got a sun setting here. So if we just generate this demo example here and let it run real quick, you can see that this is what it generated. So it basically took a combination of this image prompt here with the sun setting behind the mountains and generated this. And you can actually see that it's got the three mountains here. It actually kind of put a shadow here where the sun is. But you can see that it matched the composition that it saw. Let's try to draw our own. Forgive me, am horrible at drawing using a mouse, but let's say we want to draw a penguin. So here's a penguin body here. Let's give the penguin some wings or arms or whatever you call them on a penguin. I guess they'd be wings, right? Give them some penguin feet here. There's our penguin feet. Draw our penguin head. Here's our little penguin beak. Penguin eyes. Let's give it a little mohawk like that. There's my awesome looking penguin. So let's go a penguin standing in the snow. And because I draw drew this out, hopefully we get a penguin looking straight at the camera with a similar pose. And there you go. There's my penguin. Kind of followed my eyes a little too closely there. And instead of giving it a mohawk, it kind of put a tree in the background. But you can see it's trying to match the composition that I drew right here. Let's try one more. Let me clear this here. Let's draw some ground here. And then let me draw a palm tree. Because I feel like I could maybe draw something that kind of sort of looks like a palm tree with a mouse. Nope, I was wrong. There's my palm tree. Here's some coconuts hanging off of it. And let's just say a photo realistic palm tree on the beach. Let's generate that. And there you go. There's my photo realistic palm tree on the beach. Who would have thought that this crappy drawing can turn into something like this? You can tell it's got a little bit of work. And I can come and reroll this here, hit go again and generate a few more until it comes up with one that I really like. But the point is, you can see that it follows the composition of your original drawing. You've got the three mountains with the circle here. Three mountains with the circle here. You've got a penguin looking into the camera with the wings in this sort of position here. Same thing here. You got the palm tree sort of leaning out over here. Follows my prompt of it being on the beach. Follows that exact same angle of what I drew. Pretty cool stuff. You can find this one over at Scribble Diffusion fun to play with. And as of right now, it's a completely free tool to play around with. Now, the other one that I want to show you is also using this new Control Net, but it's using the Control Net pose, where it tries to interpret the pose of whatever is in your image. You can see that this image right here is somebody standing in this position. The guy from Peaky Blinders here. And you can see this is how it interpreted it. You can see that it figured out what position he's in. They put in a prompt here of an astronaut on the moon. Digital art. And you can see it put an astronaut on the moon following the same pose. So the hands and the head are kind of all in the same position looking down a hand here. Let's say I want to do an image of a green alien in space giving two thumbs up like that. Well, I just happen to have an image of me giving two thumbs up, just like that. So I can drop this in here and come down to our prompt and say, a green alien giving thumbs up on a strange alien world. And then digital art. Let's just add Photorealistic hd since it is using stable diffusion I like to give it a whole bunch of extra prompts. Unreal Engine October, Render Detailed. And then I'm going to leave everything else the same. Let's just let it use a random seed. We'll use the 20 steps here and then I'll click submit. And this is what it generated here. Kind of missed the thumbs up, but you can see it's trying to model the same pose that I'm in right here. And this is how this model actually interpreted my pose. You can see it's kind of got the body lean into the side with the neck going off this way with my shoulders at a slant. And then it's using that pose for the alien image here. So maybe I'll actually remove some of the prompts and see if we could get something a little bit closer to the two thumbs up we got going on here. So let's go. I actually missed the word up in this last prompt. So a green alien giving thumbs up on a strange alien world. And you know what, let me just get rid of all the extra prompts and see what it does if I just leave it to its own devices here. There we go. Now we got our alien given the thumbs up, I think because I just put giving thumbs instead of thumbs up last time, it messed it up. See, it's trying to model my pose. It's got four fingers and a thumb on this side and five fingers and a thumb on this side. Still stable. Diffusion still doesn't great with hands, but it's getting my thumbs up pose. Here's an image of me in a thinking pose, I guess you can call it. Let's do a curious metal robot in deep thought. Let's see if we can get it to match this pose of the head, the hand against the figure, and then the arm kind of in front of it like that. And let's go ahead and submit it. And there we go. I don't know why it gave it a baby face, but you can see that it's trying to match my pose here. The shoulders are sort of in the same position. Arm down, arm up, arm leaning right here, head tilted to the side. It is figuring out the position of my head here and then mapping it onto an image of a metal robot with a creepy baby face. Maybe I'll come down here under the negative prompt. Let's put child, baby, kid. That way it has a robot face. A curious metal robot in deep thread thought. Let's try this one more time. See if we can get something a little bit closer to what my mind is envisioning. There you go, that's a little bit closer. The robot doesn't have eyes for some reason, but you can see it's still following my pose. It's still ahead leaning against an arm here with the other arm sort of in this same position that my arm is here. Just a couple cool tools to play around with. This one you can find over@ replicate.com Jagili Controlnet pose Don't worry, I'll make sure I link it up below the video. And the other one was called Scribble Diffusion which you can find@scribbledifusion.com quick shout shout out to Apex proxy who suggested this one to me in the discord here. Thanks so much Apex for sharing that. It's a really cool resource that I've been having some fun playing around with, so hopefully you enjoy this video. A couple new ways to get your AI images just a little bit more dialed in. You can draw some random sketches and generate some realistic images out of it. You can throw in some poses, get the exact pose that you're looking for from an image. For me, this is starting to up level my YouTube thumbnail game because I can kind of start to get the pictures exactly how I envisioned. Obviously sometimes it takes a few tries, a few RE rolls of your generations before you get exactly what you're looking for, but this definitely opens more doors to get closer to that exact image that you have in mind. Hopefully you found this helpful. At the very least, hopefully you found it fun. I know I'm having fun with tools like this and if you're looking for more tools to have fun with, head on over to FutureTools IO where I curate all of the coolest tools that I come across. And if there's just way too many tools here and it's overwhelming to you, click on this little button to join the free Newsletter I send a newsletter every single Friday where I share my five favorite tools from the week. I share three cool YouTube videos, three cool news articles, and one interesting way to make money with AI. And you can find it all over at FutureTools IO. I hope you enjoyed this video. If you did, maybe consider giving it one of them thumbs up this down there. If you give it a like, you're going to probably see more videos from me in your feed. And if you like Nerd not about cool AI tools like I do, that's a good thing, right? Also, if you haven't subscribed yet, please press that subscribe button. I just passed 100,000 subscribers. So thank you so much for everybody that's been watching and pressing that subscribe button. It's been insane how fast this YouTube channel has grown. And I really, really appreciate all of you, every single person that watches the videos, even the people that leave the mean comments. I like you, too, because it means you watched my videos, and I appreciate you for taking the time. So thanks so much for hanging out and watching my videos and helping me get to 100,000 subscribers. You guys are awesome. You rock. I love every single one of you. And thanks for watching. I'll see you guys in the next video. Bye.
