Flux CANNOT be stopped! They Just Keep Shipping NEW AI Tools!

Hello everybody and welcome back to the MatVet Pro AI YouTube channel. Today we are going to be talking about some image generation AI updates. And all of these updates are coming from Black Forest Labs. And for those of you who might not be as up to date, Black Forest Labs is a company that develops image generation models, specifically the set of Flux models. Now back in October when these Flux bottles were announced, they were a pretty big deal and very, very high quality like state of the art level image generation, they had some open source options as well as higher quality API options for developers. Now kind of like the original stable diffusion, most image generation or AI art generation websites have some version of Flux ready to go. Quite recently in November they launched their Ultra and RAW modes which can generate up to four megapixel photos which is like four times the standard size in today's image generation. And even more recently, today actually they announced Flux Tools and this is our subject of today's overview. Specifically these are a suite of models designed to add control and steerability to their base text to image models which not only enables the modification of AI generated images, but also real images as well. As, and as we can see here with this release, Flux Tools consists of four distinct features that will be available as open access models within the Flux 1 dev model. So what are these four tools? Well, let's start off with Flux 1 fill. This is state of the art, meaning very competitive with other companies in painting and outpainting models, enabling editing and expansion of real and generated images given a text description and a binary mask. Basically the simplest way I could put this is if you had a still image of me right here, you could highlight both of my hands and have Flux 1 fill replace these hands with rubber ducks or anything you want really. Or you could give me a hat, maybe you could give me some glasses, change the background or expand the whole image beyond the borders you can see right now and you know, take a guess at to what the rest of my room might look like. And as I alluded to earlier, there are other models that exist in the market that also do inpainting and out painting quite well. So a direct comparison might be in order here. Moving on, We've also got Flux 1 Depth. These are models trained to enable structural guidance based on a depth map extracted from an input image and a text prompt. So basically they've trained a model to look at an image and try to guess what the depth is inside of the image. And when you think about it, we as humans actually do this all the time. For example, again, look at my camera, see me. There are visual cues in this image that suggest I am closer to the camera than the background is. First of all, the background is a little bit more blurred out, but also in comparison to some of the things in the background, I appear larger, which again is suggestive that I'm closer. And there are also subtle cues like shadows. The comparison of my two eyes, let's say. In fact, there are visual cues that you don't even think about or recognize that the AI can pick up on because it's an AI and sort of figure out what depth is supposed to look like. And when you figure out the depth of an image, well, it's easy to do a lot of other things with it as well. Next up, we have Flux1Canny. These are models trained to enable structural guidance based on canny edges extracted from an input image and a text prompt in relation to canny edges. I am no professional, but I assume that this canny model works in conjunction with the depth model to provide more accuracy. Could be wrong though. We will have to see. And finally, we have fluxone Redux. This is an adapter that allows mixing and recreating input images and text prompts. This one is the most interesting one to me. I believe that this allows for some level of character consistency, at least based on the quick demos I've seen floating around the Internet. So right now this is all available through their Black Forest Labs API as fluxone Pro variants. And there are inference code and weights available as guidance distilled open access, Flux one dev variants. These models are also going to be available though on a few of their partners, which are all pretty great. Fall AI is a classic fan favorite as well as Replicate also available on together AI, Freepik and Korea. So a pretty good starting lineup. Let's get to the examples. I am really excited. Okay, first off, it looks like we're taking a look at the inpainting and out painting. So we start off with this image right here. Obviously a photograph, I assume of a real person standing in a field. Although this could be a Flux generation, we don't really know that's how far AI image generation has come at this point anyways. We sort of mask out the coat and not in a way that is particularly good, by the way. It's not a perfect masking job. It looks like someone kind of just, you know, scribbled over it the best of their abilities. Right. We then prompt the AI with a text prompt like Denim Jacket and it fills in the Rest of the details basically by guessing and it does a pretty great and realistic job even keeping the consistency of the tie, which really, really impressed me. As well as the same lighting and overall style of the image. It really does fit. And as I stated earlier, this isn't necessarily something new, but it could be the best in painting we've ever seen. That's why we are going to be comparing it specifically to Ideogram, which in my opinion has the best inpainting I've seen thus far. We've got another example here with the signs. As you guys probably already know, Flux is pretty darn good at generating text. So we've got burgers and beers, we erase the top here and we change it to coffee and beers and it keeps again the same exact style, the same exact font for this light up sign. It looks like it belongs. Here's an example of the out painting. We start with this hyper zoomed in image of some dude's eye and then we fill in the rest of the details outward and it generates a person to fit that small eyeball. And yeah, it looks like it fits. It looks like it's the same person. No complaints here. I wouldn't be able to tell that it was generated just based off of this eye alone. And what do you know, they did some benchmarks as well for the inpainting and out painting and it seems like their top model Flux 1 Fill Pro seems to knock out Ideogram V2, but not by much. And it looks like Flux 1 Fill Dev is about at the same quality level as ideogram V2. But still though, this is absolutely more open than ideogram V2. As we mentioned earlier, there's going to be open weights with the inference code which is better for the whole AI generation community because other researchers such as Ideogram for example, can learn from the Flux models and improve their models as a result. It increases competition, it makes it cheaper and more accessible for everybody. And open source stuff is awesome. And I know the Flux Black Forest models aren't as open as people would like them to be, but you have to give them some credit for at least pushing in the right direction. Because let's be honest, most Companies MidJourney Ideogram OpenAI Aren't that open. Moving on, let's take a look at some of the examples for the Canyon depth models we can see. This is obviously depth up here. We start with this AI image generation of some like alien Star wars esque creatures and it does a pretty good job at guesstimating what the depth of the image might consist of. Obviously, the more bright, the closer to the camera and the darker, the farther away. And it does a good job. I mean, the ears in comparison to the background of this other creature are clearly visible. They're in a hole, obviously, and that's all dark there. You don't know how far back that goes. And then even this wall is a little bit farther away than this one. You can see it gradually moves up. So the depth model looks pretty darn good anyways. Yeah, with this depth, we can now recreate this image using that same control, that same depth control, and change those creatures into anything we want, including a painting, which is pretty cool. Now for Canny, which is more line and edge detection based. We can take this owl and turn it into any variant of owl in the same or very similar position with very similar features. And even though we're just using the candy lines here, it still does depth quite well. Although you can see it does take some of the bokeh in the background, which in this example gets turned into butterflies. So whether that's a good thing or a bad thing is kind of up to your preference. Anyways, we've got more benchmarks over here. It looks like their depth model is just a little bit better than Mid Journey Retexture, but again, not by too much. Absolutely state of the art, though, if these benchmarks are to be believed. And for Canny, it seems to be better by a decent margin than stable diffusion 1.5 and SDXL. Now we get to the fun part for me, which is the image variation in restyling with Redux. This is what I'm most excited for. So it's an adapter for all Flux base models for image variation generation. Given an input image, Flux1, Redux can reproduce the image with slight variation, allowing to refine a given image. And this again is where you get consistency. Right. We input this image, which could be a real image, and it generates almost identical loafs of bread, cheese wheels and cups of milk or cream. Same colors, same lighting. There is some variation definitely with like the loaf of bread, the cheese and the cup, but it's not that high. Just enough variation to give you some more freeform options. Right. And again, this can also be used for somewhat consistent characters as well as image restylization. So in this example, we've got this duck, he's got a coat on and a very specific hat, blue eyes. It does a pretty good job at recreating this character in different formats. These are Also a bunch of different stylizations, so keep that in mind. If we kept the same style here, we might be able to do better consistent characters. But we can see him as a glass sculpture, a 3D model, an actual duck, or an epic anime art image. And again, we have another picture, this time of a real person. We can have them as vector art. We can turn them into a hand puppet, pen and ink sketch, or again, anime artwork. It does a great job, it really does, at capturing all of his defining features. I'd love to try this on myself. Right? Because we know what we look like. We look in the mirror every day. We'll have to see how accurate it truly is and if it can really be used for consistent characters, because that's what I've seen people talk about the most. Anyways, this one seems to really handily beat SD 1.5 and SDXL. SDXL looks like it's almost twice as good here. And for SD 1.5, we're looking at a pretty handy increase of like 40 or 30%. So there's a detailed breakdown of what's new. Let's play with it ourselves. My chosen method for today is going to be Fall AI. Fantastic website to test this kind of thing out. It looks like they are actually giving us a little bit of a demo to set up right off the bat, where we've got this knight with a little icon on his armor shield. And then down here we have the mask. And obviously we're highlighting the shield and going to change whatever's on the shield for the prompt. Looks like we want to describe the entire image. So a knight in shining armor holding a great shield with fall on it. That's what they're giving you. But we're going to change that. A lemon icon with the text that reads MVP can be seen on the shield. Let's give this a shot. Hey. And there you go. Here's our generation. Really not too bad at all. Obviously keeps all those original details of the knight and just gives us MVP on this lemon shield. And it actually kind of bowed it out and made it. This 3D thing looks pretty cool. But we have to try this with Ideogram AI as well to see how it compares. And we'll do the same exact prompt. Here is that original initial image, by the way. Plop this right into Ideogram AI, which does have a very nice user interface, by the way, in comparison to these new flux models which we're using on Fall AI. But since these are somewhat open, they can be Applied to other websites through the API. So you could get a nice interface like this using these new tools. Eventually we'll go to magic fill. We'll go highlight that same exact portion of the shield right here, just like that, same exact prompt and we will magic fill. And in comparison here to our Flux generation, I gotta say they're both pretty darn good. We do have some more variations coming from Ideogram. I'm not happy with this one because this looks like total clip art. Same here with this one. And honestly, almost same here with this one. First one's pretty darn good, but I do like that 3D. Lemon, I might have to give this result to Flux. I think Flux pulled it off a little bit better. To be fair though, with a different prompt we could get much better results with Ideogram. Let's try something a little bit more difficult. I'm going to upload a photo of me and we'll see how that comes edited. I've uploaded this lovely photo to the web via Discord. Let's go ahead and plop that image URL in here. And now to make the mask, which you could do manually or they have a create button here and. Oh, this is a nice little interface. Actually. Let's start off with something fun. Actually going to erase the entire background here. Alright. And this looks about good. Not a huge fan of this masking tool. I'm gonna be real. Alright, so we've got our mask, We've got me. We're replacing the entire background. High quality photo of a man looking at the camera with a thumbs up. The background is the surface of the moon. We can see a little green alien in the background as well as the Earth in the in space. Sure, that's good. Let's see what happens. Oh, what an interesting result. It did a really good job actually, like masking me out and putting things behind me. But I have a feeling that my prompting didn't work out. Here we've got two Earths or Earth like planets, a green alien and then two dudes and then the moon in the background. It's kind of a cool looking like image regardless. But I think we need to adjust our prompt. Let's remove like the context. I think it's adding these people in because I said photo of a man. Let's retry it. All right, Getting a little bit closer here. Let's just try removing Earth and have it only do the green alien in the surface of the moon. There we go. That's a little bit more like it. It Gave me some very flowy hair. We've got this nice green alien though. And yeah, I could be on the surface of the moon. Alright, let's see how difficult it is to get something like this from Ideogram. Same exact image. We'll go ahead and use the magic fill. And there we go. Same exact prompt. Let's see how it does. And right away with the same prompt, we get a pretty much identical in quality image. I would say the alien has a weapon and is pointing it at me, which is something that we didn't see from Flux Ideograms. A little bit more crazy. We've also got some other examples though. Yeah, I would say this is all kind of about the same quality. Maybe this is a little better. I don't know. It's, it's, it's very, very close. This is probably the best Ideogram one. I mean, side by side here, actually quite similar. I was expecting Ideogram to do a better job with something like this, but as it turns out they're. They're kind of neck and neck so far. As a final test, I want to try to put some yellow lemon themed headphones on my head. Let's see how it goes. Yellow lemon themed headphones. Each headphone looks like a lemon. Sure. And we'll go ahead and do the same thing in Ideogram. Oh my. Okay. Well, you know what, that's a result for sure. I would say that's lemon themed headphones. I mean, they look ridiculous. They don't look super realistic, but there they are. Oh. Now how is this going to compare to Ideogram? Okay. I have to say I do like these headphones a little bit better. They look more realistic. Oh, and they're all quite similar actually. This is pretty good too. Yeah. I think I'm gonna have to give the win to Ideogram here. I think these are just a little bit better. To be fair. We can rerun this and see if we can get a better result. Hey, that is kind of a better result. Except for my ear is mushing with the headphone. Okay, that's. That is quite disturbing right there. I'm not a fan of that. So that's Inpainting. It's going to be the same model for Inpainting and out painting. So I think it's safe to say here. Really depends on how you prompt the thing. It looks like it's about as good. Definitely on par with Ideogram, which I would consider top tier. So it's definitely state of the Art more open, more accessible and probably going to end up being cheaper than Ideogram. Now I want to move on to Redux and this again is the variations consistent characters. Let's go ahead and upload that same photo of me. Can it just iterate. Oh, there we go. That's a decent enough variation. I mean it's definitely me. Same shirt, got the neon signs in the background. Still have the futon. It has that same setup with the little lemon plushie, the YouTube plaque. It's actually pretty good. But this is a much hunkier version of me, I would say. And maybe an older version of me. Matt prime, if you will. Let's see though if we can change it up. Let's increase the CFG scale and see what happens. Okay. It just seems to maybe adhere a little bit better. Yeah, that's. That's basically the same quality. How would. If we do the same exact image size, does that matter at all? I think the quality increases maybe a little bit if you use the same aspect ratio as your upload image. But that's about as close as it got to replicating me. Lets try variations of something else. Got a nice OG dolly 2 generation. I want to run through this thing. This is a difficult art style to replicate. What if I do like photo of a strawberry? Will it change this character into a strawberry? Keeping that same style? Oh, kind of. Still very much yellow. I do like this character though. I like what it did. Photo of a lemon character eating a cheeseburger. Let's make this guy eat a cheeseburger and let's crank the cfg. Okay. Still did not really get there. It's just kind of doing loose variations. Alright, I've switched the model over to the Flux 1.1 Pro Ultra Redux and it seems to have maybe done a little bit of a better job still definitely a different person than me. The reason guys that I've been trying to pursue this consistent character creation so much is that I saw faux fur AI on Twitter do it very well. As he mentions, the outputs aren't perfect, but these examples are a lot better than what I was getting. Not exactly quite sure why my results aren't as good as these. Like this to me is pretty insane character consistency overall, I mean it's not perfect, but it's a lot better than what I was getting with a photo of myself. Like this right here is pretty darn impressive. He's using Flux Redux dev here, but I thought the Pro model was better. Maybe I'm missing something. I don't know, let me know in the comments. I will also link this tweet down below. And finally, just to end it off here real quick, we're gonna look at the depth. I'm going to link canny in the description, but I'm not gonna mess around with it because to be real with you, I think the depth is more useful and kind of does the same thing better. Let's do another photo of me. I want to become a green alien. Photo of a green alien. Looking in the camera background is Mars. Hey, there we go. I kind of look like the Hulk a little bit, but yeah, that's me as an alien. Very happy with that. You can see though, it pretty clearly understands the depth of the situation. It was able to pick out that there's like a chair behind me. I think if we prompted the chair in there, things might look a little bit better, but it's. It's pretty darn good at understanding the depth. Uploading a cute photo of a cat. And instead of lemon trees in this little photo down here, we're gonna change it to something else. Xbox controllers. And I mean, it definitely recreated the cat quite well. It put a forest in the background, but this got pretty mushed up into Xbox controllers. Let's crank the cfg. Okay. CFG seems to have helped a little bit, but it looks like it's just trying to put Xbox logos all over the tree. To be fair, that's a difficult prompt. All right, before we wrap this up, if you want to access these models the way I did on Fall AI, like I said, all the links are going to be below. All you have to do is log in with a GitHub account and you can add credits to your account. You can see what I did was add like 10 bucks and I've been using it on and off, but all the imaging that we did today cost about a dollar, so that's not too bad. And of course you can also access these models on CREA AI, which might have some free capability and some of those other websites as well. Please let me know what you think though. This is obviously great for the community at large. These are high quality, state of the art models for doing image editing. I wasn't able to replicate those demos of character consistency, which made me a little sad. But it looks like the in painting is pretty stellar, the outpainting stellar, and it's definitely nothing to be upset about. More open access and better models for the community. Thanks so much for watching, folks. I'll see you in the next video and goodbye.