Metas New TEXT-TO-IMAGE Takes Everyone By SURPRISE! (Now Announced!) CM3leon

So Meta just released something that caught everybody off guard. Meta actually released a C3 Leon, a more efficient state of the art generative model for text and images. Or if you don't want to complicate things, essentially, Text to image. Now, Text to Image has largely already been conquered by sites like Midjourney that do provide, I would say, state of the art text image generation that I genuinely don't see anyone even coming close to, which is why this kind of technology is quite surprising. But I would still argue that some of the features that you do see in this Text to Image software is going to be kind of different than what we do see in Midjourney, because there are a few things that are just a little bit different when we do compare them. And you actually might be surprised by some of the features that you do see, because I genuinely didn't expect some of these features to be in this software, nor did we expect it to be released. Now, for some of you who are confused, you have to understand that Meta is a giant AI company and they've actually pivoted towards the AI space after spending billions on the Metaverse and realizing that it wasn't that great of an investment. So Meta basically has an AI team that's dedicated to providing the community with open source and other relatively great AI technologies that they are hoping shapes the rest of the AI landscape. And that's why we now have this Text to Image software. I do think that sometime later it's likely that it's probably going to be open source. So you can see here on the webpage, it actually comes up with this Meta AI, a small cactus wearing a straw hat and neon sunglasses in the Sahara desert. So, of course, if we were to input the same prompt into Midjourney, you can see that this is what we get. Now, I wanted to do a quick distinction because of course I know that the comparisons are going to be there. What's the point of even covering something like this if it isn't as good as Mid Journey? Well, if you keep watching the video, I'll show you the new features that probably might make Mid Journey feel some kind of pressure to release certain features. So this one that you're looking at right here, this is actually mid journey version 4, hence the reason the quality isn't as good. So, of course these ones aren't also upgraded, but I'm going to show you as well mid journey version 5, which you can see looks a lot more realistic and of course is just ridiculously high quality. I mean, for example, you can even see the reflections in the sunglasses of what appears to be the rest of the background. If that isn't truly incredible, I don't know what is. So that's why, you know, when you look at version 4, it actually does look pretty similar to this. So what we can see here is that Meta is definitely trying to increase their capabilities to in not only the text department, but pretty much every single department. Meta now has many different things and they earlier released make a video AI system that generates video from text. So they have text to video, text generation, audio generation, and now of course they have the final frontier, which is of course image generation. It seems like Meta might be loading up for their very own AI model which could be completely multimodal. So if you continue to read the webpage, you going to see something very interesting. Okay, we see something that says text to image and it says given prompt text with potentially high compositional structure generate a coherent image that follows the prompt. For example, the following four images were created for the prompts. A small cactus wearing a straw hat and neon sunglasses, which is what we saw in the first one. A close up of a human hand, high model, high quality, which is what we can also see here. And interestingly enough, this is actually quite impressive because many AI models for some reason definitely struggle to get human hands with the correct number of fingers correctly. So Meta actually chose this not out of sheer randomness, but they wanted to demonstrate that their text to image model was clearly superior when it comes to certain things. Because of course, as you know, this feature right here was something that was only recently solved in Midjourney version 5. And, and of course stable diffusion version, I think 0.9 or whichever stable diffusion version which was just released, which honestly blows everything out of the water. Then of course we can see here a raccoon main character in an anime preparing for an epic battle with a samurai sword battle stance. It looks pretty decent. But one of the most interesting ones that I want to show you all is this last one right here. It says a stop sign in a fantasy style with the text 1991. And I want you guys to really pay attention here because as you know, AI development is very interesting because largely we don't understand how lots of these models work. And one of the large problems with these models is that they do struggle with text. Now, stable diffusion doesn't struggle with text as much as midjourney does. I'm going to show you exactly what midjourney did when I used this prompt. Now, in order to be fair, I do know that Midjourney does have different variations, which. Which some versions are more artistic. For example, I chose to use V4 because V4 is the most artistic. And you can see that in version 4 of Midjourney, this is what the same prompt will give you. Now, I do know that you can alter certain things that might add stylize or whatever, but Midjourney's one fatal flaw is that it doesn't do text well. Although the text here does look high quality, it doesn't really say anything. And if you're wondering what version 5 would look like, this is what we get from Midjourney version 5. So I wanted to keep that in there just to show you why new AI technologies that are being developed, even if they don't excel on everything, there are some areas that other AI models simply just lack. Which means that on certain applications, if you do want text in an image, you might actually have to use something that you don't normally use. Now, this was the largest area to which there was much surprise. You see, this is what we have, and it's called Text Guided Image editing. Now, text Guided image editing is where you give yourself the default image and then essentially you edit this image via a text prompt. Now, there are some programs like Adobe's Generative Fill, that can actually do this. And you can see right here that with Adobe's Generative Fill, all you need to do is essentially click Generative Fill, and then you need to, of course, add yourself a text prompt like this, and this is something that you can actually use. We've done several videos on this before. Essentially you just need to sign up for Adobe's beta program. But it is something that does work really effective, and it's something that we haven't seen in anything other than Photoshop. So now that Facebook is incorporating something like this, it will be interesting to see if this is going to be a feature in future more advanced text to image AI models. So you can see we have the input image here. Then it says, what would she look like as a bearded man? And of course we can see it instantly changes it to a man with a beard, which is actually rather funny. Then of course we can see put on a pair of sunglasses. Then she should look 100 years old, and of course, apply face paint. Now, it would be interesting to test this compared to Photoshop's Generative Fill, because I don't doubt that the results would likely be very similar. Photoshop's Generative Fill has taken the industry by storm, with many different creatives Using it for many different things. And what they also state is that this is across the model, meaning that it isn't a separate model, it's simply one entire thing, which is, as the reference, Quite similar to InstructorPix2Pix, but instructs Pixapix was actually one separate thing, which is why they talk about how this is all bundled into one AI model. And of course we can see this similarities here. Swap the sunflowers with roses, add fireworks to the sky, Replace the fruits with cake, Turn it into a still from a western. All very interesting capabilities that we will see in future advanced AI models. And if you're wondering how Pix2Pix would actually handle this right here, I actually just tried it with Pix2 Pix demo and you can see that when I did ask this software to put on a pair of sunglasses, these were the sunglasses that it put on. So it's actually quite effective at doing this. And what I do hope is that just like Pix2 Pix, hopefully Meta will release this software software, at least in some kind of demo form where users can actually test it. And I'm not sure why they haven't actually released this as a product just yet, because I'm pretty sure if this was released to the Internet, just like ChatGPT was, many people would have vast uses for this software because it's easy to use and honestly, the only limit is your creativity. Now, another thing that the C3 Leon model can also do is it actually can analyze what's inside an image. So you can see it says, describe the given image in very fine detail. It says in this image, there is a dog holding a stick in its mouth, there's grass on the surface, in the background of image, there are trees. So this is something that we're increasingly see from a variety of AI models. And I would argue this is really good because although Midjourney does have a feature called Describe, in which you can input an image and then when you put describe in it, essentially somehow describes your prompt based on the image that you do have, this feature is actually going to be much better in terms of the wider uses of applications. Now, this is something that is pretty crazy to me. It says that given a text description of the bounding box, segmentation of the image, generate an image. So essentially what you're asking to do is generate a high quality image of a room that has a sink and a mirror in it, with a bottle at this location, with a sink at this location, and with a bed at this location. So Essentially what they've done is they've trained this AI model to be able to precisely put certain objects at certain locations in a final image generation. And that is something that we haven't seen before with midjourney. So this technology could accurately give us access to much finer and more detailed image recreations if we wanted it to. And this is something that we just haven't seen before. And when I saw this, it honestly blew my mind. Then, of course, we have segmentation to image. So essentially what they also had was an input image of a duck on a lake. Then of course, they extracted the different segments of that image. So what they've done is they've separated the image into bold colors based on different objects in the image. So, so they put the water as blue, they put the duck as pink, and then essentially they've reconstructed that image based on the segmentation. And you can see that the reconstructed image in generation one and generation two does look rather accurate to what we got. And the reason this is actually pretty scary. Imagine you got an extremely low resolution image, then you extracted the segmentation and then you regenerated the image. You could arguably have numerous applications for this software. Then, of course, the paper goes on to showcase how they actually.