Google's NEW 'StyleDrop'' Takes Everyone By SURPISE! (NOW ANNOUNCED!)

So Google have once again just released a new research paper and this one is sort of in image generation. Now, the thing about this paper is I do think it is really, really interesting in terms of what it provides. And it's a new, unique kind of AI software that I really didn't see coming because many AI softwares follow the same standard model. Either you are, you know, text to music, you're either text to video, or you're either image to text, which is very, very simple. But this AI is quite different because this is called styledrop, which is text to image generation in any style. Now I know this falls under the use of text to image generation, but it is still very interesting because I think this one will actually be used widely like Midjourney and alongside many other creative tools. And I think you guys should stick around for the examples of this because some of the examples are really, really interesting. We present styledrop that enables the generation of images that faithfully follow a specific style. Powered by Muse, a text to image generative vision transformer, styledrop is extremely versatile and captures nuances and details of a user provided style, such as color schemes, shading, design patterns, and local and global effects. So essentially what we have here is a tool that simply can learn any style and then replicate that style in any other artwork. So it's really, really interesting and really, really effective at how cool this tool is. And that's why I'm saying I do think this will be used a lot, because as you know, tools like Midjourney, one problem that everyone has is consistency. And if you don't know what that means, sometimes when you generate an image, it comes out in a certain style, but it's very hard to replicate that style unless it's a notorious style. Maybe, for example, it might be a specific artist like Van Gogh, or it might be a specific artistic style like Steampunk. But if you create your own style, there's not going to be a specific name for that. That's where styledrop comes in. So let's take a look at some of the examples that they've provided us with, because we want to really see if this is actually good or not. So what we can see here from the first example is colorful flowers in V sty style and the style reference image. So this is the image that they've used for reference. And essentially, if you don't know what style this is, this is kind of like a watercolor style, which if we were to Google that, you can see that this is likely what Midjourney would think we were talking about. And this is why this tool is very important. You see, if someone wanted to generate an image on a text image generator and selected the watercolor style, what styledrop actually allows you to do is customize it to that specific one of the image. Because if I wanted an image in this style, I likely wouldn't be able to get it. But that's where stardrop comes in. But here you can see a baby penguin in the specific style, a banana in the same specific style, a bench in that specific style, and we also have many other examples. And I think this is going to be really, really useful for those who are running different creative projects, because essentially any creative project that people work on, if you do have a creative project that requires a specific style or brand sort of theme, this is going to be the tool that people do use. So I'll be excited for this release. And to be honest with you guys, so far, looking at these examples of this watercolor style, it does look very, very effective. And of course just a caveat, because this is something that you should know is that when these kind of research papers are done, of course they will cherry pick certain outputs because it's going to showcase the best that the AI software has to offer. But it still does prove to us that this software is very effective. You can see right here that this is a style that we talked about in the beginning, which is of course the Van Gogh style. And, and of course you can see right here this is going to be something that people do want to reference. So now with this example, I think this is one of the styles that many people who know about art are going to be more familiar with. So this is by the famous artist Pank. So this is by the famous artist Van Gogh. He was an artist, or I guess you could say painter that did some paintings that made him really famous, but he's really known for this style. Now what I wanted to do was I also wanted to see just how effective this was when comparing it to other image generations like Midjourney. So we all know if you don't know what Midjourney is, essentially it's a really cool AI tool that you can use to generate realistic and hyper realistic images from simple text prompts. Now what Midjourney is also good at doing is if you have a text prompt and you decide to type in in X style or whatever style that you reference, it's going to be able to get that. So you can see right here that there are four at the top. And I Decided just to take these top four, but because I wanted to see how these would look if I used Midjourney. Now remember, this isn't going to be able to be done for every single kind of image because with Mid Journey we have millions of image in Van Gogh style that can be referenced. So let's take a look at what a baby penguin would look like in Van Gogh style and then compare it back to Dream Booth and then compare it back to Google's new tool just to see the different nuances in these AI tools. So here is the first four images generated with Midjourney. And if you're curious about the text prompt that I use for this, I typed in a baby penguin in Van Gogh Starry Night style and this is what Mid Journey gave me. Now if you're wondering which engine I was using, this is version 5.1, which is currently the default. And you can see here that compared to the Style Drop one, I got to be honest, Mid Journey's one, of course, as you know, is a little bit more detailed. But here's the thing, this is not a dog on Style drop because we have to understand is that if you don't have a physical reference like for example, Starry Night style, then you're not going to be able to get any specific style for, for your specific artwork. Which means that styledrop is going to be more useful in the future as newer styles get created and newer color schemes are involved. So I do think that although on one off scenarios where you do manage to reference an already popular style, it can work. But what is cool about styledrop is that it aims to replicate that from one simple image. So I do think what would be cool though is that if Midjourney is able to somehow integrate this into their software, maybe Google releases this with an API to you get a simple style from an image in midjourney and then you can use that to create different pieces of content. I think that would be really effective. But as we know, Google does a lot of research, but they don't always release these projects. Then let's look onto the second example which is a banana in this specific style. You can see, I got to be honest with you guys, that one thing that I'm starting to notice, as well as looking at these that I just wanted to add, is that all of these images do look pretty much on point with all of the other images. I mean, if someone said these are all from the same book, I would 100% believe. Whereas with Mid Journey, I got to be honest Some of the styles do look a little bit more different than the other, which would lead to you probably having to select many different images and kind of regenerate them. For example, the top one on the left looks the most Starry Night, whereas the bottom one on the right doesn't look that great compared to its official style, which is definitely very interesting anyways, when you take a look at the banana, I got to be honest, some of these artistic pieces are really interesting to look at and we do like them. But if we do compare it to the original, if I'm being completely honest, I do think that styledrop does this much better because I just think it actually gets the complete style in terms of the consistency, which is something that you do want if you are going to be creating something. And I think that's really, really effective, but it's going to be interesting to see how this evolves and how this sort of comes into actual use in the future. Then, of course, we have a bench in that specific style. And then of course, you can see here are the benches that we generated. Now, what's also interesting is that you can see that on the bottom left, it didn't actually generate the bench, because it just simply added the bench outside, which I did find pretty interesting. But nonetheless, it just goes to show that with a specific tool like styledrop, you're going to have a lot more consistency. And yeah, I'm not hating on Mid Journey at all. I think that both these tools are really, really incredible. But when we look at the reference image and then when we look at the consistency, it's definitely going to be that specific style. Then of course, the last one here, we do have a boat in, you know, that X specific style. And then of course, you can see these are the examples here. Now, I do think that with more fine tuning, you are going to be able to get the perfect style from Mid Journey or whatever. But I do think that this is just an interesting thing to show how these different models interpret the same kind of style and how they arrive at their final destination. Now, you can see right here, we can also look through some more examples. And this is what I'm saying, that if you manage to simply manipulate an image and want to have a specific style, this is going to be really, really interesting. I mean, look at all these different kinds of images and, and I don't know what specific style this is. This is what I'm saying. Midjourney wouldn't be able to help you here, although you might be able to describe it. By saying it looks black and yellow, only colors, a pastel drawing or whatever kind of, you know, descriptive words you can use, I think that styledrop is going to be much more effective. So this right here as well is something that I don't even know how you would get this done in mid journey. I mean, some of these images are just honestly. So I wouldn't say surprising, but just so interesting because this is something that I'm guessing is clearly maybe a kind of breakthrough in how they've done this. Because I've seen many different AI tools and what we've seen before is that many different AI tools are usually incremental gains on others. But I haven't really seen a tool that implements the style like this. Now one thing I do want to talk about after we go through this is other tools that are quite similar to this. So in the paper, what they reference is they reference other stuff like Dream Booth. Okay? And the thing is that this is actually different to Dream Booth. Now if you don't know what Dream Booth is and you're like, well, I don't even know what Dream Booth is. So let me just show you what Dream Booth is and why I'm referencing it. So Dream Booth is fine tuning text to image diffusion models for subject driven generation. So essentially Dream Booth takes an image, but rather than take the specific style of the image, and this was done by Google as well, they take the subject from the image. So for example, you can see right here we have a nice little corgi dog. And then of course you can take that corgi dog and then put it in the Acropolis swimming, sleeping in a doghouse, in a bucket, or getting a haircut. And that's what Dream Booth is. Now this is very different, although similar to dreambooth, okay? Styledrop is where you can simply just get the style and then you can get a completely new image in that style. So according to the paper, styledrop outperforms other recent methods such as dreambooth and textual inversion using imaging and stable diffusion as pre trained text to image backbones. The evaluation was based on prompt and style fidelity metrics using Clip and a user study. Therefore, styledrop is superior to other methods in terms of quality output. As for efficiency, StarrDrop is also built on a few crucial components including a transformer based text to image generation model, adapter tuning and iterative training with feedback. The authors also of this paper claim that their method is efficient and only requires a small number of training parameters. So if you're wondering also about some of the nuances of how this works. Essentially, StarrDrop captures the nuances of the user provided style in a text to image generation by fine tuning a pre trained text to image model with a single style reference image. The method efficiently learns a new style by fine tuning very few trainable parameters and less than 1% of the total model parameters and improving the quality via iterative training either with human or automated female. The resulting personalized text to image model can capture details such as color schemes, shading, design patterns, and local and global effects. So now I think if we continue to look at some of these examples, these are really, really cool because I mean, this is something that people do want. Okay? So for example, you can see right here that this is an illustration and this is something that I didn't show you before. And in the video, so you can see right here it says a person drowning into the phone in X style or whatever. And of course this is the one reference image that they have. And then of course you can see all of these different generations that are in the exact same style. And I got to be honest, it captures the style perfectly. Like, imagine you're someone who's trying to create an illustration for a kids book, or maybe for a website, or maybe for whatever brand that you're working on. This is going to be a really effective tool. Now I'm going to show you as well, right here. Okay. This is where we have really cool illustrations. Now some illustrations are in certain styles, but I do think that this is absolutely incredible in terms of the actual consistency of this style. I mean, it even gets the shape right, it even gets the color scheme right. And I think what's crazy about this, that most people don't even take into account is that certain things are just natural to this AI tool, like certain color schemes, certain hues, certain variations, certain width, certain lengths that would be very, very hard for a normal person to immediately get if they don't have some kind of artistic aspect. So for example, I know that sometimes even now, when people still use images from midjourney and stuff like that, what they'll try to do is they'll try to get these images that are all kind of similar. And for example, if you use a website like Freepik, where maybe you're trying to get a few vectors for maybe the front page of your website, you can see that lots of the time what you do have is you do have a lot of these packs that are sometimes in the similar style. Now sometimes as well, you don't always get them in the same, similar style. And you do have to sift through many different images and fine tune many different images to, to sort of suit that style with the same colors, but with style drop. This is actually really, really effective. And honestly, I got to be honest, after seeing more of these images, I'm definitely impressed. They definitely have fine tuned this model to be very, very effective. I mean, look at this sticker picture. And you can see right here that these do look incredible. So we're going to continue to look through these. And what's absolutely incredible is this one right here, this was one that I saw before and I was like, okay, this is crazy because even if I did see this, even if someone asked me to do this, I wasn't even sure how I would go about doing this. Now, I don't consider myself an artist, but still, it definitely seems pretty crazy. So a baby penguin in this style, a banana in this style, definitely goes to show you just how cool this is. Because what we often see with many AI tools is we often see easy examples, things that are very simple to do for that large language model or maybe that AI tool. But this you can see right here is definitely a difficult, would arguably be a very difficult style to emulate. I mean, what style would you even call this? This is just original colored smoke, maybe off a background. But we already have stuff that definitely looks really, really effective. I mean, you can actually see a Formula one car, a Christmas tree, a boat, a bench. It definitely looks really, really cool. Now some more of these as well, like I said, definitely would seem quite difficult to do. I mean, I'm not even sure what kind of style this is, but Stardrop manages to capture it perfectly. Now, something I also noticed when I was thinking about some of the recent AI tools that got released was something from Adobe. So Adobe recently did announce Firefly something on their webpage, which they didn't really announce. Announced was really, really interesting. You see if you go onto the Adobe page and you scroll down, you can see all of this amazing 3D modeling AI stuff, which is all cool. But if we scroll down to their stuff, where it shows you what they're working on, they say we're actively researching new ways generative AI can help creators express their ideas. And that's when I came down and I realized that there was a new thing. So this says. So this says customizable diffusion. It says apply your unique style. With AI in image generation, it can be challenging to achieve the precise visual aesthetic required for a project. With customizable diffusion, the Creator can select which images inform the generative AI. This offers a more creative control over individual images and a simpler way to apply creative choices across a body of work. So essentially Adobe are working on a tool that is quite similar to like this. Now I'm not going to get completely into Adobe's tool. I do think that if they do manage to fine tune it enough to the point where it works completely, it's going to be immediately added to the Creative Suite. So yeah, we're not going to get into this too much, but I will leave a link to this this in the description. But you can see that their multi concept customization of text to image diffusion is essentially something where you can get your specific image, just like Dream Booth and then you can get a specific style, like Drawings by Aaron Hertesman and then you essentially can get the subject in that specific style. So I guess it's combining Dream Booth and Google Style Drop into one specific tool that they are going to be creating. So definitely something that is really interesting and something that we should watch out for. And also if you are interested in AI, there is this thing called CVPR 2023 which is a conference about computer vision. So essentially the 2023 Conference on Computer vision and Pattern recognition will have a large amount of different papers that will be published. And you can see right here the publication date is June 18, 2023, which means in around seven days from the publishing of this video, we're going to have a lot more interesting generative AI tools that are going to be available and largely those that are related to computer computer vision. So back to style drop. You can see that as we continue on, there are many, many different examples that do look absolutely tremendous. I mean, looking at these current examples, I don't think that there's a tool that beats this in terms of the consistency and the level of quality that it is outputting. I have to say that Google definitely do work on some pretty crazy stuff. And what's also cool here, and I think this is going to be one of the game changers, is that this is a 3D style. And what's crazy is that 3D stuff is really, really hard to generate, especially when it comes to the shading, it comes to the ambient occlusion. Everything that goes Into Generating a 3D image is definitely quite hard, especially from 3D images. And to be able to generate these 3D models, even though they are just 2D images, is definitely something that is worthy of recognition. So I would say that Stardrop is definitely one of the most recent tools that I have seen that has really, really impressed me because honestly, this stuff right here is just really, really crazy. And the more examples I look at, just the more crazy this absolutely is. So I don't know why this tool wasn't more widely recognized, why I didn't see more people covering this. I guess because it's not a tool that you can actually use right now. That's probably why this wasn't as popularly covered. But if you go to the Google's website and you look through these simply on your phone, you'll see that this is definitely something that does look really, really good. And not only doesn't it just cover a base level of 2D images, it covers 3D images, it covers different styles, it covers quite a bit lot. So whatever tool they are using, I think that Google should release this. And I do think that if this does get released, it will be a product that people use literally every day. So with that being said, are you guys going to read the paper? Do you think this is going to be really cool? Are you excited for CVPR 2023? I think Google once again has broken the mold and shown us exactly what we didn't think we would see. Another AI tool that really just go ahead and changes what we thought possible. And if you enjoyed this video, it'd be great if you could subscribe to the channel.