What This AI Can Do Will Blow Your Socks Off

So when it comes to cool AI tech and research of things that are sort of in the pipeline but haven't really been released yet, or that maybe most people don't know about, I love to save up these kinds of things and then a couple times a month make a video breaking down all of the coolest AI research and tech that I've come across recently. This is one of those videos and I've got some awesome stuff for you. Like video outpainting. Some of the most impressive text of video I've ever seen, putting cool 3D robots into scenes like this, and much more. So let's get into it. For this first one, I need to thank RetroPunk AI for bringing it to my attention. It's called Magic Edit. It does things like video stylization, the ability to edit what's actually in the video, what they call video Magic mix, and even video outpainting. So taking a peek at those categories, here's the first example where it took a video of a monkey, gave it the additional text prompt of a monkey playing on the coast sea. And you can see it took pretty much the same video with the same motion and put a C behind it. And there's no real flickering or weirdness to it. They also use the prompt a teddy bear eating apple green grass with flowers. And it made this little teddy bear animation. Here's another one with a dog where it turned it into a cat with still kind of dog ears, but like the body of a cat. Here's one with a source video of a seal and it turned it into a husky and a sea lion. So it almost looks similar to what you can do with something like Runway Gen 1, but almost to this whole new level. Here's some examples of scenes. I love this one of the motorcycle and then they put the motorcycle as a submarine underwater. Here's some where they completely restylized humans. Here's the source video. A pretty girl, pink dress in the living room. Here's one with a pretty girl, white dr, dark hair, flowers. It also does local editing where you can take videos and just change little elements of the video. This is similar to like inpainting with an image, but now with video. So this one has a woman actually pouring a smoothie. Here they changed it to a handsome man and pretty much everything stayed the same, except now it's a man's face. Here's a source video. They put the prompt wearing sunglasses and it looks like almost the same video just now with sunglasses. Here's one where they Changed a woman to a senior lady. Here's one where they changed a senior lady to a young lady. Here's one where they changed a man to a pretty girl. But I think this is the feature of this research that's the most impressive to me is this video outpainting. So we've seen tools like Midjourney have that zoom out feature and we've seen Photoshop generative fill where we can uncrop essentially where we zoom out and it fills in the rest of the details. With this Magic Edit model, we'll be able to do that with video. You can see here's a source video of a man running, but the legs are cut off without painting. It actually figured out what was supposed to go below the video and add that in. Here's a source video of a dog walking around and it filled in above and below the dog. A woman at the ocean at like sunset. It filled in the rest of the details above and below her head. Here's another one with like a bald eagle with the ocean in the background. And here's one with a bear walking and it filled in the rest of the forest details. Here's some examples of some different aspect ratios where here's a smaller square image and they sort of zoomed out on it to get a larger video. Here's a square image where they went wider with it. Here's one where it gave them shorts and here's one where it gave them long pants. So this research again is called Magic Edit. Now if we click on the code here to go over to their GitHub, it doesn't look like the code is available yet, but this is something I'm really excited to get my hands on because we'll be able to go and generate text to video or image to video using something like Runway gen 2. And then using tech like this, we'll be able to do in painting with things like glasses out painting to expand upon what's in the video itself. So this will be really, really awesome for creators. It's finally available. Speaking of video, here's another piece of research where the code actually is available over on GitHub and they even have a Google Collab available where you can mess with this in the cloud on Google's servers. Their project page is pretty cool too because they've got this little slider where you can see the videos before they've been affected and after they've been affected and slide it around to see the difference. And you'll notice there's really no Flickering or weirdness from the video? I mean, hard to say there's no weirdness because most AI generated videos are still a little weird. But you can see there's no flickering and it's fairly coherent. What we're seeing here's some other video to video examples. And then they have more sliders down here. Like on the left you can see this is the original video that was taken on the right. They entered the prompt Chinese ink painting and it turned this video into this video. You can see when I move this slider around, what it's doing is it's looking for all of the stuff that's static in the background and it's looking for the things that are moving inside of the image and then it's essentially separating them so that it only focuses on processing the non static stuff in the video. The outputs we get from something like this look very similar to what you might expect from something like Gen 1 from Runway, but they're so much smoother. Because it's not necessarily trying to process everything in the video, it's only trying to process the moving parts of the video. Staying on the theme of AI video, here's a research paper that came out recently called Dual Stream diffusionnet for Text to Video Generation. Now, there isn't a specific project page around this research paper, but I did find some video outputs that were generated with this research paper. In fact, if we look at some of these videos like this panda bear here and this couple dancing and this girl skiing, and we look back at the research paper, it's the same images. Here's the couple dancing, here's the girl skiing. Scrolling down the paper a little bit, we can see here's the panda bear. The video images that we're seeing on this page are the result of this research paper here. This dual stream diffusion text to video generation. The two are dancing sideways, black background around modern dance, blue, black tone. The result of that prompt is this video right here. Here's another one. A girl is skiing, black dress, blonde hair. That's the entire prompt. And we get this video from it right here of a girl with black dress, blonde hair, skiing. This prompt, a panda is walking down the street. Gives us this panda walking down the street. A horse is galloping on a street. Here's our horse galloping on a street. A man is dancing in the rain. Here's our man dancing in the rain. A girl is dancing among leaves with curly hair. We get this video from that prompt. Now the way this model works is it actually runs two processes in parallel. Essentially what it's doing is it's generating a whole bunch of images from the prompt that you gave it with the motion that you asked it for. And then the second process that it's running is trying to smooth out the sort of animation in between each frame so that you don't get all that flickering and weirdness that you get from other text to image models, or even some of the other video to video models that are out there. And the end result is probably some of the best pure text to video I've seen so far to date. Now, again, this one doesn't actually have a project page yet where we can view or get access to code or anything like that. Once this one is available, I have a feeling a lot of video creators are going to start using this in its pipeline. Because if I'm being honest, from a pure text to video standpoint, it seems to be doing a lot better than something like Pika Labs or Gen 2 right now. Saying that some of those tools will probably use some of this research in their back end and you'll be able to get videos like this from something like a gen 2 or a pika Labs. Now let's talk a little bit about AI image generation. It's crazy how far we've come in the last year with AI art just looking at midjourney alone. These are the types of images I was generating back in July of 2022 when I tried to give it the prompt of someone combing a mustach with a fork. Fast forward to now and we're getting stuff like this. So it's kind of hard to imagine what the next iterations of AI are going to be. But I recently came across this research called Concept Lab Creative Generation using diffusion, prior constraints. And what this is is it tries to generate things that don't exist. For example, new pets generated using Concept Lab. Each pair depicts a learned concept that was optimized to be novel and not match existing members of the pet category. Running our method with different seeds allows us to generate a variety, a variety of different brand new concepts. So these are all animals that look like they could be pets, but they don't actually exist. In these examples, the little S star represents the thing that is going to generate. So a professional high quality photo of A S photorealistic 4K HQ. And you can see it generated some completely new creatures. A watercolor painting of this creature, a beanbag chair, a plush toy. You can see it's using the same creature for all of them, but it's A creature that doesn't actually exist. A new type of reptile, a skeleton of that reptile, a plush of that reptile. Now, it's not just trying to generate novel things like a new fruit, a new animal, a new superhero, or a new building. It's also able to generate new art styles. A dog in the style of new style. A horse in the style of new style. You can see all of these images are sort of matching the same style, but it's a novel style. It's a style that doesn't exist yet. This isn't modeled after any specific artist. This is a completely new style here that this generator dreamt up. Here's another style. The idea behind this was to create an image model that doesn't just mimic what already exists like most other models out there, where if you ask it for a dog, it knows what a dog looks like and creates a dog. This was designed to create something new and to let the AI actually be creative and generate something that nobody's ever thought of or generate a style that nobody's ever played with before. So one of the places we go with AI art from here is allowing these AI art generators to actually be creative and dream up new ideas that humans haven't come up with yet. Speaking of AI art, I want to shout out today's sponsor, which is wirestock. If you're not familiar with wirestock, it's a single platform where you can upload images and it will distribute them to all of the various stock photo websites where you can sell your photos. And many of the stock photo websites today now allow AI images. Sites like Adobe stock 123RF, FreePik, Dreamtime, Imago, and more. Now, in a previous video, I showed you how you can use the wirestock Discord bot here. If you click on More, go to Apps and click on Upscale, you can upscale the image. Click Publish on wirestock and it will directly submit this image to the various stock photo platforms. Or you can submit this to one of the wirestock challenges, where you can earn cash prizes for your generations. Well, Wirestock now just rolled out their own art generator. So even if you don't have a subscription to something like Midjourney, you can use their Discord bot to generate images and then submit them directly to wirestock. And they actually partnered with Stability AI so that when you generate images, it uses Stability's SDXL model. So with the wirestock Discord bot installed, I can do Slash Imagine. And you'll notice I actually have a Wire Stock Imagine option when I select that option here. I can give a prompt, a monkey with a top hat on the beach. And then we can select the orientation. Let's go ahead and do horizontal. We can select a style for this one. Let's do digital art and submit this and see what we get back. We get a couple images of a monkey and a top hat on a beach. I think I like this top right. Let's go ahead and upscale number two. And now that it's upscaled, I could publish straight to Wirestock or participate in one of the challenges. No midjourney subscription required. You can install this bot directly in your own private Discord server. And if you do decide to upgrade to a premium version of Wirestock, make sure you use the coupon code Matt to get 20% off your wirestock membership. Thanks so much again to Wirestock for sponsoring this video. Continuing on with the theme of AI imagery, we're actually about to get a lot more control over what we generate with this research here called IP Adapter Text Compatible image prompt adapter for text to image diffusion models. They never make those names easy. Now this is designed to get better image outputs using images as part of your prompts. This model can actually work with any other fine tuned model. So for example, with one of these image prompts, using these various models, you can generate images that look similar to your original image prompt. So for example, this image of a woman up here, regenerated with SD 1.5, you get this with realistic vision 4.0, you get this anything v4 this and so on. Here's a stained glass window of a dog. You upload this image, here's what it looks like when you put it through any of these other models here. And you can see all sorts of other examples where you give it a source image and then the model will generate something that looks similar to your original image, but in the style of that model. And it's fully controllable with things like control. Net. So check it out. In this one, for example, they gave this image prompt of this famous wave photo here. This looks like a depth control map here. And by blending these two, you get this photo of a woman looking at the camera, but in the style of this wave. Here's another example of this line drawing image prompt conditioned with this with control. Net. And now you get the same pose but with the style of the original image. Here's the starry night painting with what I believe to be a canny control. Net image here. You combine them and you get the same outline to this Original image but in the style of Starry Night. And there's all sorts of other examples. Here's one with open pose. This is the original image. Here's the open pose updated. Now it repose this same character but with the open pose generator. So many people always ask about how do I get a consistent character with AI? Well, this is going to help you do that. You can upload an image of a character and then use something like open pose or this open pose with the face and the hands here. And you'll be able to generate these characters in whatever pose you can imagine. If you check out IP adapter GitHub IO, you can see all sorts of examples of this in use. They even used it with inpainting. So they took this image here, masked out the area of the face and then uploaded this image as the image prompt with no extra text prompt. And it replaced the face from this image in the gap where they inpainted with the face from above. They did it with this line drawing as well. Mask this off with inpainting and something like automatic 1111. Upload this as your prompt instead of a text prompt. And it replaced the face in the image with the style of the line drawing. Here's one where they uploaded this image and then gave image prompt with it of this little like island image and it made a sort of composite of it. Here's one where they uploaded some rocks as the image prompt and it made a example that sort of blends the rocks with this one. It's also multimodal so you can give an image prompt with no text, an image prompt with text like wearing a top hat or a red horse to get an output. So this is some pretty exciting extra controllability we're gonna be able to have in tools like Stable Diffusion. Now if I click on the code here, it is available on GitHub. So this is one that I wanna play around with and I'll make a tutorial on this in the future of how to actually use this once I figure out how to use it myself. One interesting thing to note that I just noticed while Looking at this GitHub page is one of the contributors is S X E L A K A Alex who is the person behind Stable Warp Fusion. So pretty cool. He's got his hands in a lot of cool projects. Here's something else I recently came across from Javi Lopez here all about 3D Gaussian splatting for real time radiance field rendering. So I've talked quite a Bit about nerfs in the past, or neural radiance fields, which basically you go and take a whole bunch of images or a video of an area or an object, and then all of those images or that video footage is sort of stitched together to create a 3D version of that area or object. Well, 3D Gaussian splatting does something similar to that, but not exactly like a Nerf, without getting too much into the weeds. Basically what it's doing is everything it's seeing as you scan it turns them into, like digital clay. And so as you're scanning stuff, it's taking what you're scanning and essentially virtually molding it with this digital clay to represent what you're seeing. And then the position, the size, the orientation, the colors, everything that we're seeing in those images are essentially overlaid over that, like virtual clay, to then represent what we saw, but in 3D. And that's an oversimplification. I might have even made it more confusing than it really is. But it's actually a much faster method than what nerfs can do. And what it generates are actually much cleaner generations that look more realistic than what you see from a Nerf. In fact, they've got some of these cool sliders here where we can see what it generates. When I slide this to the left here. Here's an image of a bike using a Nerf. Here's what it looks like using their method. Here's one of an actual photo and what the photo looks like. And here's what the generated model looks like. It got rid of some of the reflections on the windows, but it looks pretty dang accurate to real life. All said and done, done. I'll link up the research page below this video if you want to dive deeper into this and check it out. And from what I can tell, it seems to be a more efficient, more accurate, more realistic version of what previously we were getting out of nerfs. So it's exciting to see where this is going to go. And as of right now, the code is available on GitHub. So it's something that people can use and mess around with. Right now, personally, I'm waiting for a hugging face space or a Google collab so I can mess with it in the cloud. But if I can find the time, I will probably try to download it and see if it can be run on my local machine and see what we get from it. Now, speaking of 3D Gaussian splatting, there's this really cool tool that I'VE been seeing on Twitter that I don't have access yet. It's in Waitlist right now called Simulon. And basically what Simulon does is it allows you to select a 3D object, and then once you've selected that 3D object, select where that object is placed in a room. And then in augmented reality, on your phone, actually see that in your world, and actually walk around and look at it as if it was right there in front of you. Now, what this video is that's been circulating on X recently is somebody used Simulon to place this 3D mech into their room. And then once this 3D mech was in their room, they actually filmed all around it and created a Gaussian blur of it, kind of like you would with a Nerf. If you walked around an object to create a Nerf and it generated a 3D version of that, they did the same thing. They just did it with this newer technique of the 3D Gaussian splattering. And now we have this 3D scene that you can move around, and it looks like that mech's actually in the room, although that mech was just there in augmented reality. And here's another example that was shared by my friend Linus here. I'm not a hundred percent positive, but what I think's going on here is somebody generated a 3D animation. So that character sitting on the chair, looking around, that was a 3D rendered animation. They use Simulon to pull this animation into their real world. And then they were able to use their phone and walk around as that character was being animated. And as they're walking around it, you can actually see that the light in the room is affecting the robot. So that is absolutely insane to me. Again, this Simulon tool, it's in wait list. I personally signed up for the wait list. So hopefully I can get access to it and play around with it and make some more videos about it. But I'm pretty excited to get my hands on it. Now, in one of my recent videos, I talked about this tool called Suno AI, which is a tool that allows you to generate actual songs with AI directly inside of Discord. I actually generated a heavy metal theme song for this YouTube channel. Now, it's not available yet for everybody, but I did get access to a private alpha of the next version of this. And the next version of this tool actually allows you to choose the style of music. If you remember from my previous video, you either plug in song lyrics or you let chatgpt generate song lyrics for you and then it just created a song in whatever style it wanted to create it in. Well, this new alpha version that's not yet publicly available lets you actually choose the style of music you want to generate in. So now I get this prompt box that says describe the style of music. So if I put hip hop and then have ChatGPT generate lyrics for me, I can do a hip hop song about why you should subscribe to Matt Wolf's YouTube channel that's all about AI let's click submit now. It generated two versions for us, so let's take a look.

Listen, yo, listen up I got something to say about a YouTube channel that's causing a craze Matt Ro spitting knowledge in a whole new layer AI is a topic it's time to amaze subscribe to Matt's channel Come on, let's go unlock the secrets Let your knowledge grow AI insights He's got all the info subscribe to Matt's channel It's a must.

Follow and not too bad let's listen to the second version yo, listen up.

I got something to say body YouTube channel that's causing the crazy Matt Wolf Spitting knowledge in a whole new way AI is the topic it's hard to amaze subscribe to Matt's channel Come on, let's go unlock the secrets Let your knowledge grow A insights he's got all the info subscribe to Matt's channel It's.

A must follow and I actually got to choose the style of music this time. So if I wanted, I can use the exact same lyrics. Copy those, Use the same prompt again. This time let's go ahead and make it heavy metal Enter are the same lyrics again.

A whole new way AI is the topic it's on the maze subscribe to Matt's channel Come on, let's go unlock the secret section on that scrum AI Insight he's got all the info subscribe.

To Matt's channel that's a so again, that's called suno. You can find it over at SUNO and jump into their discord because they do have a version that anybody can play with right now. It just doesn't have the ability to select your own genre yet. And then finally, I want to wrap up with this fun one that I came across, which is a new tool that allows you to actually have your ad reads done by AI if you're a podcaster. Here's one for this week in startups with a commercial for Duff Beer. Let's take a moment to thank our sponsor Duff Beer. Now, if you're anything like me, you know that running a startup can be a 247 endeavor. It's passionate, it's exciting, but let's face it, it can also be incredibly stressful. Now if we take a look at this here, they just entered the sponsor, Duff Beer. The base copy. As a startup founder, you need to relax sometimes. Get a six pack of Duff Beer and mention Twist at checkout on your next order. And then the reader was Nick Calacanis. The approximate duration and then it just generated and read the entire ad read. And this is something that's actually available right now in this tool that I just came across called Podcast AI. So something fun that you can try. It's a credit based system, so I don't really know what a credit gets you, but seems fairly pricey. I don't know, something I'll have to look into more. I just thought that was a cool feature. You can also do this with 11 labs. Have ChatGPT write the ad copy for you, copy and paste it into 11 labs and it seems like it would be sort of the same outcome. Just something I came across right before hitting record. Thought it was interesting and thought I'd share it with you. So that's all the cool AI tech stuff that I've come across over the last couple of weeks that I saved into the MyMind app and have just been sort of piling up because I wanted to share them all with you in one video. I'll probably make another video like this in a couple weeks with all the cool new AI research and tech that I've come across in the next couple weeks. So if you haven't already, make sure you like this video, subscribe to this channel, and if you really want to make sure you see my videos when they come out in the future, make sure you click the bell button because that's really the only way that's going to make sure that you see the videos. But whether you do or not, I still appreciate you watching this video. So thanks so much for tuning in. If you haven't Already, check out FutureTools IO. This is where I share all the cool AI tools that I come across every day, share all the AI news, and then I've got a weekly newsletter where I give you the t tldr of everything that happened in the world of AI for the week. So check it out over at FutureTools IO. Thank you again for tuning in. I really, really appreciate you. And thanks once again to Wirestock for sponsoring this video. I'll see you guys in the next video. Bye. Bye.