An EXPLOSION of AI Video is Happening Right Now

So the pace of advancement in AI video has just been mind blowing lately. Runway just rolled out their Motion Brush feature, which allows you to animate just specific areas of an image. They also added a ton of new options for motion and camera angles with the videos that you generate in Runway. It's pretty crazy, but it's also kind of mind blowing how easy these tools are becoming to use as well. So check out Runway real quick. I could upload an AI generated image like this one that was made in Leonardo here. Click on Motion Brush and let's just mask out the hair on this image. With just the hair masked out, I can give it some details of which direction I want the movement to go. Click Save. And then I have all of these camera motion options. I can have it zooming in and doing a clockwise roll at the same time. Click save, click Generate. And within about a minute or so, I turned this image that I started with into this animation where you can see some motion in the hair, but it's rotating counterclockwise as it zooms in on the face. And it took me less than two minutes to make. I can also do it with text prompts. For example, a young redhead woman underwater looking into the camera. Let's apply a 3D cartoon style to it. Let's add a pan and a tilt animation to it and just see what we get. We'll click generate and literally less than a minute later, we have this video. We have a cartoon animation of a red headed woman under water with her hair flowing and it's looking pretty, pretty good. I mean, yeah, there's some funkiness going on in the fish, but we've definitely come a long way with these AI animations. And on December 4, the day that I'm recording this video, Runway actually announced a partnership with Getty Images. It looks like this partnership is going to help enterprise users train their own AI video models. Says here. Runway Enterprise customers will be able to fine tune RGM using their own proprietary proprietary data sets. This enables companies in all fields, Hollywood studios, advertising, media, broadcasting and more to enhance their creative capabilities and provide new channels for video creation by powering entirely new content workflows and making it easy to craft delightful experiences tailored to enterprise styles and brand identities and their unique audiences. That's not a run on sentence. I don't know. It is. But anyway, RGM is referring to the Runway Getty model. So pretty cool that in the near future, we're likely going to be able to train our own AI video models tailored to our own needs. It May just start at an enterprise level, but it's only a matter of time before it becomes more and more accessible to everyone. We also got a brand new AI video model from Stability AI called Stable Video Diffusion. Now it's pretty much on par with what we're getting out of Pika labs and runways Gen 2 right now. But there's been a little bit of a viral trend going on with the Stable Video Diffusion where people, people are taking these popular memes and having Stable Video Diffusion animate them. Like this one where this guy gets real creepy real fast. Or this one of this little kid on the beach looking determined but then also kind of getting scary. And the girl in front of the fire here. We've all seen this meme. Well, here's what it looks like when it's animated. And this is just the stuff that's come out recently. There's a whole bunch of really cool new tech in the pipeline that's coming super soon. It's going to give us even more control around AI video and I'm going to show you all of this cool tech that's coming out. But first I want to tell you about the sponsor of today's video, which is TubeBuddy. If you're not familiar with TubeBuddy, it's an amazing tool for YouTubers to optimize their YouTube channels and they've been adding some really cool AI features. For instance, they just recently added a suggested shorts feature where it will help you pick viral YouTube shorts using AI insights. For example, here you can see my latest AI news video that I published last week. If I'm in my YouTube studio and I click on the TubeBuddy icon over here, I could click on suggested shorts. It's going to analyze this video within a few seconds. It suggested a clip from 7:57 through 8:12, which is me showing off the clip drop real time image generation. It's also suggesting some short clips from some of my other recent videos. For example, from 20 minutes and 36 seconds to 20 minutes and 51 seconds from my make a Viral movie trailer video. That's just one of the cool AI features TubeBuddy has. I also really like the thumbnail analyzer. I spend so much time working on generating thumbnails and this tool really helps me figure out how well a thumbnail is going to do. Same thing. I could come to my Studio page in YouTube, click the TubeBuddy button on the side. Click on thumbnail analyzer. You can see it shows a heat map of this thumbnail. I could drag another thumbnail right in here. It's going to analyze that thumbnail and actually predict which one it thinks is going to do better. It actually thinks this one is going to have a 6% improvement over my original based on the heat map that it's looking at. And then if I wanted, I could click the run a B test on thumbnails and split test the two thumbnails immediately. They've also got an AI title generator, but that is just scratching the surface of all of the tools this has for YouTubers. So if you're creating content on YouTube and you don't have a TubeBuddy account, make sure you check it out. I'll put the link in the description below. I use it for my own channel. I can't recommend it enough actually. One of the secret weapons to the YouTube growth that I've had so far. So thanks again to TubeBuddy. I'm sporting your jacket today. I really appreciate the sponsorship. Now I've got way more AI video tools to show you, so let's go ahead and keep going. Pika Labs, one of the more impressive AI text to video generators just announced Pika 1.0, which looks to be a huge leap in what we can do with AI video. Now we don't have access to it yet. It should be rolling out to some people this week, so I imagine I'll be able to play with it and show it off more soon. But just their demo video is pretty dang impressive. The videos that it's generating already looks better than what we're getting out of any other AI video model. But that's not even what makes it so impressive. What makes it so impressive is some of these newer features that they're rolling out, like the ability to upload a video, give it a prompt and change that video into a completely different video. Or like we've seen in Photoshop, Generative Fill and other AI image tools, the ability to out paint or essentially uncrop an image. So you have a one by one aspect ratio and you want to expand it. You can add to the video and it will fill in those extra gaps in the video with footage that looks like the surrounding video. But what's most impressive from Pico 1.0 to me, is this new sort of inpainting feature where you can take a video and select just an area of the video and change things like the color or add glasses to the monkey in the video. Again, it's something we don't necessarily have access to yet, but it should be rolling out really soon, like this month. Now. Speaking of having more control in your video, here's some research that recently came out called video booth diffusion based Video generation with image prompts. What's cool about this one is you can give it an image. Combine that with a text prompt like dog drinking from a bowl of water. Dog swimming in a lake happily, dog in a park, portrait of a dog looks out the car window and it keeps the consistent dog. All of these are videos here with this consistent dog from the original one. So we can now get very consistent characters. That's been one of the issues we've had with these text to video models so far is inconsistencies in the character from one scene to another. They always kind of look a little different. Now on this research page, all of the examples they give are animals. Like here's one with a cat, a cat sitting on a white background close up of a cat on top of a vintage chair. A cat in a garden. And they're all the same cat. Here's another example of this cat all being the same cat. There's one of them, a horse, where you're getting the same horse every single time. Here's one of a panda, where you're getting the same panda in every single video. And here's one of an elephant, where you're getting the same elephant in every single video. What I'm not seeing on this research paper is whether this actually works with human models. I'm imagining you can upload a picture of yourself and then give a prompt of like person drinking water, person in a garden, person running. And it will use the same face. But they don't have any examples of that on the research. So I'm just kind of speculating that we'll be able to do that. But I imagine if it works to get a consistent animal character, it'll likely work to get a consistent human character as well. If you've been on AI Twitter lately or AIX lately, you've probably seen some of this research going viral called Magic Animate. Temporally Consistent Human Image Animation using Diffusion Model. What's interesting to me about this one is that it's created by the National University of Singapore, also with ByteDance. And if you're not familiar with ByteDance, ByteDance is the company that owns the TikTok. They also own the editing tool Capcut. So if you're wondering if the future of TikTok is going to have AI dancers in it, I think you might be getting your answer in Some of this research here, but basically what this is is you can upload a reference image, upload a motion video and then it will actually animate the person from the reference image in the same way the motion shows. And here's a few examples. You've got another one here with this girl getting animated in this way you can see four different examples. And then down here you can see comparisons of other similar tools. On the left you got the reference image. On the very right you've got the output from this tool and you can see it's dramatically better from all of the previous options that were available to do something similar. As I always do, I will link up the research below so you can see some more of these examples for yourself. But it is pretty impressive from these demonstrations here, right? You've got the Mona Lisa, they showed it running animation. And here's an animation of the Mona Lisa running. Here's Wonder Woman running, a running animation and an animation of Wonder Woman running. But what's really cool is here's an image of Wonder Woman running and then like a yoga pose sort of thing. And then here's an animation of her doing that yoga pose. Same with the Mona Lisa up here. They were able to get the Mona Lisa to do this yoga pose by uploading this reference video. And there's all sorts of interesting examples here that you can see and get an idea for what this is capable of. But what's even cooler is, does actually have a hugging face space where you can play with it yourself. What I found playing with it myself is these are probably cherry picked examples of some of the best outputs they're getting out of this model. Because when I tried it, I uploaded this image of myself. You can see here's the little animation video that was there that I uploaded. And this is what it put out on the right here. Kind of put me in like a romper, one piece kind of thing. And it also deleted my beard. I mean it nailed the pattern of the shirt. And I would actually argue that it got my eyebrows and nose right, but it got rid of the beard and made my shirt just my full body outfit. But it's really simple to use. You upload a reference image, you upload a motion sequence and you click animate and it will try to reference the motion with the image and animate it and you get stuff that looks like that. Now I just took one of the motion sequences that they already had available for me right here and use that motion sequence with this pose here. And this was the result But I also wanted to try it with a AI generated image. So this is an image that I generated. I believe this one was done in mid journey. And then here's another animation right here of this sort of dance going on. So this image, this dance, and this was the output that I got. So as you can see, doesn't quite live up to the references from the example page. Here's another piece of research called Magic Dance, this time from the University of Southern California, also in collaboration with ByteDance, the owner of TikTok and Cap Cut. And this one's very similar, just a little more focused on dance it seems. But also makes this one interesting is that if you look at that sort of subtitle here, realistic human dance video generation with motions and facial expression transfer. So if we look at some of these examples here, you could see the source video of someone dancing and then it extracted this little outline person here, but it also tried to extract the facial expressions and then they uploaded this reference image of this girl here with the glasses. And you can see she's doing all of the exact same poses in each scene, but she's also got the same similar facial expression, same with this sort of cartoon drawing of a man here. Same poses, similar facial expressions throughout. Here's another example where you can see the condition. Down on the bottom right is this research's output and you can see the pattern that was generated. This one is GT that means ground truth. This was the original video input essentially. And you can see some examples of it being recreated down here. Here's some more results using some different cartoon images. And then here's some more results where you can see the animation here and then the reference images here. Now this one, if we click on the code page and head over to GitHub, you can see that the code isn't actually available yet. So this isn't one that we have access to play with ourselves. But the one that I've been seeing go most viral and get shared around X the most recently is this Animate anyone, which is another similar idea to the last two that we saw, but looks even better from their examples. Now I don't know if these are cherry picked or not. If we click to go to the GitHub code page, you can see the code's not available on this one yet either. So we don't have access to use it. But the examples that we get on this one are pretty crazy. You can see, here's one of the examples. This is the reference image over here on the left of this woman that they uploaded. Here's the driving pose. This is the sort of sticker figure animation. And then here's three variations. You can see dreampose, bdmm and then ours. This is the animate anyone version here. If we go ahead and play this here, you can see how well this adheres to the initial image and the pose. This one over here, she didn't even turn around when the pose turned around. Here's another one. Look how consistent the dress is on the right where these other images, they can't keep it consistent. Same with this one. The dress is changing like crazy here. This one just looks like an actual video. You wouldn't even know another one. Here's the reference image. Here's the dance. Here's the output on the very right of the girl dancing around. Here's another reference image with this girl on the left, the driving pose. And then the animate anything on the very right here. And one more example, reference image, the driving pose. And then animate anything on the very right here. And you can see how good these come out. And there are just a ton of examples. Here's the reference image on the left of a girl standing there. Here's the sort of motion that they tell her to do. And you can see on the right as the motions actually happen, the image animates in the same way that we're seeing the stick figure drawing being animated. Here's Messi here of just an image of him and now he's posing in the same way so we can animate pretty much anyone. The implications are cool, but also I guess, kind of scary because I feel like we could take any image and make anybody do anything. I'm not going to go too deep onto the implications of that, but I'm sure you could use your imagination of where that might lead to. But just tons of interesting examples of this in use. Like an image of Iron man and then here's a dancing Iron man and it looks so dang good, like it is crazy. But again, these could be cherry picked. We don't have access to it yet. And all of this just shows how much more control we're about to get over our AI videos. Between Pika 1.0 having both inpainting and out painting and a video to video feature having tools where we can upload a reference image and then get a video that uses that same consistent character in every video. Having the ability to upload motions and then have any image follow that motion pretty closely and pretty dang realistically as well. And to further extrapolate this, that also means it's only a matter of time before you're just giving a single prompt and generating full on TV shows or full on movies. Now, what if you don't have footage for a dancer or a runner or something like that, and you want to create a driving pose for your character, and maybe you're too lazy to go shoot video for that driving pose? Well, there's a really cool tool that I came across called Ocean, and as of right now, it's free. And what it does is allows you to create animations of a 3D character through a text prompt. At the moment, they just have text to motion, but in the future, they're going to have video to motion and motion to motion. Now it is only in discord, but if I pull up the discord here and show it off a little bit, you can see some of the stuff that other people have created. Like here's a video of someone swinging a katana, right? The one on the left looks like they're swinging a katana. I don't know what they're doing on the one on the right. That almost looks like they're conducting a symphony or something. But look, you've got the nice animation of somebody swinging a katana from literally the text prompt swings katana. Here's an animation of somebody throwing a dart from the prompt, throws a dart. Here's one where the prompt is doing gymnastics and you can see the character does a cartwheel here. And we have a full 3D animation that we as an FBX file or a movie file. And again, this is free. I can just come here to the motion Discord. Type motion. Give it a prompt of twerking. Let's use the character bot. And in place, I'll just set that as true. Let's see what we get. It took less than a minute to generate, probably 25, 30 seconds. And here's our twerking animations that it gave us. The one on the right looks a little more twerky than the one on the left, but we've got some dancing animations that we just created in about 30 seconds totally for free. And now I've got some driving animations for any of these other animation tools where you upload an image and a driving animation. Again, this one's called motion.com and as of right now, can't guarantee it's always going to be this way, but as of right now, it's free to use. Check this one out. This one's called Story to motion synthesizing infinite and controllable character animation from long text. This one is another one where the code doesn't seem to be available. There's no demos to use. The first time I saw this, I almost thought it was too good to be true. As of right now, while I'm recording this video. So I'm still not even convinced that this is 100% real. But take a look at this demo here. They actually use ChatGPT to provide a series of motions for a character. So chatgpt suggested motion one. Dancing in front of the modest buildings. In front of the modest building. 7.5 seconds. Dancing around the trees, amongst the trees. 7.5 seconds. Ordering and savoring a mouth watering hot dog at the food cart. Five seconds. Jumping over a puddle, over the puddle on the sidewalk. Five seconds. Raising her right hand and offering, offering a friendly wave, standing nearby. 2.5 seconds. So check out what this generated here. Ocean dancing in front of the modest building. 7.5 seconds. He's dancing in front of buildings. Next walk in front of the modest building amongst the trees. She starts walking through the area. I'm assuming this is taking 7.5 seconds here. Next, dancing amongst the trees. Duration 7.5 seconds. Now she's dancing. Now it's not amongst the trees, but the next motion. Walk amongst the trees to the food cart. And then you can see she's approaching or he or they are approaching the food cart. I guess it's a she. Now eating hot dogs at the food cart. 7.5 seconds. You can see the animation of them eating at the food cart. Then walk at the food cart near the puddle. And the character is walking. We can see a puddle, walking, walking, walking. As they get to the puddle, the animation then suggests jumping over the puddle, over the puddle for five seconds. Now they're sort of tiptoeing, waved right hand, standing nearby. And that was all from this sequence of prompts that they had Chat GPT generate. Now I will link this up. You can see how they got Chat GPT to generate it in this style. They had this big long user prompt and Chat GPT sort of followed along to their prompt and then generated this prompt prompt to give this tool. But then it generated this whole sequence of scenes from a giant long prompt. This whole sequence of scenes then happened following on from that. And again, this one almost feels too good to be true to me. I can't wait to get my hands on it and actually see it. This is a. I need to play with it to believe it. But this is insanely impressive if true. This one's called Story to Motion. It'll be linked up in the description. Next up I want to show you scene or sign. I'm actually not sure how to pronounce this here, but Short to long Video Diffusion model for Generative Transition and Prediction. Now we can see here the image to video generation. We've got an input image of, you know, cars in the snow and it just kind of generates a sort of three second clip like what we'd get out of Gen 2 or Pika or stable video diffusion. These aren't super impressive over the existing tools that are out there. What's really cool about this specific project is the ability to transition between two different scenes. So here you can see a scene with spider man on a train and then sandcastle or somebody made out of sand or something. And then if you watch the animation, you can see it transition pretty smoothly between the two scenes. Here's another scene with clouds and then a scene with like farmland. If you look at this, you can see the clouds kind of start to part and then you see the farmland underneath. Here's another one with these blossoms and these trees. And then it's actually showing three different variations. Actually quite a few more variations than three, but we'll just take a peek at three. You can see this one how it sort of merges into the trees. Here's one where they sort of fade to the trees. Here's one where the sunlight sort of blurs it out and then it goes to the trees. Here's one with a raccoon playing guitar and a raccoon playing trumpet. And three transitions between the raccoon. Here's another one where the raccoon just kind of pops over him. Here's another one where it looks like he trades instruments. And this ability to transition between scenes and make it look like a fairly smooth transition is something really new that we haven't seen yet. So if you are making a longer form video, you can make it more smoothly transition between scenes without it having to be such a harsh cut in between scenes. Just another little tool in our arsenal that's going to make it easier to control the final output of any AI video we want to create. And then finally, I want to show off a few cool videos that I've come across from other people recently. Martin Harlan, if you're not following him on X, he's definitely worth following. He shares a bunch of really cool AI generated videos. And here's one that he created recently where you can Kind of see the before and the after. So if you look at the top of this video, you can see how he was actually posing for the video. And then the bottom, you can see the result of it. So check this out. He's holding the stick with the newspaper on his head. And now he's like a French soldier holding up a Frosty's box. Now he twisted the hat and he's become somebody else. Now he's cheersing somebody and passing a drink and I don't know, skydiving or something. Now he's at a picnic and now he's back to the French military person. But it's really cool to see his process of how he made all of these as he made them. And he used Gen1 to do this, which is a video to video sort of model where you upload a video and then you can kind of reskin that video and make it look different. And probably the most impressive video that I've seen recently is this one here from Nick, Nathan Shipley. And same kind of thing. Up at the top you can see the final result. On the bottom you can actually see the input that was used. But if you pay attention to this top animation here, it's a really, really impressive, coherent cartoon that looks like the same style throughout. So somebody's picking leaves, a kid looks up at the sky. I'm sure that that little shooting star was added in with, you know, after effects or something like that. That same with this animation of the smoke. I think that's like an after effects or something like that. But if you look at the cartoon on the top, it's a very consistent looking cartoon. The style, the character, everything they're doing in this cartoon looks ultra, ultra consistent. And you can see every step of how this whole video is being made down on the bottom. Super, super impressive. Once again, this is from Nathan Shipley on Twitter. I'll link straight up to this video so you can check it out yourself. But again, I just wanted to make a video to show you all of the crazy stuff that's happening in the world of AI video. It is starting to blow my mind how quickly the pace of these advancements are picking up. We are literally getting to a point where anybody, despite the skill level they have in video creation right now, is going to be able to create really cool looking videos. And I'm excited to put more and more of these tools through their motions. Last week I made a video where I made an AI video trailer. I mostly used gen 2 and 11 labs and a few other tools and literally the day after I released that video, more cool AI tools were announced like Pika 1.0 and some of these other tools that I showed you in this video. So now I'm going to give it a couple of weeks, but I'm probably going to do that same video again but use the more advanced tools because it's just moving so fast and the tools are getting so much better. And if you want to stay in the loop with all of the latest cool AI tools that are available for you to use, make sure you check out futuretools IO. I curate just a ton of AI tools on this website. You can even click on on video editing and text to video and you can see 73 different tools that use AI to help with both text to video and video editing. And you can even sort it by Most Upvoted and see the tools that people like the most on the site to create AI generated videos. I also keep the AI News page up to date on a daily basis. So if you're interested in all the latest AI news, this is the place to check in on a daily basis. Because I'll keep you in the but if you don't want to check in on a daily basis, join the free newsletter twice a week. I'll send you an email. One email will be all the coolest tools that I found throughout the week. The other email will be all of the most important news that I think you should know about for the week. It's totally free. You can find it all over at FutureTools IO. That's all I got for you today. Video is getting crazy. I'm excited about it. I can't wait to play with more of these tools. I'm happy that people like you love nerding out about this stuff with me. I'm having a blast talking about this stuff and playing with these tools. Thank you once again to Tubebuddy for sponsoring this video. You guys are awesome. It's one of my secret weapons for growing this YouTube channel again. Link for TubeBuddy is in the description. Thanks again to them and I really appreciate you. I'll see you in the next video and we'll nerd out some more. Bye bye.