Meta Announces the "World's Best" AI Video Generator. No Joke.

I know a lot of you are probably sick of hearing it. New AI video generation model changes the game, shocks the industry. But the thing is, is that it's true. If I had to sum up this year in AI, if I had to categorize it by one particular modality, it would absolutely be AI video generation. It has exploded this year and I feel like I say it all the time, constantly. I just did a video like 3 days ago talking about AI video gener generation news. There was a ton to talk about, there were new models to talk about, and new forms of research into video generation specifically. But then today, Meta comes out with this right hook and delivers what might actually be the best video generation model we've ever seen to date. Which is just insane because we knew Meta was working on video generation, but I didn't know it was this serious. And right off the bat, this cannot be a coincidence, that is Mu Dang, the first thing you see on the blog post is literally the world's most famous hippo advertising to you that moviegen from Meta is here. But in all seriousness, the quality of the video is pretty outstanding. The movement is supernatural and fluid. We have fish moving in the background that aren't morphing or doing anything strange or weird. The lighting is very much what I would expect for some underwater camera video. The reflections on the surface of the water are even more impressive in that regard. And also the flowers that are just floating on the surface daintily. And Mu Dang's movements here, him swimming around, paddling around through the water, also looks freaking fantastic. The anatomy here, and the consistency over time in the anatomy is gorgeous. So in their own words, it sets a new standard for immersive AI content. They claim that these are research breakthroughs and demonstrate how you can use just simple text inputs to produce custom videos and sounds, edit existing videos, or transform your personal image into a unique video. So yeah, we have a lot of features of already. So not only do we have normal text to video, but we can also get sound with these videos, edit videos that already exist, which is massive. The only one that I've seen do that is Pika Labs, and it wasn't too great by today's standards. And then of course, they have the ability to upload an image, which has classically been probably the most powerful feature in video generators. And we've got some pretty nice examples. This is a girl running across a beach holding a kite. She's wearing jean shorts and a yellow T shirt. The sun is shining down and it pretty much Gets all of this, right? The only weird thing I would say is maybe the kite is a little too low to the ground, but otherwise it's still a very watchable video. There's background blur again. The anatomy of the person running is very accurate. We have physics on the T shirt and the rest of the clothing and the hair. She's also kicking up sand with her shoes as she runs across the beach. It's really, really awesome. And it's also high resolution. This generates 1080p video, which is. I mean that's just the standard I upload in anyways. Sure it's not 4K, it's not movie quality maybe, but dang, it's. It's ready for Internet usage, right? I will say some of the finer details on the sand look a little bit wishy washy. They look a little bit conformed into each other. So that's something to note. Obviously it's been a problem for video generators since the beginning, but as far as all the generators go, the fine details on this thing look pretty much state of the art. I would say we've got another one here. And this actually has an image input for the character which looks crazy ass except for maybe the hair length. A woman is sitting on the grass of a pumpkin patch. She is wearing a scarf and holding a cup. The background is filled with rows of pumpkins. I mean, this one is nearly flawless. I can barely tell that it's an AI generated video. Except if you look at again the finer details, she's got like this big scarf and the scarf looks great, but you can just tell it's a little bit fuzzy. It's a little bit AI generated esque. We've got something that's a little bit more cinematic and movie like. This is just a straight text to video. THUNDER CRACKS LOUDLY with an orchestral MUSIC TRACK I think this one's supposed to have sound, but I'm not getting any sound through the web interface right now. If I directly save the video, I mean, not bad. It goes with the video. The lightning strikes actually happen and the sound effects happen at the exact same time. So pretty good. Although again, I would say it's a little bit cartoonish sound effects. It's not very realistic. Thunder and lightning still a huge leap forward. We haven't seen anything like this before. Before we dive into the rest of today's video, I do have a quick message from today's sponsor, which just so happens to also be Meta, but for Llama 3.2, huge thanks to Meta for supporting the channel. Again, this is all my own first reactions to this new Meta movie gen, but the Segment about Llama 3.2 is sponsored I thought it would just be easier to wrap all the meta stuff up into one. Meta has just released an update to the Llama collection of models, so I'd like to introduce all of you to llama 3.2. And first up, for those of you who aren't aware, the Llama models have grown in user base 10 times over the course of this year. They've quickly become essentially the go to if you want open, modifiable, cost effective and high quality AI models. Now first up, I want to talk about two models, the 11B and 90B models. We have significantly enhanced image recognition capabilities, meaning it's not all about taking in text, it's about taking in text plus images, which is by far one of the most useful things that a large language model can do. Open source large language model image recognition is a game changer. It's not something we haven't seen before, but we haven't seen it at the quality levels of the AAA big players. And finally for developers, that's here. I'm really excited to see what people do with this. Meta is Also Rolling out their first official distribution of the Llama stack basically makes working with Llama models easier. You have seamless integrations with the full suite of Pytorch libraries, allows you to more or less streamline your projects and get less bogged down with the setup. And honestly, what's double sweet is the fact that those Llama 3.2 models are available right now for download on hugging face and llama.com. they're also ready to go on Meta's extensive ecosystem of partners. Whether you use AWS, Nvidia, Google Cloud, Microsoft Azure, it's all there at this point. If you're developing a large scale user based app or just something personal, there's probably a Llama model for you. And that leads me into talking about the two final models. We've got an ultra lightweight 1B model and a similarly quite lightweight 3B model. Now obviously these aren't meant for complex tasks and are great for things like summarizing messages or giving you a basic rundown of your schedule. And they'll run on a large number of these little guys. As usual, Meta is truly, truly pushing the boundaries of open innovation in regards to AI technology. This is better for everybody, and I'm really excited to see what people are going to make with these new models. I can already think of a few simple Killer apps that people could make in like a few days. If you're a developer, get out there and create. Definitely. Check out Meta's resources below and get started today. And oh yeah, remember this video is brought to you by Meta. So huge thanks to them for supporting me and our greater community. Without sponsors like this, I would not be able to pursue this channel full time. Let's dive into the rest of the news. Welcome back folks. Now, editing videos with text again. This is like state of the art almost never before seen stuff. So we have an original video input here and we literally just give it a text input, add blue pom poms to his hands and it pretty much keeps almost the exact same video and just adds the pom poms. One thing that's very important to note here is that you don't want to edit a video over time continuously. The more edits you do definitely is going to have an effect on the overall quality and output. You can see in the original video. There's not as much contrast going on in the video. It looks a little bit more natural. And all of these edited ones, they have a little bit more contrast, they look a little bit more burnt, a little bit crispified. And that's because they are being AI generated. This is something that we also saw in the similar editor for Pika Labs when that first originally came out. But regardless, the editing still is really, really cool. We can see it definitely puts pom poms in his hand. We could turn him into an inflatable dinosaur running and we can also add a bunch of cactuses and put him in the desert. And that honestly to me is the most impressive part. We completely change his environment but he is pretty much exactly the same as before. So that's totally nuts. And they have plenty of examples to go through here. We have a VR headset. Obviously this is a Meta Quest 2 so they're kind of doing a little bit of product advertising. But yeah, there you can change it to some crazy binoculars. We can actually remove the headset entirely which is pretty insane. And I also think it's funny that you can kind of still see the strap holding her hair in there even though she's not wearing it anymore. And then obviously we can do some crazy futuristic bubble effects. It's just, it's crazy. This level of editing to previous clips that you've already recorded is very much unprecedented and I am shocked at this things abilities. We've got another one here. This is, you know, just a slow mo guy. If someone getting ready to work out we can give him fire sparklers in his hands. We can change the background to northern lights, or we can put him in a stadium. And the stadium, for me is the most impressive part because this actually looks like it could be a clip from a movie or something. Right. It's all blurred in the background. So changing something that simple actually isn't as big of a deal as you think. And it looks freaking good. All of this honestly looks very, very much good. We can change the poodle to blue. We can put him in this beautifully manicured royal garden. We can give him a little pink onesie, something like that. I mean, it's really, really insane. It's honestly just like this is one of those moments. You know, you have moments when you look at AI technology, you're like, damn, we're already this far in. This is not something I was expecting to see this year. This quality of AI video editing, it's. It's crazy that they're able to do this. I mean, yeah, like, like, can you guys see anybody else in the industry that can even come remotely close to this level of quality in video generation, as well as editing and changing aspects? And of course, as earlier mentioned, we do have the ability to do the style transfer, the transfer of people. Right. Consistent characters. So these are all real photos of real people and we can put them in the videos. And yeah, this is just examples showing this off. I think it does a pretty damn good job. It's obviously not going to be exactly perfect, but it definitely good enough where you can make some very fun movies with you and your friends and family and stuff like that at the very least. I also think that this is almost ready for some professional applications now. The sound effects are also like a whole nother cherry on top as well. This is like very much unprecedented. Along with the editing and the character transformation where, you know, we've got some like epic music right now. Definitely hear the waterfall. Overall, it sounds decent, I guess. All right, now I want to hear this one. This is a snake. Pretty decent. The snake looks a little whack because of those finer details. Yeah, we got some nice beats in the background for this one. Pretty decent. I think it looks a little bit awkward, especially with the wheels, but that's classically very difficult for AI video generators. We've got some skateboarding. It's crazy how it's able to go along with the video generation as well. That's like a mind blowing thing for me. I don't think we. We did see some similar technology a while back. This Year, maybe early this year, something like that. But I. It was not at this level, that's for sure. Okay, then we've got some penguinies over here playing with giant bubbles. Orchestral piece that evokes sense of wonder. And then we've got fireworks. Pretty good. All right, so they do have a research paper which is already good for the community. I like to see that. But obviously from Meta, my expectation is going to be open source since they have been such a pioneer in that specific area. Looks like it's not available just yet, which is pretty disappointing. At least they are, you know, taking us under the hood though, with the research paper. This is gonna be linked down in the description below. This is going to benefit everybody as well, but not as much as if it was directly open sourced and accessible for people. It looks like this is still in the research early alpha stages right now as we continue to improve our models and move towards a potential future release. Potential future release. Met. Oh my gosh. You need to open source this thing. Who are we kidding? This would be great. Work closely with filmmakers and creators to integrate their feedback. Well, you guys know already from today's sponsor that I have been working with Meta in some capacity. I'm gonna try to see if I can get access to this thing and help give feedback. Maybe I can give you guys a better sneak peek. They want to take a collaborative approach, ensure that they're creating tools that help people enhance their inherent creativity. Imagine animating a Day in the Life video to share in reels and editing it using text prompts. Creating a customized animated birthday. Greed. Yeah, we, we understand that this is powerful, exciting technology. I just want more than a research paper, Meta. I want to see this model. I want to use it. Well, the good thing is Meta has time on their hands to not take the route that OpenAI did with Sora and get everybody excited and then never freaking release the thing. I understand safety is a huge concern with these models, especially video generators that are this realistic. It's starting to get to that echelon where we don't really know all the time if it's real, especially if a video has been shared across and compressed on a bunch of different platforms. And you know, that's a, that's a concern. It's a reasonable concern. But there are already a lot of good video generators out there and so far, so good. Dare I say there are going to be bad actors either way. Not to mention Pliny is around, who is a notorious AI jailbreaker, and he's probably just gonna jailbreak your model right away. Anyways, anyways, the overall community is absolutely pumped for moviegen. It's being shared all around. Twitter X AK provides us with a couple of more details that come from the research paper. So obviously up to 1080p HD videos, which is insane. You do have different aspect ratios as well and synchronized audio seems to be built in. So their largest video generation model is 30 billion parameters, which is actually a little bit smaller than I thought considering the quality. But it's a transformer model trained with a maximum context length of 73,000 video tokens and a pair apparently that corresponds to 16 seconds at 16 frames per second frame generation. AI has come a very long way, so you could easily double that frame rate if you so happened to want to. But there is a bunch going on in this paper. There's multiple technical innovations and simplifications to the architecture, latent spaces, training objectives and recipes. Overall it's definitely yeah, very very good quality, right? This is awesome. Just freakin great job to the researchers I have to say, but we want access. General community reactions though, I mean if we scroll down here we can see, you know, best in class quality Meta has outdone itself. Definitely agree. Yes, this is a good point. Meta is presenting a research paper, not a specific model or service. Would be great if they actually fully released it. Some people say it looks better than sora and Google's videogen. Someone's asking if it would run on a 4070 to which I wouldn't be able to say necessarily. It's not open source right now anyways, so not like it matters. A lot of people seem to really be excited about the AI avatar feature and the ability to personalize them. For me, I'm probably most excited about the editing and just overall generation quality. There are some people who are already saying that it's not shipped out yet very much like Sora, so people are worried that Meta is going to take that same direction that OpenAI did and not really release the model. We'll see. There is still a lot of speculation that Sora could be shipped this year, and whether or not it matches up to current video generators is going to be another question. But if it is good and it does ship this year, that could push Meta to release this as well. And this seems to be very fully featured, probably more so than Sora. I don't know. Overall I am super excited. Tell me what you guys think in the comments below. Meta has a pretty good track record at releasing things open source and actually getting their AI products into the hands of people. My biggest fear is that they're only good at doing so when there's significant competition on the board. So Llama, for example, I mean, that's an LLM flagship model directly competing with ChatGPT, anthropic, etc. And the reason that that was open source and so accessible might partly just be because they wanted to use that as a strategy to be more competitive where the. The video generation space is a little bit different. This is very much best in class. They might not want to go with the same exact approach just because they're kind of ahead, possibly. I'm just shocked that so much in the AI video space has happened this year as a whole. It's crazy that we've gotten to this point. This is definitely industry leading. I appreciate that Meta is investing so much into AI and really appears to be an AI first company. They're releasing models that are second to none in a lot of certain circumstances. So huge props to them for that. And also thanks to them for sponsoring today's video and continuing to invest in the community. I'm really pushing or going to push for an open source release from them and see if I can get access and test this thing out for you guys so we can get a better glimpse at it. Maybe, maybe we can get some information on a release date potentially. I don't know. Let me know what your thoughts are and I will see you guys in the next one. Goodbye.