Even More MASSIVE Video AI Upgrades & New Models!!! it just does not stop!

You won't believe it guys, but there are even more video generation updates and they're good. They're incredible actually. Today's video, we're starting off here. This is spatiotemporal skip guidance for video diffusion models, AKA stg. Now what does guidance mean in the context of video diffusion? You can basically think of it as an attachment or an add on to the video generation model to help create guide it towards producing more accurate and better results. Now this new STG can work independently or in conjunction with the typical classifier free guidance that you see on most image and video generation models. And you can see they've provided a quick little demo. Top is normal CFG and the bottom is with STG applied you can see pretty much everything gets enhanced with the video generation model. In that first clip that we see pretty puffs of smoke come out, but they are much more detailed and more realistic feeling. With the STG model, the physics feel more real. There just seems to be more detail in the smoke. And obviously there's also the staggering factor of this woman who looks more like a creepy haunted doll in the top CFG and looks a lot more like a real living person with the stg. And again all combined with these impressive smoke physics effects that are going on and it slowly covers up her. Next up we have another really great example and I think this shows off what it does best. The top is blurry, it's mushy, we can tell it's a butterfly, we can tell it's a woman's face, but it's not very clear. And the bottom, a lot of this is cleaned up. The butterfly is sharper. We can actually see its individual little legs, its antennae. We can see the woman's face way more clearly. This bulge in the background seems to have been converted more clearly into this woman's lip and her teeth. And rather than just being this random skin blob. And what's really impressive to me is how well this works across motion. It's really easy to do this stuff or we really understand it in the context of image generation models. But to have it temporally scale across all of the video frames is another thing. It looks really great and it seems to be a no compromises solution, so to speak. We'll kind of go through the rest of these here. But you know, better motion obviously with this woman's hair. Much better details in the face, more highlights as well from the sun. Way more details on the trees in this example. Again more details in the castle and the clouds in this one in this one we can see a 3D cat from behind, but on the top it's facing the front and when the cat jumps you can see on the top it flips to behind the cat. But this one stays consistent throughout the whole thing. The cat doesn't morph and switch positions immediately on the bottom like it does on the top. Overall, just impressive and very much useful for video generators, especially ones that are open source and can have this applied. Now this one also does have a full paper and a project page. All the examples I just showed you are from the Mochi generator with this STG enabled, but it can be applied to things like stable video diffusion, also open Sora, where we just see clear improvements across the board no matter the model. For you devs out there who have some experience, I would love to see this thing applied to the new LTX generation model that is open source. And I believe this whole project is also open source because it says the code is based on Mochi. So at the end of the day, what does some technology like this actually mean? What is the impact to the AI video generation space? Well, it should bring better quality across the board, especially for those lower end models that are meant to run on consumer hardware. So. So this should lead to better results at home on your own PC, which is great. And if this one catches on, hopefully the bigger competitors in the space can also attempt to implement it, leading to just better quality across the board. It's a great idea and it seems to work quite well. I don't quite see the downsides to this one, let me tell you. Props to the devs for I assume, releasing this as an open source contribution. Next up folks, we have a big one. We have a brand new fully open source video generated model and it is top tier, I'm saying some of the best AI video we've ever seen. This comes from Tencent, which is a massive Chinese company, so they've got the resources and money to develop something like this, but they released it fully open source. It has weights and everything. Now as you can see from these little sample videos, this thing's smart. It has a fantastic understanding of physics and the real world and lighting and just producing good quality videos over time. It's really a stellar model up there with the greats Minimax, Cling Runway, Sora, if you want to count that. Phenomenal. Again, it's fully open source so it can be implemented into other workflows like this dancing one. Now since this just released, it currently requires quite a lot of vram and for those of you who aren't familiar, that's video memory, memory specific to your graphics card. People are saying it requires about 60 gigabytes of VRAM right now, but the open source community is going to be working to get that down to hopefully consumer levels. This is a big bad model, so I don't know how exactly they could do that, but they're geniuses, so they might be able to do it. Yeah, 60 gigabytes of VRAM is a lot like way more than any consumer grade GPU has these days. Overall though, this is a fantastic gift to the community. I know people can typically be a little bit more wary about the Chinese models, especially considering a lot of people believe that they are not taking safety necessarily as seriously as a lot of the American video generation models. But the fact that it's open source, I think speaks volumes. It shows that at least Tencent in this regard is willing to work with the rest of the AI community and lift up everyone as a whole by releasing their models with open weights and code. I want to get into a few of these demos because they're just insane. In this demo right here, I don't really need to explain it to you, but unparalleled levels of understanding in regards to physics and of course that background is so stable. What's really impressive to me, honestly, and it's not what you're thinking of, is the fact that the hair in the background seems to swish back and forth like that's just such a minute detail that you would never think about. I actually think the physics right in the center here that everyone's going to be focused on are less impressive and more unrealistic. Looking here we've got some really artistic shots. Lots of detail in here. If I move my head here, you can see how consistent the pattern on the chair is in the background. It's not perfect, but it's close. Really difficult for the AI to get right and all while doing, you know, other things in the background. We've got all these people walking, we've got this dude chilling, lots going on. We've got some slow mo video with a child in fire right next to the face and it's lighting up the face in a very realistic way. We've got a nice desert with a bunch of camels walking. The camels look pretty consistent as well with their feet. The walking doesn't seem too unrealistic and like that's a lot of individual little camels to produce in a correct way. There are definitely some morphing and weird little glitches going on. It's not perfect, but dang, that's a lot to have going on at once for the model to produce accurately. And then it also cuts to this demo with this dude trying to eat some sand or maybe smell it and you can see it fall and sift away from his hands in a pretty satisfactory way. Maybe not perfect, but good enough. Got a really nice little slow mo video of a cat mowing down on a cheeseburger. Again, this is pretty complex. We've got this cat which it has to keep in order with all of its legs and fluff and you know, it's a whole entire creature and he's moving down these steps and then finally we have to have a full cheeseburger at the bottom, which the cat then eats. This is difficult stuff for video generators to get right right now. And it did it pretty darn well. All while that background is also consistent and smooth for the most part. It. The fact that this is open source is just awesome because it's a full big bad model. Something really competitive and it's not going to just be limited to, you know, some website where we have to pay for a monthly plan. The world understanding the dynamic of this model is really, really good. It's up there again with the best that we've seen so far. I'll have some more links in the description below if you want to see more examples of this model. Obviously I can't run it, I don't have the GPU power. But hopefully soon we're going to see some AI video generation websites that are running this on that more expensive hardware and we can have a pay as you go system, for example. Stuff like this just increases competition across the board and it's going to make AI video generation better for everybody. Next up, we also have a brand new tool coming from another Chinese video generator. This is minimax, aka hailuai. And this is 12v01 live, which transforms static art into more dynamic motion. It seems to specifically be for 2D illustrations. Enhances smoothness, vivid motion and lets your characters move and speak like they haven't been able to before. Now let me tell you what this absolutely is not. It is not a model where you can upload, you know, a video of you speaking and it transcribes that motion onto the video. It's just a new model specifically made to do the 2D art style and do that motion well. So it's trained more on that stuff and less on the realism video. The demo video they have here shows off quite a lot. We've got this girl reaching for a paintbrush, I assume to begin painting. And this seems like a hybrid anime artistic illustration style. But it looks good. The most impressive thing here for me is the fact that it's really keeping that character quite consistent over time where she reaches over and grabs things. It looks good. This kind of motion sort of reminds me of those animated Spider man movies. And you can see as she moves her hands there, this one seems to be a little bit screwed up and it brings it a little bit closer in a kind of strange 3D esque way. But the way that this hand pulls back is very, very pleasing, I would say. And again, in comparison to other models that are attempting to do motion with illustrated things rather than irl, this is impressive. But you know, she draws this image of a man and then they bring it to life with another generation where his hair flows independently. This right here is a more traditional anime style and it's getting pretty close to replicating that. Well, oftentimes with more traditional anime esque styles, you'll see motion that skips frames, so to speak. And it's kind of doing that here. You can see it generates a few frames of this hand in the same exact position, which is exactly what you would want. And then it slowly moves up and then it's midway through and it's supposed to hold there, which it does, but there is some slight motion going on, some slight warping. It's not perfect. And it then transitions here and does that fairly well. So played back, it looks like this. Again, it kind of still looks a little bit weird. It doesn't look like true anime animation, but it's starting to pick up on those nuances and is something that the AI is learning to do properly. I thought this motion was again, really, really cool. Again, that sort of reminds me more of the Spider man esque style of animation, but maybe a little bit more realistic. I love the way the hair moves and I like that it kept that art style consistent. This is a really cool demo. And you can see she actually, you know, flips her head around and then walks away into the painting, which again is pretty darn cool. You can see it then transitions over to this one. And this is more of a, I would say, flash style animation almost where most of the parts are completely independent static images. But then we have little individual hair pieces moving with some physics. Again, we have some more traditional anime focused stuff. They go through a plethora of art styles here, but a lot of them come out really good. And it seems to be able to tell just by the way that the image is how it should be animated or how it should be expected to be animated, I suppose based on the. The art style. That's just crazy to me. It's good. It really is good. If I had to do animation, this is probably what I would choose here again, you know, some anime art style stuff that it did well. We continue on. I'm just gonna play through the rest of the demo. Instead of nitpicking and showing off every little thing, you kind of get the overall idea. It's impressive to me though that it can pick out and understand what certain styles of animation are supposed to look like. So great work over to the folks at Minimax Hilluai, I think this came out pretty great. This is one that I'm thinking I'm gonna have to spend a little bit more time with and test out myself. I'd love to do like a little animation demo. Animation is a personal favorite of mine. Again though, just pointing it out folks. This, while very, very cool, is not open source like we just saw from Tencent. You could take the bones of the Tencent model, so to speak, and then retrain it specifically to do something like this. So keep that in mind. Next up, Google finally has made their VO AI video generation model available. It's on their Vertex AI platform, which I believe is for developers. Yeah, you can see Google Cloud right here. It's in private preview. Sean Raulston brings us some quick demo shots here. Slow motion of marshmallows roasting over a crackling fire. We've got crowd at an EDM concert. It looks okay, but you know, nothing too glamorous in comparison to a lot of the newer top end models we've seen. Google VEO was announced quite some time ago and it has taken them quite a long time to actually bring it to users and in doing so I think has lost a little bit of ground. As Sean Raulston points out here, Google VO is a little bit underwhelming and again I think that's just due to the fact that when Google VO was first announced, it was sort of top of the line and it's been some time since then and the model hasn't seemed to improve a whole lot over where it previously was. Obviously Google gives you a nice cherry picked demo here with a very nice dog running and leaping around and that's like a perfect video gen example. But we all know the real tests are for more complex prompts. We've got multiple entities interacting with each other in a way that needs to understand physics, for example, and be very temporally consistent. So that's going to be the true test. And again, this is still private preview, but you never know. OpenAI seems to be pretty competitive, especially when it comes to Google releases. So the fact that Google made an announcement that VEO is at least getting, you know, some private preview Access could push OpenAI to want to release Sora sooner. And speaking of that, I am going to leave this off for you folks today. Some OpenAI devs kind of went a little bit crazy on Twitter this morning, being very excited, saying things like OpenAI is unbelievably back again, just trying to like drive up hype it seems. But when it comes from the smaller folks at OpenAI like Bill Peebles or Roon, it feels a little bit more realistic than just trying to drive up hype for marketing like Sam Altman will do. So I'm inclined to believe that some pretty exciting things are on the way. The fact that Bill Peebles is responding to Roon though in this tweet suggests a Sora release because Bill Peebles is the lead of Sora. Although we do have some good comments here from the community saying to be honest, if this is about a Sora release, it's sort of boring at this point because as we know, Sora was just leaked and the public kind of got access to it for a few hours. A lot of people are hoping for more. As Dr. Bobby says here, raw intelligence, power at the tips of her hands instead of a video generation model that's mid ranged. Regardless though, I do want to leave you guys with the idea that we're likely going to see a release from OpenAI this week. A lot of people are predicting it. There's a lot of things on the line. The full release of OpenAI AI01, for example, Sora, as we just mentioned. But then again, this could just be OpenAI people getting very excited that they cracked something like computer use, which was promised in January by Sam Altman a little bit ago, so we'll have to see. But keep your eyes peeled for this week. It seems like it's going to be a large week in AI news. This is all stuff just from today and yesterday and I've got even more that I didn't talk about because like I said, it's just too much. I'll likely be doing some videos midweek with the larger announcements that I seen and then kind of wrap things up on Friday with a full overview, so stay tuned for that. As always, guys, thank you so much for watching my videos, leaving likes, comments, subscribing, and of course, being active inside the Discord community. It means the world to me that this is my job. And man, that AI train just does not slow down. I'll see you in the next video. Goodbye.