AI News Roundup: Pyramid Flow, Video Input LLM, Gemini 2.0 & more!

Welcome back everybody to the channel. Today I've got a nice AI news roundup for you all. Not as big as the previous weeks, mind you, but still, there's a lot going on. First up, the AI video generation tirade continues. Open source fine tuning of video generation models has just become a little bit more accessible. Apparently a 5 billion parameter video model should be possible to tune with a single 24 gigabyte GPU. And again, most of us don't have that much vray ram and our GPUs, but for developers this is very, very much accessible. They're releasing COG Video X Factory, which is a repository containing memory optimized scripts to fine tune the COG family of video models, which is the older open source video generation models. I advocate for open source AI technology, especially in the video space, because there isn't a lot of competition in terms of open source things. So I think this is pretty great and I would love to see what fine tuning can do for AI video generation mod as well. Can we fine tune for animation specifically, let's say, or could we fine tune for just upscaling? Right? Could we turn it maybe with a little bit more complexity into an upscaling model? Next up, Runway ML's Gen 3 Alpha Turbo just got a little bit of an update. As you guys know, image uploading into video generation models is probably one of the most powerful things you can do with them. And Runway just took it to the next level. Before it was either the first or the last frame was the image that you upload, but now you can have it be both. So I can choose an image to be the first frame of the video and I can choose an image to be the last frame of the video and it will generate in between that. So if they're only subtly different, let's say you can have a lot more controllability theoretically over that generated video. Or even if they're different, you can have very interesting transitions between the two. There's a lot of creative things that you can do with this tool and I wish more AI video generators would implement it like this. Either way, it's good to see that Runway is still consistently shipping new features because I really like their interface. And honestly, in terms of reliability and speed, Gen 3 is towards the top of my list for video generators. This is one that I might want to play with in a larger video because it's pretty darn cool. Next up, guys. This is probably going to be the coolest thing that you see in relation to AI video today. This is a brand new fully open source text slash image to video model. It's MIT licensed and it rivals Gen 3, Pika and Kling. It's called Pyramid Flow. You might have heard of it by now but it's training efficient auto regressive video generation. It uses flow matching. It trains on open source data sets which is pretty cool. High quality 10 second video generation which is on par again with the likes of all of the top generators right now. Just a tad over 720p for the video resolution. So again this is pretty much in line with the rest of them. Cling Runway ML I know cling does have a new 1080pmodel and that's pretty awesome. But for open source this is definitely still awesome. You got a 24 frames per second frame rate which is the standard for film and of course the big one, image to video generation probably the most powerful tool again in relation to text to video generators model. Checkpoints available on the hub. Does this guy know you can't just say the hub these days. Anyways, yeah, they're on. They're on. Hugging face. I'm really excited to start seeing some easy installs, like quick installs for this thing. Possibly we'll see something in Pinocchio, let's say somewhat soon. Although I'm not really sure what the requirements are to run this thing. Obviously though, I mean this uplifts the entire community, right? Because now everyone can go and see exactly how they did this and they can also implement it themselves and make tweaks to it and adjust it because that's what open source does. It's going to be improved quicker than probably the closed source models and of course it's going to be free. You're going to start to see eventually at some point websites offering free video generation. It will probably be based on Pyramid Flow. But yeah, the generation quality if you kind of get up close and look at it, is yeah, I mean I would say pretty close to being on par with that of the top generators. Maybe not as good as some of them, you know Minimax Kling comes to mind the new ByteDance model but it's pretty darn good. It's leagues above anything we've ever seen from open source video before. And that's a big deal. It seems to do these nature esque landscape style things pretty well. You can see the waves crashing into the rocks very realistically and the camera sort of pans over to look at the ocean. The colors are good, the motion is very smooth and realistic. They've got Some better examples over on the project page right here, for example, we've got Tokyo in the snow and this looks pretty decent. Although the people don't really have faces and they're quite blurry as they get close to the camera. But the walking looks decent and you can't really make out any of these signs. So I think it's safe to say that this is probably below Sora quality because there's a very similar Sora generation that's quite a bit better. I thought this one was pretty decent as well. This is cars driving at dusk. Some of the cars don't have front wheels though. But the overall background is consistent. I think the road is decently consistent. Fireworks look alright. I think the burning food actually looks pretty appetizing. This is not bad at all. You know we've got a man over here in the desert. Similar to that original Sora demo. It doesn't stack up perfectly. It's a little bit weird in the background. But then again, open source available now. It's real. Some more food. We've got a cat. I thought the cat cat was a little bit creepy. I've seen this little demo video before. It looks like he's got like a weird tongue kind of hanging out and his eyes are just kind of like lacking pupils. He looks like he has cataracts, no pun intended. But yeah, it's clearly easy to see what the strong suits of this model is right now. It's all gonna be in the things that are in the training data quite often. So, you know, burning fire, cooking food, like stock footage style stuff. Again though, I would love to see some fine tuning maybe of this model. We could fine tune it to do different things. We could fine tune it specifically for animation, specifically for cars, specifically for drone shots, etc, etc. That's the exciting part is the modifiability and the future prospects of an open source model like this. Next up, guys, we actually have a little bit of OpenAI chat GPT news and you can see it right here. The interface for ChatGPT has been adjusted a little bit. It's more of a Google style with the chat bar right in the middle. And again, this is different from Canvas, although they've also updated it pretty much across the entirety of Chat GPT. On first glance, it's really not all that different. Like if I just type some stuff in and then send it, it kind of goes back to the regular ChatGPT. But the difference is when I press slash. If you press slash, like a Minecraft command, for example, we can see we can actually command it to do specific things like generate a picture with Dall E search the we which is back. Hey, this is finally back, guys. And honestly, I don't know if this is the new search model they've been working on or if this is the old one, but in my impressions of it, it is still not better than Perplexity AI. I still find that Perplexity AI is faster and it is more detailed and gives you better results overall. So yeah, searching, like web searching is back, but I still would rather just use Perplexity. And now you'll see once we've done web searching interest, interestingly enough, now we've actually lost some commands. We can't even pull it up now. Hello, commands. Okay, for some reason it's having problems. Let's do a new one. Anyways, another thing that I was really excited about was the O1 preview reasoning being able to just get commanded inside of Chat GPT. I was like, oh, can we upload an image and then have it, you know, reason with this new model about the images? The answer unfortunately is no. If we command the new reasoning model, it does pull it up and the image gets sent to Chat GPT, but it's unable to actually see the image, so that's still blocked off entirely. So no new real abilities here except for that small command function, then searching. But still it's nice to see small consistent updates rather than big, large ones every couple of months. Oh, and I've just realized I've forgotten to talk about another update in the AI video generation space. So we're going back into that real quick. There's a new AI video generator called Dreamina AI version 2.0. It's a powerful all in one AI generator developed by ByteDance. This is the parent company of TikTok, and this model was also codenamed Seaweed. In fact, I've already talked about it in a previous AI news video, but now apparently it has an American name called the Dreamina AI. It's in beta testing for the new version 2.0, but they do have a 1.0 link below and this is like a completely Chinese website and it wouldn't let me translate it, so I have no idea what's going on here. I don't think that this 1.0 model is anything in comparison to what you already have access to, but the 2.0 model most certainly is. It's very, very impressive. As you can see from these little demo reels, it is crazy good. It's probably the Best video generation model we've ever seen. Like I said, I've already talked about it in a previous AI video. It's. It's wild. You got to see some of the examples from this thing because it is breathtaking. But apparently it's got a beta now. Don't know how to get access to it. That might be a follow up video that I do. Or we'll see if maybe some of the people in my Discord server can get access to this. But apparently the updated 2.0 version offers a wide range of features including image and video generation, of course, AI music creation, storyboarding, similar to LTX Studio motion brush similar to Cling Canvas Tools. Not even sure what that means. And more essentially covering all the major creative functions you'd expect from AI. I guess. So it's, it's interesting. Seems like they're taking a bunch of different models and combining them into one platform, but. But the video generation seems to be what they've worked the most on. Apparently if you find an Android device, you can download it in the unofficial store. Oh yeah, okay. It requires this douyin available only to Chinese users right now. Anyways. Yeah, China is kicking some serious butt in the AI video generation space and I'm all for it. I mean, I'm happy to see competition a little bit concerned maybe about some of the practices, the ethical practices surrounding how these models are trained in comparison with the United States. Although I'm not really a lawyer and I haven't looked too deep into it. Do you guys know anything about that? Tell me what you think. But yeah, they're clearly not holding back because these models are crazy impressive. In other news, there's a new Vision Language model out that is apparently pretty impressive. I might do a full video in fact, on this one because it has something that I've never even tried with AI before. It's a model by rhymes, AI and it's only 25.3 billion parameters, but it's multimodal and it can intake not only images into the, the text model, but also video inputs and that's a big deal. Again, you can't upload videos to ChatGPT, you can't upload videos to Gemini or pretty much any of the other large language models that are multimodal. That's. This is like brand new stuff. So I am really interested to see how this thing works. And like I said, I think I want to do a video on it. It's also kind of open Source here, Apache 2.0, I believe that's limited open source and they also have fine tuning scripts release. Okay, so they're promoting fine tuning as well, which is awesome either way. Merv here tested it extensively. Merv was able to get it to debug a screenshot of code, which is pretty crazy impressive for such a small model, that's for sure. Perhaps the multimodal training is allowing it to become better at doing tasks like this. It was also able to recreate some handwriting and scribe it out as text instead, so that's pretty impressive. ChatGPT can do stuff like that too though. Decent real world knowledge. So this is obviously a photo of a specific landmark and it's able to pick out that specific landmark then give you some information about it. That's pretty darn cool. I'm really interested in the video uploading capabilities though, so I think I might do a full video just testing those out. So if that's something you're interested in seeing, please let me know. Trying to kind of get a gauge a reading on the community for what is interesting to them. I will be reading comments. Promise. Anyways, we've also got confirmation that Google Gemini 2.0 is in the works and we've actually got some specifics on it. This comes from Prashant here on Twitter. This is from the Google DeepMind event in San Francisco. This is basically what we have to work off of Gemini 2.0 obviously next generation of their frontier models, so larger, bigger, better models. Apparently it's also going to be multi turn, which I don't really quite know exactly what that means. It might mean that it does some work autonomously. Maybe they're saying that it will recursively self prompt to solve problems similar to the new OpenAI 01 model. And we've obviously also got vision somewhere in there, so vision understanding, we expect that audio. So I don't know if this is audio intake only or audio generation as well like the OpenAI GPT4 Omni model, of course, advanced voice mode in the Chat GPT app and also embeddings and much more. I don't know, I thought this was pretty cool to see that Gemini definitely is getting an upgrade. It's TBD though we don't really have a release date for it. At least it's confirmed officially. OpenAI would never do anything like this, like confirm it offhandedly in the, you know, a little Google DeepMind event. It's good to see the competition is still competitioning. Other than that this is just absolutely absurd. So Elon Musk has a whole army of its Tesla robots walk out on stage at its Tesla Live event. I think he called this event like we robot, something like that. Anyways, yeah, they've got these crazy robots, they're walking around. They definitely look pretty decent. Though I have to say in terms of, you know, real world robotics, it's impressive. One important thing though that I think is worth mentioning is that while it's still super impressive, as Christian points out here, any bets on where Tesla is keeping the dudes tele operating these robots? Because they definitely aren't all just self operating. There are a lot of AI based robotics, humanoid robotics that are being developed by a number of companies across the world and I haven't seen any of them doing anything close to something like this all on their own just yet. And I doubt that they got it working for this event or they would have been bragging and showing it off. It's definitely still being tele operated by an employee somewhere. Regardless though, teleoperation is pretty difficult. So it was able to do things like obviously play rock paper scissors here, but they also had them pouring and serving drinks which was impressive regardless. The dexterity and the ability for the robot to perform human tasks is the impressive part. I think here the AI autonomy and the training is something else entirely and I don't think think we've seen that just yet. It's interesting too in the comments down here we have people saying that it's a vision model doing function calling, which is like no way. This is a vision model doing function calling. And it's also like has real people just speaking to you from the robots. Like people were having conversation with the robots, you know, and they're, they're taking drink orders and stuff. It's clearly someone working on the back end just teleoperating and speaking for it. It's just insane to me the amount of people that think it's not teleoperated. Like it's still impressive. I still this incredibly impressive even if someone is still remote controlling it. Humanoid robots, I don't know if you guys knew this are very difficult to get right. It's standing like a person. It's crazy. Anyways, finally, to wrap this AI news video up for the week, meta AI just got their very own voice mode. So it looks like they have a bunch of famous people's voices cloned inside of the meta AI and it seems to be similar to the old voice mode that OpenAI had. So. So it's not directly multimodal, which means it's taking your voice in transcribing it to text, putting it through the model. The model gives you a response and then they take the text from that response and turn it into some audio based on a voice generation model that they've made. And that's all fine, but I think advanced voice mode is way cooler because it's all natively multimodal. It natively takes in your voice, understands it, and then natively outputs speech all in one model. And it's way more natural. I mean, it can understand your emotions where this definitely can't do that. Google also has a similar voice mode. I've been asked on multiple occasions like, oh, are you going to check out the Google voice mode? Are you going to check out them the Meta AI voice mode? Now the answer is maybe I test it out on stream or something like that. But it's really not that big of a deal considering for a very long time we had essentially the exact same thing in chat GPT. Maybe this would be a little bit cooler because you can have, you know, the voices of famous people. Anyways guys, that's it for the news for this week. I know I didn't really do much uploading this past week week and my sincere apologies to you guys, but the plan is for next week to be a good week full of many videos. I'm trying to up the rate of video output for sure. So stay tuned on the channel and as always, thank you so much for watching the video. See you guys in the next one. Goodbye.