Open Source Avalanche! MEGA AI News Dump - Are you ready?

We have so much to talk about today. The AI train does not stop. It's always full steam ahead. But that's a good thing. There is a lot of cool things going on in the space. We've got open source text to speech models, we've got new open source large language models that Compete with DeepSeak, R1 plus, updates from OpenAI, updates from Google, and a ton of other really fun, cool demos. So buckle up folks, because we're off to the moon. So let's kick this off. I want to talk about image generation or native image generation from Google Gemini. It's been the rounds in the AI space and there are a lot of really awesome capabilities and use cases that I've seen people online come up with. So I want to show a few of those off to you guys today. If you don't know much about the native image generation capability surrounding Gemini, the TLDR is this. The large language model is multimodal. It's trained on text and images as well, so it can reason across both. You can send it images and it can recognize them like a photo of your dog. Probably give you the exact dog breed as well. But now with the unlocked image generation capabilities, it can natively look at that image of your dog and then say, oh, here's what he looked like with a top hat in a cane or something like that. But as I mentioned, the use cases get a lot deeper and a lot cooler than putting a hat on your dog. So here is the first one I want to show off. It's by Victor M. We upload a photo online of a bunch of different assets for a pixel art video game. You know, walls and floors and stuff. And then we ask it to a realistic dungeon room for the game using this specific sprite sheet layout. Think about the best setup step by step, then output an image and it goes ahead and does that and it looks pretty good. It used all these same exact assets. We can actually see these specific doors over here in the corner, obviously the walls and textures. It made a little outline and it looks pretty cool little dungeon. Victor also followed up here, asking the AI to highlight the area that the player should be able to walk on in the space in just pure green pixels. And it does that as well, kind of highlighting everything. This really demonstrates the key difference here between native image generation and pinging an API like Dall E. It understands the content of the image with natural language so you can communicate with it and it can highlight things like a person could not just a prompt to an end Image here. Image modifications over time designed for specific use cases and scenarios. Here we've got a more light heart one from an awesome community member, Thakano on my Discord server. By the way, if you want to see stuff like this, all kinds of cool examples from the community, check out my Discord server link down below. He paired Google's native image generation with VDU AI to bring drawings to life. Something that native ImageGen really excels at is the ability to create realistic looking drawings and then use VDO to just bring them to life as animated characters that really do look like it just jumped out of the page just as if you drew it as a kid and you're like, oh, I just wish that I could talk to the Pikachu right now. There he is. So cool. This would be awesome to try if you had kids bringing their drawings to life. I know VDU is obviously doing all the heavy lifting here, but all these images were initially generated by Google native image generation, so I just thought this was really cool. Someone from my community made this and I knew you guys would like to see it. Next up guys, new open source music generation and it seems to offer operate a little bit differently than the traditional music generators we've seen thus far. This news is brought to us by Poonamsoni here on X Notagen. It's Pre trained on 1.6 million pieces of sheet music it seems, and it looks like you can actually input sheet music into it and it will go ahead and play that sheet music out. So compared to the traditional generators that just have songs in with text pairs and then the model is trained like that, it seems to be a little bit more on the genuine note structure, melody structure, musical side of things. I'm no musician, but this looks really cool. Let me know what you guys think. Here's a piano example. Sounds like pretty good piano to me. I have untrained ears. All right, like I said, I'm no musician, but that sounds nice. We've also got chamber violin, one violin, two viola. You can actually see, see that the music is split between the instruments like you'd have in a real orchestra. This seems like it would be able to, at least on the surface here, give you a lot more individual control over specific instruments and how they play their pieces out. This is a whole almost seven minutes of this too, guys. So we're not looking at short generations. How freaking cool is this, Dude? I love that the individual instruments are split up and it just goes through the sheet music. Here's a full orchestra. Sam, That's 15 separate instruments guys. And then an art song. What does that even mean? Oh, Singstein voice pianoforte. Incredible. There are a ton of more examples linked in Poonim's thread here, so I highly suggest you check it out if you want to listen to more. But like I said, this baby right here is fully open source. Nodagen has the full model weights down here with different parameters from 110 million to 516 million, so pretty lightweight overall. Also has a patch length context length here with these small and medium models actually having larger context length, but I assume worse quality. Inferencing locally might require up to 8 gigabytes of GPU memory. I think there's some optimization for sure to be done. And there's a comfy UI node as well. You can just paste the sheet music in. This is so cool though. If you guys are genuinely into AI music, especially like the open source side of things, running things locally, this is it could be the beginning of a new way of doing AI music as a whole. Before we dive deeper into today's AI news, this video does have a sponsor. Sponsors keep this channel alive and kicking. I would not be able to pursue this channel full time without sponsors, so I greatly appreciate you guys putting up with them. Also, this one is pretty related to some of the topics surrounding today's AI news, so check it out and then we'll continue on. Today's video is sponsored by Hume AI. I just did a full length video on Hume AI's Octave text to Speech and let me tell you, it's pretty incredible. This is a large language model designed for text to speech, so it can actually understand the words that you are typing for the text to speech task, meaning it can put emotion and emphasis on certain words to make it sound more realistic. Oh dear traveler, is that some gold I smell in your pocket? Us goblins love gold. Would be a shame if it went missing. Not only that, you can actually give the large language model acting instructions for specific pieces of text. So it's directable, kind of like an actor. They also allow you to create new voices simply with text prompts, which is pretty awesome. Anything from an old raspy wizard to a creepy goblin. So not only is Hume New LLM powered at text to speech state of the art, but it's also the cheapest on the market, which is pretty crazy. Their plans start at a mere three bucks a month and they offer you more character generation than 11 labs for the same price. Also, you can try it out for completely free which is pretty cool. But yeah, if you want to learn more, either check out the Hume website with the link in the description or check out my full dedicated video where I spend a lot of time going through and testing out this new LLM based text to speech. Huge thanks to Hume for sponsoring today's video. Now back to your regularly scheduled content. Let's check out our next piece. Next up we're talking about Zyphra. This is another text to speech model, but this one is brand new and open source as well as being very much state of the art. Just like today's sponsor Huma AI. This is a text to speech model, but this one, unlike Hume, is open source and uses traditional text to speech generation methods. But it's new and I gotta say it sounds really good. Take a listen to their Zonos model here. This is a quick demo. Yeah, they do have a pretty nice website where you can try it out, but like I said, it's fully open source, weights and all. Hey folks, I just came here to let you know that I am hopelessly addicted to growing lemons. I've sold my house, car and dumped all of my savings into building the world's largest lemon farm. I'm broke. Please help. Yeah, you can see it's definitely not as natural as you might hear with Hume for example, or maybe even 11 labs. But it's still very much crystal clear audio and it's good enough especially for being open source. Plus they have voice cloning available right out of the gate and a couple different languages to pick from. And Hume is right now unfortunately only in English. So it's a compelling alternative to a lot of these paid non open source text to speech variants. For those interested, here is the GitHub Zonos OS. They've got a Gradio interface for it which is pretty nice. Should make it pretty simple to mess around with and use locally. Not to mention some pretty modest VRAM requirements. Only 6 gigabytes of VRAM minimum, but no Windows support. Linux or Mac OS only right now I'd love for someone to convert this though considering it is open source. Yeah, this sucker is under the Apache 2.0 license. High quality competitive open source text to speech. Now I will say though some of you are going to bring up this point sure in the comments. Kokoro 82 million is a smaller open source text to speech model that definitely could be a better option for some people compared to Zyphra. This one's already clearly well known considering it has 1.6 million downloads. It's also Apache 2.0. This model is definitely more lightweight in comparison to Zyphra. Kokoro is an open weight TTS model with 82 million parameters. Despite its lightweight architecture, it delivers comparable quality to larger models while being significantly faster and more cost efficient. With Apache licensed weights, Kokoro can be deployed anywhere from production environments to personal projects. Yeah, I think this one is definitely going to be a little bit better for the devs that need something cheap, fast and good, but Zyphra definitely has more customizability. Either way, both Kokoro and Zyphra are open source and with Humai's brand new way of doing text to speech using large language models instead, it looks like the text to speech space as a whole is really heating up with some competitive players coming to bat. Should 11 labs be scared as the previous kings moving swiftly along? Let's talk about our good friends Google and OpenAI next. We're going to start off with Google. This comes from Tbor Blaho, a phenomenal account to follow by the way. He's always tweeting the latest, especially regarding these big companies and some open source stuff too. But but Google has started to roll out their very own Canvas mode for Gemini, which supports code including react, writing, link sharing and is very similar to the Chat GPT Canvas and Claude Artifacts. So they're a little bit late to the to the game with this, especially considering ChatGPT Canvas is pretty darn good and Claude Artifacts is great for building quick little coding demos. So a little bit late to the party here Google, but glad to see that they're trying to stay competitive. And as we talked about in the opening to this video, they are in some cases, especially with things like native image generation. Oh, here's an important note though. It does support Python, but there's no actual Python code execution. However, there is a quick and easy export to Collab, so if you guys are coding with Gemini, should be the easiest way to do it. And continuing our awesome insights from Mr. T Bor here, we've got some stuff from OpenAI's CPO Kevin Wheel a timeline for GPT5. He answers, I won't give you a time. Oh, okay, but it's soon enough. All right. It really seems like people at OpenAI are really trying not to give specific information surrounding GPT5 or its tentative release date. Like all we really hear is that it's soon, which I guess hopefully means this year. Hopefully means, you know, in the next few months. He says that they're not talking about it, but they're very serious about it and they are working on it as he speaks. And then O3 is also just coming soon. So basically we got no dates, but we have information that they're working on things. OpenAI is being pretty darn secretive and that's something to note here. It seems like ever since the early days, the pre chat GPT days, OpenAI has ironically just gotten more and more closed off and it's got to be due to the competition, right? I mean, think about how much has come out of the woodwork. Deepseek R1 comes out of nowhere slapping a decent reasoning model that's fully open source on the table, cheaper than OpenAI's variants. Sora was premiered earlier last year, but was quickly wiped out by a slew of again, Chinese open source video generators. I think whatever OpenAI is cooking in the background, they're just too scared of the competition and too worried about releasing additional information to give competitors an edge. So they're, they're just shutting their doors really and saying like, hey, yeah, the, the next stuff is coming soon. We're working on it. One other thing to point out, Advanced Voice mode in Chat GPT is going to get some improvements soon, which is good. As you guys might have already known, applications like Sesame AI have shown off that really, really good AI voice is on the way and definitely better than what we currently have from Advanced voice mode in ChatGPT. Next up we've got a new state of the art super resolution model that uses neural heat fields to achieve aliasing. Free arbitrary scale image upscaling. This comes from counting sort. I mean you can no doubt see that's a blurry image of a giraffe and it becomes not so blurry quite easily. By the way, the name of the model its is Thera and you can try it for free on Hugging face, which is pretty great. The link will be down below. You know what, let's try it on my face. I know what I look like, so we'll see if this thing is accurate. Alrighty. I have severely reduced the resolution of this random image. You know what, I'm gonna draw a little smiley face on the side here and see what it does with this too. Okay. It is processing as soon as I put it into the website. And there we go. You know what, I'm pretty impressed overall, especially with the lemon. Oh my God, look at that gu. Look at how few pixels we're working with here on the side. Boom. Wow, that's actually insane over there. Definitely did a pretty decent job with me as well. Here is the original. I mean, just looking at the lemon difference here, like that's what we had to work with and it, it absolutely brought that one back to life accurately. Dude. Same thing with my shy guy in the corner wearing the headphones, man. That's crazy. I mean, it missed a couple of details, but holy cow, this is shockingly, shockingly accurate for how low resolution it is. Jeez, man. I like the face too. It definitely is artificial. And yeah, it was artificially placed. That. That's mind blown here. This thing's crazy. And boom. Also open source. You can Download Checkpoints Apache 2.0 as well. So it's, it's open source too. And it's freaking awesome. Upscaling. Look at all these fantastic releases from the open source world of AI today, man. Love it. And it does not stop there, guys. Next up, ak Again, fantastic account, by the way, if you want to keep up to date with especially like smaller open source projects. We've got Recam Master, a camera controlled generative rendering from a single AI video. Essentially we've got a typical video that you input and it's able to change the camera perspective of that video by synthesizing it with AI. How incredible is that? Just these random demos you can see here are pretty staggering. So let's take a closer look. The classic Great Gatsby clip right at the top here. But instead of him looking right at the camera, we've got the camera rotating as he lifts up and tips his wine glass. You can see it has to hallucinate a lot of the background here, but it does a pretty great job. I mean, it really does look like a different camera angle. I'd love to know how accurate it truly is. I don't know. How does Leo's face look though, as it tilts? Do you think it gives him maybe a little bit more weight or is this accurate? It looks pretty good. Here's Yann Lecun making a nice appearance as well as we just zoom in on his face. Get a little bit of extra detail in his glasses and stuff. I also like how the background gets more blurred as we zoom into Yan's face. Quite incredible. You can notice though, I think that it does add some more detail, some more wrinkles and stuff into his face as it zooms in. So it might be something to note with this model. How accurate are those additional extra details? Anyways, we've got another one. I believe this is Planet of the Apes. I haven't seen any of those movies unfortunately, but yeah, as he looks up, we've got a different camera angle again, twisting around. Same one for this video on the bottom. Bottom. Taking a shot that doesn't really turn all that much in the original, but making it really turn, revealing the rest of the city, which it does a pretty great job at hallucinating, I will say though. I mean, this structure, whatever machinery is in the background here does look a little bit AI generated esque. But as a whole, this video is quite blurry. This I thought was so cool because in Friends, if you've ever watched it, you never see any alternate camera angles. It's a sitcom, it's a built set. So typically you wouldn't really be able to see those other angles without revealing the rest of the set. But with AI, we can absolutely hallucinate a whole new kitchen in the back there. And similarly with this bottom video, we have a completely different camera angle generating entirely different people. You can see this woman's face is pretty mushed up. Depends on your circumstance here. I mean, we've got Spider man, we've got Breaking Bad, the classic pizza scene, making it look more like a drone shot, giving the pizza the attention it deserved. And even the Titanic scene here. And I think the reconstruction arguably a little bit better. Except for her hand here, which becomes a little bit of a stump if I move my face there. In general, guys, this is so cool. I mean, we're synthesizing brand new camera angles that never previously existed in the original shot. This is the kind of stuff that you can do with AI. We've also got video stabilization as another application of this, and it does that really well, recreating everything. Can imagine this being built into the phones of the future. Stabilization is a pretty big part of phone camera technology. We even see potential applications for robotics and self driving, giving us new imagined angles from a higher up point of view of the traffic that's ahead and what to expect. I mean, when you're driving around, let me know if you guys do this too. But you might look at something like this might be your actual view, but in your head you can kind of picture what things probably look like. To give you a better idea of how you should react in the future. AI's got a lot, and I mean a lot of potential use cases, guys. Even if you think it's just for video work, you know, reshooting a scene that needs to be reshot for a movie or something like that. Nope. Also robotics and self driving and all this other stuff, you didn't even think about this one. Does indeed have a project page showing off a ton of different examples and a pretty deep dive into how this thing actually works. But unfortunately, as cool as this project is right now, it is not open source. It's very much closed as just research. And they're pretty much just taking in random videos by uploaded users essentially. Send them a video to change the trajectory of to their, their emails essentially and they might get back to you. That's. That's where we're at with this one. Hey, they do have a Google form for it at least, which makes it a little bit easier, so give it a shot. I'll have all this linked below. As always, touching down towards the end of today's video we have a brand new AI model from Baidu Ernie 4.5. This is a deep thinking reasoning model that is also multimodal. They claim it delivers performance on par with Deepseek R1 at half the price. And as Robert Scoble points out, we have an AI price war on our hands. All of these companies competing for the cheapest, best AI large language model physically possible. You can also access this model via their website as well. If you're wondering about the actual Pricing regarding this, Ernie 4.5 has input and output prices as low as 55 cents per a million tokens. I assume that's input tokens and then 2.2 dollars per a million output tokens. So ridiculously cheap. Now it's not open source just yet, but Baidu does say that they are planning to open source the model in the next few months. If you're still looking or you still need something that is absolutely fully open source, DeepSeek R1 might be your best bet. But this is a very cheap alternative that will eventually be open sourced according to Baidu. Let's see if that that ends up reigning. True. And finally today we got a little bit of an announcement from Huan. Coming soon, Huanyan T1, the first ultra large Mamba powered reasoning model. So I assume this is going to be a text model, hopefully also multimodal that is built for reasoning. We'll see if it tops benchmark charts. We'll see if it competes with Deep Seek R1 or OpenAI03. I'm very excited. Just thought I'd make you guys aware. They posted this a couple hours ago. Just coming soon, a new ultra large reasoning model from Huan Yan. This was a big video and we had a lot to talk about, but I still want to mention this. I'm going to be giving an RTX 4080 super away for completely free. It's free to enter and free to win. Literally no charge to you. If you win, I'll ship it. I'm paying for the shipping. I've partnered with Nvidia. They sent me an RTX 4080 Super. This is just a regular 4080. It's for the visual, the real ones in the box back there. But yeah, Nvidia's GTC is happening right now and all you need to do to be entered to win this giveaway is to sign up for a free Nvidia GTC online session with the link that's provided in the description. After you attend the session, all you have to do is fill out the Google form that I have also provided in the description below, Tell me about the session and which session you attended, provide some information and you're good to go. To be entered to win. Someone is going to win an RTX 4080 Super. 16 gigabytes of VRAM. Guys, we're talking about running local AI models on your machine or upgrading your gaming rig. Whatever you want to do that needs a honking gpu. So yeah, all of the information is in the description for this giveaway. Seriously guys, this is a great opportunity to just win a free gpu. Everyone always thinks they're oh, I could never win any of these giveaways. But you could not. Too many people have signed up yet. I'm checking. That thing could be yours at any rate though, guys, thanks so much for watching today's video. I am matvid Pro AI. Hopefully you found this useful and informative. I'll see you in the next one. Goodbye.