Tech Overlords have Blessed us with New AI Updates!

Hello everybody, and welcome back to the Matvid Pro AI YouTube channel. I've got a nice Friday AI news roundup for you all. And I'll admit with the US holiday of Thanksgiving, AI News has been a little bit slower as of late. But I've compiled everything from the last about week and a half that I've seen, and there's definitely a few things you're not going to want to miss in today's episode. So let's dive in. To kick things off, I've got some AI news coming from Nvidia. They have developed what they call the most flexible sound machine in the world. This is a generative AI model that generates sounds. And in the AI world, when you hear that, you think instantly about sound effects. But as the title says, this is the world's most flexible sound machine. So it can do music as well. It can do voices and sound effects. It's a full audio model, not limited to one subcategory in the audio world. And, and for the image here, they give us a nice cat wearing some headphones. But I have to say, dude is not even listening to the audio like Nvidia. His ears are up here. Bro's just feeling the vibrations. At any rate, this model is called Fugato, short for Foundational Generative audio Transformer Opus 1. And talk about like a word salad of all of the hot terms in AI, right? There are some very specific adjustments in this model in comparison to others that make it stand out. For example, during inference means meaning when it's generating. The model uses a technique called composable art, and it combines instructions that were only seen separately during training. So it should be able to do more complicated prompts, like asking for text spoken with a sad feeling, specifically in a French accent. This obviously, as Nvidia explains, gives you more fine grained control over the text instructions. Apparently, the model can also generate sounds that slowly transition and change over time. They call this temporal interpolation. The example they give here is that it can create sounds of a rainstorm moving through an area with crescendos of thunder that slowly fade into the distance. And apparently it also gives users fine grained control over how the soundscape evolves over time. The model is two and a half billion parameters. But enough talking about this model. Let's listen to it.

Fugato is the latest generative AI breakthrough from Nvidia. This new model allows you to create sounds, speech and music from text and audio inputs. You can guide Fugato to create unexpected sound effects where familiar Sounds take on surprising new qualities and invoke new experiences.

Man, talk about a creepy one to start off with. Anyways, that was deep rumbling bass pulses paired with intermittent high pitched digital chirps. Like the sound of a massive sentient machine waking up. Very highly specific prompt and very highly specific output that matches that prompt. Anyways, there's no doubt that this is obviously a somewhat cherry picked example, but let's see what else they have. This is a train passing by and slowly it becomes a lush string orchestra. Okay, I like that one a lot. It does a really good job at blending it slowly and seamlessly. I don't think I've ever heard an audio generation model that can do something quite like that. That is impressive. It also, as we saw earlier, takes an audio input as well as text. So in this example, piece of music.

Is just as easy.

They can isolate. Specifically just the voice in comparison to the audio. Seems like they've got a real winner on their hands here. And they've taken some strides in act producing an audio generation model that does things that previous ones haven't. So. Nvidia, I'm impressed so far. This is another example of something I just haven't seen elsewhere in the AI space. This is brand new stuff. Here we've got some speaking examples. Calm voice with an American accent says kids are talking by the door.

Kids are talking by the door.

Pretty good. Very realistic sounding. Maybe not, you know, book narration quality. It's no 11 labs, but gets the job done.

And if you want a different delivery, fugato can do that too.

Kids are talking by the door. Wow. So it can actually take its own output and use it as an input and then adjust the sound on the fly. That's really cool. I might have to do a deeper dive video into this.

Or complete.

Man, the editing on the fly is almost too good to believe. Turn this MIDI melody into a female voice operatic scat singing style. Wow. And where's the original melody?

Melody they wrote?

It's pretty good. Man, that's crazy. It understands audio quite well. And at only two and a half billion parameters, man, it's impressive. This is a good model, Nvidia. The music's not bad either.

Fogado is a groundbreaking foundation model that gives you sonic superpowers, opening up new possibilities for creativity and production.

Pretty cool. Well, bravo Nvidia. I mean, this was a very good demo video. We'll have to try it out for ourselves though, to see if it's really worth all that talk. As you can see from the comments though, on this blog. People want to know when how this will be available and I'm firmly in this camp as well. The best announcement ever. No links, no way to use the product. Let us know when we can try it. Fair enough. Nvidia is pretty good though. I trust them to actually end up releasing this to people. Hopefully they make it open source. Don't know if they will. Looks like there's already a research paper on it though, which is a decent sign and obviously I think will get API access at the very least. Regardless, looks like a really impressive audio generation model, but let me tell you, people in the AI space are sick of teases and demos. They want to see and use the product for themselves. So Nvidia, please get on that. Sincerely, the entire AI community All right, before we dive into the rest of today's news, I have a a quick word from today's sponsor. I really appreciate you guys sitting through and putting up with sponsored content. It is the reason I can pursue this channel full time. InVideo AI v3 is here and it has really turned into your automated video creation agent. Unlike a lot of the traditional AI video creation tools in video, AI v3 does all of the heavy lifting. It does script writing, it prompts AI video generators, it adds editing, sound and subtitles. And not only can it create shorts, but it can also make full, full length videos. I'll be doing a full review of InVideo AI v3 here on the channel, so stay tuned and get subscribed for that. For now, take a quick peek at what InVideo v3 is capable of.

I bet you've tried to cut back on sugar before, only to find yourself reaching for the candy jar again. Well, you're not alone. Sugar, particularly fructose, has been increasingly recognized for its addictive properties. But what's really going on in our brains when we consume sugar? According to a 2013 study, sugar affects our brain's dopamine system in a way similar to addictive substances. This creates a cycle of cravings and consumption that can be hard to break. And it's not just about willpower. It's about biology. Excessive sugar intake is linked to a host of health issues. We're talking obesity, diabetes, cardiovascular disease, liver damage, cognitive impairment, and even mental health issues. A 2015 study in the Journal of the American Heart association found that consuming too many sugar sweetened beverages was associated with an estimated 184,000 adult deaths per year worldwide. That's 184,000 deaths per year from sugary drinks alone. What about natural sugars, you might ask. While they're certainly a valuable energy source, the key word here is moderation. Moderation is crucial when it comes to any type of sugar.

Now you can try InVideo AI for free. However, if you want to Access the generative AI features with InVideo v3, your best bet is to go with the generative plan. That is $96 a month and while it is very expensive, this is going to be your best bang for buck plan with the new AI generated video minutes. However, if you already use Invideo, you can purchase an add on for those generative video credits. There's a link in the description below to sign up. Huge thanks to Invideo for sponsoring today's video. Now back to your regularly scheduled content. Welcome back folks. Next up guys, I have news coming from my very own Twitter account. If you want to hear these things earlier than in a video like this, I highly recommend you either follow my Twitter account or join my Discord server. Anyways, yes, I actually did do a little roundup on Twitter about this one, but LTX Studio released their own video generation model. It can generate 5 second length video in only 4 seconds and it runs on consumer hardware, fully open source. Granted you do need an RTX 4090 which is quite a beast, but it's a pretty darn good video generator. And like I said, open source meaning people will likely try to optimize it for lower end hardware. And still it's coming out of the gate swinging here open to consumers and not just people with beefy server grade GPUs. They've got a little demo video here showcasing some of the clips. It looks like it's about the same quality level as other top of the line video generators, maybe a little bit worse. It's probably not the best one, especially in comparison to Sora which we just got a sneak peek at. But this is open source, it has the capability and actually definitely is very cheap right now and possibly could even be free in the future with the right websites and access. It's pretty temporally consistent in terms of backgrounds and like this walking here isn't too too bad. I think with this model it's the finer details where you're going to notice a slowdown in comparison to other models like Runway, ML Cling or even like I said, Sora seems to have a pretty big focus on humans and generating their movement and likeness. Very realistically, I think this is a pretty nice little nature demo. For example, we've got all of the different Plants swaying in the wind. Pretty realistically. This jellyfish looks quite good. Again, these are pretty easy, simple prompts. When you get to the more complex stuff like this space woman, it seems to have a hard time keeping her face normal. And by the end of the 5 second mark, this person kind of changes and morphs into a different person. But hey, the elephants look pretty good. If it's in the training data, odds are it's gonna look pretty good. And I think that's true for most video generators. Apparently users have reported running as low as 12 gigabytes of VRAM so far, which is pretty darn good. That means we're in 4080 territory already. And like I said, open source. So the community has already got their hands on optimizing it and making it work for the rest of us. Faux for AI, a fantastic account by the way, has spun Lighttrix LTX Studio as its own model on replicate.com which is a great website for testing these models out. It'll do text to video, image to video, and you can change the aspect ratio, length, resolution and CFG scale. So pretty nice controls here. And the cost is the best part, at only a cent per three seconds of video, which is cheap in comparison to a lot of other video generators with big expensive plans. This is fractions of the price. And I think this undoubtedly already probably will beat something like Sora from OpenAI in terms of pricing, not to mention the rest of them, Runway, cling, minimax, etc. Actually, I have to change the way I say that, et cetera. One of you viewers mentioned that I keep pronouncing it wrong, so I'm gonna try to say it correctly from now on. This really has been the year for AI video generation. I can't believe how far we've come. For example, just this demo clip of someone cooking. I mean, if you showed that off earlier this year, people would be like, wow, that's like one of the better AI video generation models I've seen to date. Really impressive. And now at the end of the year, this is like the sort of lower quality but open source and highly cheap model. So exciting. Continuing our AI video generation trend, Runway ML has some interesting updates as well. First up, we have Expand Video. This allows you to transform videos into any aspect ratio you want by generating entirely new areas in new portions of the video based on whatever you're inputting into it. So they're calling it, you know, Expand Video. It's gradually rolling out and soon everyone has access. And this was a while ago by now we might all have access. I probably don't need to explain much to you guys. You get the idea. We start off in this example with a zoomed up close photo, obviously a video of a woman at a waterfall. We can then expand that into 16 by 9 by keeping that center portion and just filling in the new parts to complete that aspect ratio. And it does a really good job at generating the rest of her body, the rest of the waterfall, the rest of the scene as a whole. It understands the concepts pretty well. And then they also show another expansion, making it an even bigger scene back to 16 by 9. And this also, in theory, gives you a higher resolution, at least in the center here where this woman's face is. And it's not perfect with this expansion, but it's good enough. This example was a little bit more impressive. We start with a full screen of a guy holding a camera pointing upwards, and then we generate this miraculous, beautiful cave. And it's really good at managing the correct depth, the correct camera angle and movement. As the camera slowly pans throughout the cave, it looks like originally it was this shot the whole time and not just this zoomed in video of a dude with a camera. So very much impressive here from Runway. They've got a few more fun examples here. You know, this guy's all stressed out, we don't know what situation he's in. And then it zooms out and it turns out he's surrounded by people playing instruments. They also urge you to try it with unexpected prompts to try and produce creative scenarios that you previously wouldn't have imagined, like this dude sitting on a massive rock shaped like a head. Regardless, I feel like this is one of those features that you never knew you needed in the first place. And now that it's here, I think it's going to be a welcome addition to the Runway lineup for Gen3. Also coming from Runway is something called Frames. It's a new image generation model that focuses less on necessarily being the most consistent or prompt accurate and more focusing on the stylistic control, which makes sense for Runway since they are more focused in the creative side of things, the filmmaking side of things. The demo video they have here shows off a variety of examples in a ton of different styles. Now, whether or not you can recreate these styles faithfully in mid journey, ideogram, whatever it is, remains to be seen, but so far it's pretty darn cool. They show off, you know, stamp collection style, for example, and this one looks like it has accurate numbers along with the details so it's no dumb slouch of a model. It seems like it can still hold its own impromptu, but with a focus that's more in the realm of film and more in the realm of creativity that stems from the film and art world. And I think this is a nice focus, it's a nice approach that differentiates itself from the other image generators of the world. It's not meant to compete at that same prompt coherency level. It's meant to compete in this area of stylistic control and evoking certain emotions with the images that it generates. And this kind of, I think, puts it more in line as a competitor to Midjourney rather than a Competitor to Dall E3 or Ideogram. It's more about the style, it's more about the feeling, less about actually getting, you know, some wild prompt through like a kitten surfing with a lemon on its head eating a hot dog or something crazy, right? I will say though, for an example like this, where we have the disposable cameras, it's absolutely not the first model fine tuned to produce images that evoke this sort of emotion. There are glyphs created by my very own community on Discord that do exactly this, and they do it pretty much as well. And also those ones are completely free to use. So while this is very, very cool, tuning to a specific style certainly isn't something that is brand new to the AI generation world. Again, might have to do a deeper dive, deeper look into this. But hey, one thing you can't say about RunwayML is that they don't ship, because they certainly do. They've been on a rol lately adding new features and releases to the platform. So good on them for that. Continuing on with more updates from the AI video world, Luma AI upgraded their Dream Machine and Minchoi has a pretty great thread breaking all of this down. For example, you can combine the Mona Lisa and some abstract art and mash these styles together along with adding bird like features, and it turns out it does a pretty decent little mashup. Most of the examples we see here decide to take the stylistic choices from the Mona Lisa and apply some abstract elements. But we did get an example here that used the abstract elements the other way, which I thought was pretty cool. Following in the path of Vidu, which we talked about recently, consistent characters also seem to be somewhat possible with this new Luma update. You just drop an image of the character in and it will go ahead and generate multiple images of that specific person in all the different scenarios you propose. They also now have start and end frames which a lot of other video generators already had, but it's good to see this feature added and it's good to see them staying competitive in that landscape. One underrated thing that they did that I found pretty impressive is the fact that you can just chat with it in your natural language the way you speak to a person. Can I have the flowers in this way? Can you make it like this? Add this over here and it will do its best to accommodate you and I think that's a lot easier and a lot more natural. Anyways, I highly suggest you check out this thread for more. Again, Minchoi is another one of those accounts that I really recommend following to stay up to date in the AI space. People seem to be having a lot of fun with it so far, mixing and mashing different styles into different characters and ideas around to explore the latent space built inside of Luma's Dream Machine model. Pretty darn cool. Moving on, it seems like there is a lot of competition brewing in the reasoning and chain of thought area of large language models. People saw that OpenAI had their O1 series of models with chain of thought and reasoning and competition is catching up quick. We've got an example here coming from Chubby on Twitter. This is a model by Myza and they're calling this Agentic Function Studio. It does reasoning very similar to OpenAI where it just mysteriously does reasoning in the background and you don't really see the chain of thought and then it plops you an output. Although in the UI it does seem like you can check the reasoning process, which is good. Chubby reports that they got early access and the classic reasoning tasks like how many words does your answer have and each sentence with the word Apple or which number is greater, 9.11 or 9.9 were all answered with 100% accuracy, similar to the OpenAI models. And this company, Maija isn't the only one. Matt Schumer, for example, has just released Open Reasoning Engine. It's an open source test time compute engine that works with any OpenAI style model. As Schumer states, it's an early experiment, so this one isn't perfect, but inference time, reasoning, inference time, chain of thought is really catching on in the large language model space. Moving into some smaller tidbits, we've got Google releasing code base upload for Gemini, which allows you to upload a full folder with up to 1000 files inside that folder and 100 megabytes total so you could send it some decently sized projects and it apparently looks like it might be able to work with them. To what degree we don't really know. Testing Catalog notes that with very large code bases it seems to break down, but it works good with smaller ones. Apparently it's also giving signs of quote unquote reasoning. While Gemini is operating with RAG people from the community or noting here that Google is working on stuff they plan to ship competitive models, whether they get there or not remains to be answered. It is also worth noting here that this year in AI isn't done yet. The AI community as a whole is expecting releases throughout December. People are expecting OpenAI to release Sora in some public way finally, and not just, you know, a leak. People are also expecting OpenAI to release the full 01 model, which we still haven't seen yet. Again, advancements from Google are being teased. We still haven't seen Anthropic's Claude 3.5 opus. We also haven't seen the ElevenLabs Music Generation model. Udio also hasn't shipped a new model in a while either. There's a lot of hype. There's a lot of hold on the line for big releases and big drops before the end of 2024, and maybe things could roll into 2025 as well. Trust me, in the AI world, sometimes things take a little bit longer than usual and they don't get released as planned. So we'll see what happens in the next few weeks leading into 2025. I think there will be some pretty big releases and I'm really wondering if any of the big players in the game have anything up their sleeves, you know, something they haven't told the public about, something that no one is really thinking about and it's gonna blow our socks off. These AI companies, man, I'm telling you, they love to do this all the time. They keep things hidden under wraps and then release them and the whole space goes nuts. Finally, we're going to move into our final piece of news for the day. This is from Anthropic. They have added a new ability to tailor Claude's responses to your own personal writing style. I've used this and it actually really impressed me. As you can see, you can actually select the style they have normal, concise, explanatory, formal. Or you can create and edit your very own style, which is obviously the most fun. The custom styles are simple. You can basically just drag a piece of writing that you had done in the past, whether it be an essay or if you're just spewing thoughts down and it will really try to grasp and understand it and take what makes that specific writing special and translate it to a new task. It's quite good. I think it works more proficiently than just asking the AI to directly do so. So, such as if you went in chatgpt, pasted, you know, an essay in and said, hey, write me, blah blah blah, but use this writing style. They've done some work under the hood here to make this pretty seamless and basically just work right off the bat. I have been testing it and I can definitively say it's one of those Chef kiss moments where it's a new feature, not a huge feature, but something you never really knew you needed. And it makes generating with the AI a lot more fun. For example, if you paste in some text that sounds very intelligent or text that literally reads intelligently like it's intelligent text with self reflection.