A YouTube video by LARRUZ (Dr. Zsolnai) demonstrating NVIDIA’s new sound foundation model (referred to as Fugato/Fugaro). The presenter walks through five example demos — text-to-sound, a text-to-sound morphing into orchestra, speech synthesis, music augmentation (adding drums, transforming piano to singing voice), and novel instrument mixes — and highlights capabilities like voice isolation, emergent generalist performance that can rival specialists, and small-model efficiency (some models may run on phones). The video includes links to the research paper and demos and finishes with a call-to-action and sponsor mention.
Chapters
This is Fugaro Nvidia's new AI and they say it is the Swiss Army Knife of sound. But why does this even make sense? There are previous AI systems that can generate sound effects, even songs. So what is new here? Well, check this out. Oh goodness. So previous systems can do a lot, but none of them can really do everything. But this one can. So they say. Well, let's have a look together through five amazing examples and see for ourselves. First, tech text to sound. Just write what you wish to hear and. There you go. Hopefully it does not work for a two minute paper script to put me out of business. We'll see about that. Two Text to sound. But this time crazier. A train passing by and a sound morphing into a string orchestra. This sounds impossible. Let's listen. Wow. Now let's do something crazier. Wait, what? Synthesizing. Someone talking. Kids are talking by the door. Okay, I am going out of business. But wait, maybe not yet. Because some of you say that the best part is. And I put some so much emotion and enthusiasm into these videos. That's always going to work. Well, check this out. Kids are talking by the door. Oh goodness. That is really convincing. Not perfect yet, but if you apply the first law of papers. Yes, a couple more papers down the line and it will surely be excellent. Now three, on to musicians. Imagine that you have a little something down, but not a full song. Now let's add drums to it. But it gets crazier. Way crazier. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Or get this four, this is going to sound like science fiction. Just play something through the piano and transform it into a singing female voice. Foreign. With a tool like this now we can all become musicians. And we don't need expensive hardware and instruments to make it happen. That is so cool. Or five, this is one of my favorites. Remember, they promise emergent properties. That means that it can combine two things without having heard that combination. And that would be the definition of some sort of intelligence for me. So I am super excited about this. For instance, it can put together a mix of instruments arranged in unexpected ways. This comes out nearly instantly. So now hold on to your papers, fellow scholars, and just let your imagination run wild with a howling saxophone. Or of course, let the memes flow. With this one, a dog barking over electronic music. Fugato is a groundbreaking foundation model that gives you sonic superpowers. It can also perform voice isolation, so when you drop in a full song, it can isolate and get out Just the singing or for karaoke purposes, just the instruments. I am unable to show you this here due to YouTube reasons, but of course I put a link to it in the video description and we are just getting started as the research paper reveals some unexpected insights. Look, this is incredible. The new system Fogado is as good as other generalist systems. Systems that can do multiple things. But what about the specialists? AIs that can do only one thing, but they do that very well. Those are the best, right? Well, hold on to your papers, fellow scholars, because occasionally this one can be those two, which is absolutely stunning. A generalist system beating specialists. That is crazy. These systems are super optimized for one thing, and the generalist can still win. Imagine an Olympic swimmer suddenly walking over and winning a medal in wrestling. The that would be probably impossible. And that sort of thing happening here in AI is absolutely insane. What a time to be alive. And here comes perhaps the best part. Oh my goodness, look at that. These don't require supercomputers. These are tiny, tiny models. Two of the three AI systems may be small enough to run even on your phone. I can't believe that. So cool. And just imagine combining it with ChatGPT to write the lyrics for you. Add the lyrics here and you have your own song. Or just imagine combining it with the notebook LM AI that can create text or even a podcast about a technical research paper. And before you ask, yes, it is me in human form speaking into the microphone, trying to hold on to my papers and flipping out every episode. So what do you think? What would you fellow scholars use this for? Let me know in the comments below. We need new tools for the era of LLMs and weights and biases. Now has Weave a lightweight toolkit to confidently iterate on LLM applications. Use traces to debug how data flows through each step of your app, and use evaluations to measure your progress. It is the best. Try it out now at WNB Me papersllm or click the link in the description below.
