Google's NEW TERRYIFYING AI 'Soundstorm' Shocks The ENTIRE INDUSTRY! (NOW ANNOUNCED!)

So a couple of days ago, Google actually released a stunning new paper which actually does host some potentially terrifying impacts for society. But it wasn't picked up by the wider AI community since this wasn't a tool that was released, rather a demonstration of what is now capable with certain AI software. The tool in question is Soundstorm, a new AI tool that can leverage current AI capabilities to create hyper realistic sounding voiceovers with a, a new sort of architecture. Now this is very interesting because it does link to many different previous AR voiceovers that we've heard in the past, but this one is definitely striking in the terms of accuracy that we do see. Now in this video I'm going to show you the tons of different examples that we can see and why this is possibly one of the most dangerous pieces of AI to come out, because the implications which they discuss in the paper are actually quite bad if it does get into the wrong hands in terms for wider societal impact. So let's get into this because this one is very interesting, but the demo you're about to hear isn't based purely on how the voice sounds. Now try to understand the small inflections you hear when someone is speaking, for example, the pauses, the urms, the little things that make a voiceover human. And I'm going to show you another example from Google that demonstrate this in full effect, which they released a couple of years back.

Did you hear about Google's paper on Soundstorm?

Um, no, I must have missed it. What's, what's it about?

Well, it's a parallel decoder for efficient audio generation, so it can even be used to generate dialogues.

Oh, interesting.

Yeah, yeah. Like this one was generated by Soundstorm.

Wait, what, what you just heard there was the official demo that they had on their research paper. Now there are tons of more demos which you will get into, but before we do, I just want you to, you know, pay attention to, to the little inflections that we hear when someone speaks, because these are the important things that determine whether or not someone interprets a voice that sounds human or not. But Soundstorm manages to capture that very, very well. So I think that it's very, very interesting as to how it does that. So let's look at some of the other things on the page and see just how realistic this stuff is. So there's going to be three parts to this that we need to talk about. And the first part is the dialogue synthesis. So Soundstorm, what it can really do really well, and something that I found very interesting was that it can actually Generate dialogue from two people talking just from around three seconds. Now, you might be wondering how does this differ to 11 labs? This actually makes it sound really, really realistic in terms of the little inflections, like we talk about four in terms of the little inflections that we hear in the dialogue. And it sounds really realistic. So this is the text, this, this is the voice prompt. And then of course, we have the synthesized dialogue. So what you're about to listen to is the original recording. And then you're about to listen to the synthesized dialogue, AKA the AI generated dialogue. So he listens to the voice prompt.

Where did you go last summer?

I went to Greece. It was amazing.

And then what we're going to listen to now is of course, the synthesized dialogue.

Where did you go last summer?

I went to Greece. It was amazing.

Oh, that's great. I've always wanted to go to Greece. What was your favorite part?

It's hard to choose just one favorite part. But yeah, I really loved the food. The seafood was especially delicious.

Yeah.

And the beaches were incredible. We spent a lot of time swimming, sunbathing and exploring the islands.

That sounds like a perfect vacation. I'm so jealous.

It was definitely a trip I'll never forget.

I really hope we'll get to visit someday.

Now, some people might think that's not impressive, but the thing is with that is that that's pretty insane. Now, I'm not sure if this is linked to Google's previous AI voiceover software in which they demoed ages ago. And when I'm talking ages ago, I think it was around 2018, which they demoed something that sounded so realistic it genuinely freaked me out. And anyone I showed to it freaked you out. I'm going to show you a little clip of that. But when I show you a clip of that and then replay you the clip here, you're going to realize just what these implications are.

So let's go back to this example. Let's say you want to ask Google to make you a haircut appointment on Tuesday between 10 o' clock and noon. What happens is the Google Assistant makes the call seamlessly in the background for you. So what you're going to hear is the Google Assistant actually calling a real salon to schedule the appointment for you. Let's listen.

So how can I help you? Hi, I'm calling to book a women's haircut for a client. I'm looking for something on May 3rd. Sure, give me one second.

Mm.

Sure.

What time are you looking for Ralph? At 12:00pm we do not have a 12:00pm available. The closest we have to that is a 1:15. Do you have anything between 10:00am and 12:00pm depending on what service she would like? What service is she looking for? Just a woman's haircut for now. Okay, we have a 10 o'.

Clock.

10Am is fine. Okay, what's her first name? The first name is Lisa. Okay, perfect. So I will see Lisa at 10 o' clock on May 3rd. Okay great. Thanks. Great. Have a great day.

Bye.

So yeah, just there you were able to see exactly how realistic this can sound when put in the right environments. Now of course we're going to talk about applications later but I just wanted to show you that when you actually know the applications of this kind of stuff, it definitely is pretty crazy. So let's listen to another example right here. We're going to look at the voice prompt.

Something really funny happened to me this morning.

Oh wow.

What?

And then let's listen to the synthesized dialogue.

Something really funny happened to me this morning.

Oh wow.

What?

Well I woke up as usual.

Uh huh.

Went downstairs to have breakfast.

Yeah.

Started eating. Then 10 minutes later I realized it was the middle of the night. Oh wait, that's so funny. Something really funny happened to me this morning.

Oh wow.

What?

Well I woke up as usual, went downstairs to have breakfast.

Yeah.

Started eating, then 10 minutes later I realized it was the middle of the night.

That's so funny.

So yeah, I listened to both of those and it's interesting because the second one did sound a lot better. But I gotta be honest, this is definitely just. Just a little bit too realistic for my liking if that is an AI voice. Because if I was listening to that conversation, I would definitely say that those people are human. So this is definitely starting to freak me out just a little bit. It's definitely impressive I would say, but at the same time it's a little bit scary. Also listen to one more example and I think you guys are gonna like this one because this one also does provide more context as well. And, and it does sound really realistic.

I'm going to Istanbul for the Champions League final.

That's awesome. Who are you supporting?

Liverpool.

I've always been a big fan.

Oh Liverpool is a great team. But I think it will be a close match.

Yeah, I can't wait. You know I'm super excited to be going there.

Yeah, I can imagine.

Are you coming as well?

No, unfortunately I can't now I gotta.

Be honest, the mail in this one right here definitely sounded really, really realistic. And like I said, even if the text is great. It's definitely starting to sound a bit too realistic now. There are some subtle cues that this is AI generated. Like some of the audio does sound a little bit. I'm not sure what the word is, not crisp, per se, but it does sound just a tad bit generated in terms of the actual quality, but not in terms of the linguistics and the way that they speak, if that makes sense, any sense. But we're gonna move on now. And I do think that this one right here was pretty interesting. And if you want to know how this will work, like we said before, the text was controlled by Google, they of course had two people speaking and then they were able to replicate that conversation. And I do think that one thing that's going to be interesting was to see how exactly this is going to be when it comes to video games. Because, as you know, AI generated conversations, real time could be something very interesting in terms of a new world. And we did see that recently with Nvidia, which we will be making a video on, because Nvidia demoed a huge amount stuff at their recent conferences just to provide us with a lot of stuff in terms of the AI future. Then essentially we have the paper scroll on down to here where we've got these bass lines. And essentially we look at the original audio. Then we look at what these other AI tools can essentially replicate when trying to make the voiceover sound exactly like the original audio. So here we have the original audio.

His heart full of charity and severity at the same time, same time.

Then we have audio LM descend with.

His heart full of charity and severity at the same time.

Then we have Greedy descend with his.

Heart full of charity and severity at the same time.

You can see here that in Greedy's example, it does sound a little bit more robotic, which is one of the things I did talk about earlier, that sometimes you can hear that sort of echo, roboticness sound in certain kinds of voiceovers. Then we have sound storms.

He must descend with his heart full of charity and severity at the same time.

So another thing that we did want to talk about was Prompted and unprompted generation. So with Soundstorm, essentially, and I need to show you guys another example because this is actually really, really interesting. So you can see right here on the left, this is the original. Then this is the unprompted, and this is the prompted. So essentially what this is, this is the original audio, which is taken from a huge library of audio spoken by a variety of different people and is kind of like, I guess you could say a benchmark of what they use to see how people actually sound compared to the AR voices. And the original samples are taken from Libri Speech Test clean. And that's the sample thing. So essentially right here, this is the original. And then of course, you can get it converted into other human voices. So let's listen to the original. Then we're going to listen to how the AI voice mimics your own voice. Okay. And what you'll notice is that the original and the prompted sound exactly similar and. And the unprompted actually do sound like other people as well. So let's actually listen to the original.

Mr. Meadowcroft, the elder, having not spoken one word thus far, himself introduced the newcomer to me with a side glance at his sons, which had something like defiance.

And then let's listen to the unprompted. So this is going to be sounding like another person.

Mr. Medicroft, the elder, having not spoken one word thus far, himself introduced the newcomer to me with a side glance at his sons, which had something like defiance and glance, which now we're going.

To listen to the prompted one, which is essentially the AI voice that says sounds like the original voice.

Mr. Meadowcroft, the elder, having not spoken one word thus far, himself introduced the newcomer to me with a side glance at his sons, which had something like defiance in it.

So essentially why this is very, very interesting is because when we have the prompted and unprompted generation is that you're able to generate high quality voiceovers with only three seconds of voice. Now, one thing that I do find interesting about this is that in the testing of this is, is that we don't see examples compared to other softwares, for example, 11 labs. So what I'm going to do is I'm going to download the original sample, then I'm going to go ahead and see how this compares to 11 labs. Because if I show you quickly on 11 labs, the voices that I've cloned, they actually sound really well. So I'm going to show you a voice that I personally cloned of Joe Rogan and one of Donald Trump to show you what they sound. So like we said, it would have been good if we had the benchmarks against some popular figures because then it would be interesting to see how, exactly how they sound. But of course, as you know, they decided to use that specific cohort of voices because those are the ones that are the industry standard. So this is 11 labs. Many people, if you're not familiar with it, it's a voice spins, it's a voice AI tool. And essentially if you want to clone a voice, it's very, very simple. All I had to do was simply click add a voice, then just click add. Then, of course, for example, if you wanted to clone Joe Rogan or anyone, you'd simply type in the name right there. Not if the name has anything to do with it. You can call it what you like. Then you just upload a file. You don't need to upload 25 samples. You can literally just upload a standard sample and then essentially just within a matter of seconds, you essentially have something like this. So I'm going to show you the clip that we just got.

Hey everyone, welcome to the Joe Rogan podcast, episode 100. We are here with the AI grid and we're going to talk about artificial intelligence.

So I think that does decently sound like Joe Rogan if you ask me. I mean, other people will be able to tell that it's an AI voiceover, but it is still very, very good from what we've come so far. And I think with Soundstorm and other technologies, it's also going to sound very good. Now what I'm also going to show you is one more example, and this is of Joe Biden because more, because more people are going to know exactly what he sounds like.

Welcome to the United States, the greatest country in the world. And today we are speaking with the Igrid.

So I think that one there also did sound quite realistic. You have to understand that certain ones won't sound that realistic. But the point of this is to demo and show you how these tools actually can be used in very, very quickly. Now what I'm going to do now is just show you some examples of where this was essentially used malicious. So you can see from this Washington Post article, it says they thought their loved ones were calling for help. It was an AI scam. Scammers are using artificial intelligence to sound more like family members in distress. People are falling for it, losing thousands of dollars. The story goes on to say the man calling Ruth Card sounded just like her grandson Brandon. So when he said he was in jail with no wallet or cell phone and needed cash for Bailey, she scrambled to do whatever she could to help. It was definitely the feeling of fear. She said that you've got to help him right now. So this 73 old woman and her 75 year old partner ran to their bank, withdrew 3,000 Canadian dollars, which is around $2,200 in US currency, which is the daily maximum. Then they hurried to another branch for more money. But essentially a bank manager told them that this was a scam because they'd learned earlier that someone else had a similar scenario and, and that this person that was on the phone wasn't their grandson. And that's when they realized that they had been duped. And in an interview they were like, you know what? We've been duped. And this is why these tools are very, very scary. But at the same time very, very good. I mean, it's good for content creation. For example, I could clone my AI voice and then essentially maybe if something happened to my voice, or potentially if I had a cough or something, I could simply use that to write the script. And then essentially I could read out that script and people wouldn't know, which could essentially save a lot of time because voice actors are very expensive and sometimes pronouncing words is qu. So with a technology like elevenlabs and of course with technology like Soundstorm coming out, it's going to be very, very hard to see exactly what kind of things are going to be real and what kind of things are going to be fake and what kind of identity verifications we're going to need in the future. So for example, if we check out this article here from the Guardian, it also entails us of some of the problems that are going to come up. So if we check out this article by the Guardian, it says AI can fool voice recognition used to verify identity by Centrelink and Australian tax office. It says a voice identification system used by the Australian government for millions of people has a serious security flaw. It says essentially this government office uses a voiceprint with other information. But the problem is, is that the voice print is just someone's voice over the phone. And of course we know now with these AI generated programs that a voice trained to sound like a specific person could be used to access phone banking services overseas. And their system was able to be fooled by this AI voiceover. So we know that this is going to be a problem in the future. One thing that I did want to talk about and touch on and something that was very, very, very important because I'm going to show you. Well, link to another video in the description because it's a really important video that you do need to see and not enough people have seen it. Essentially this is the broader impact. So it says Soundstorm is a model for high quality, efficient generation of neural audio codec derived audio representations. Yada, yada, yada. So basically, okay, it does essentially say here that this is dangerous. So it says in Turn. The ability to mimic a voice can have numerous malicious applications and including bypassing biometric identification and for the purpose of impersonation. Thus, it is crucial to put in place safeguards against the potential misuse. To this end, we have verified that after replacement, the generated audio remains detectable by a dedicated classifier, 98.5% using the same classifier as Boros. Hence, as a component of a larger system, we believe that soundstorm would be unlikely to introduce additional risks to those previously discussed by Boros. So essentially, this broader impact talks about how if someone is able to clone your voice simply with three seconds, which people are currently doing, and if they're able to get it at this level, where it sounds even more realistic in terms of the levels and how you talk, it's going to have potentially malicious applications. But they also talk about how that even when you generate audio through soundstorm, you're able to detect this if it's generated by AI with a dedicated classifier. Now, I do want to be honest, the only problem I have with this is that and one of the recent examples that we did see where people have been scammed by, you know, their grandsons or relatives that have been asking for money. If you haven't been made aware, there have been several scams that have been going around, essentially where someone may receive a phone call from a loved one. Now, if you receive a phone call from your loved one and you hear their voice on the phone and they're asking you to transfer money into a certain account, you might not think twice about it. And that's happened to many different people. And with the rise of the voice cloning from many tools like 11 labs, this has been made free on a global scale. Now, of course, with every great tool, there's always going to be some misuse of the application, but I think with this, the risk is definitely higher. Now, it does say that it's unlikely to introduce additional risks, but I guess we're going to have to see that's where this product comes in. Now, most of you in the AI space probably haven't heard before this, but what's interesting is you've likely heard of the person who created this. This is worldcoin, essentially created by Sam Altman. If you don't know who Sam Altman is, it's the guy that created ChatGPT. Now, what Worldcoin is, it's essentially a project that is going to fix digital id, allegedly. And essentially how it works is it's an orb that scans your eye, and that is the only form of ID that you're going to have. And it's going to be kind of tied into this huge, I guess, digital wallet. And you can see right here that it's essentially a digital passport that lets you prove you are unique and a real person while remaining anonymous. And you can see right here that this is essentially how this work. They have this orb, essentially, it scans your eye, and then essentially in the database you have your digital id. And that's how they're going to prevent, or they're working on preventing this kind of AI revolution where we're not going to know who is who. So let me know if you're excited about the World coin or are you a bit on the other side. Maybe you don't want a digital id. Maybe you think this problem isn't a big problem. Are programs like Google, Soundstorm, and for example, 11 labs, where we do see professional voice cloning, which is going to start very, very soon in July, where it's going to be indistinguishable from the reality. Are these things likely going to be problems in the future? I'd love to know your thoughts and comments down below because it is definitely a very interesting debate. As the rapid rise of AI increases, all the kind of different problems that we didn't see before starting to come to light.