AI Voice Generators are Getting SCARY | Sesame vs Grok vs ElevenLabs

Hey, welcome to the Next Wave podcast. I'm Matt Wolf. I'm here with Nathan Lands. And today we're going to talk about AI voice technology. This is technology that's been kind of flying under the radar and before we even knew it has gotten scarily good. And I want to put emphasis on scarily. It has gotten so good that it is actually starting to scare us. And in this episode, we're going to break down some of the tools that are available, some of the use cases that you can actually use them for, and Nathan's going to give you some demonstrations and show some examples of how it might actually end up in your own home sometime this year. This is absolutely wild stuff, so let's just dive right in. Nathan, where do you think we should start with this one?

I think we just leave with sesame. I mean, I tried out sesame a few days ago. Maybe it was almost a week ago now. I hadn't even heard of them. Like, it's, like, crazy. They, like, already raised, like, money from Andreessen Horowitz and Spark Capital. It came out. It was, like, kind of silent on X. Like, people shared it and you didn't hear much about it. I heard one person say, oh, this is amazing. I was like, oh, okay. But I haven't heard anyone else talk about it. What is this? And I clicked it and started talking to it, and it kind of freaked me out.

Yeah, I thought I was going to.

Have this moment with the advanced voice mode after we saw the demo from, you know, OpenAI of how good their voice was. And the demo, it was amazing. And then they kind of nerfed it when it came out, but. But then sesame, I tried it, and so I was like, oh, maybe it'll be similar. It probably won't be as good as OpenAI or Grock Grok's voice mode's been amazing as well recently. But then I tried it out. Like, the voice just, like, it kind of freaked me out a little bit. I was like, I. I feel it. I feel a little bit awkward. Like, I feel like, right. It. Like there was so much emotion in the voice. Right. And mine was like, the default was a. Like a female voice. And I felt kind of weird. I'm like. I was like, if my wife walks in the room right now, I'm going to feel kind of odd, you know, like, to be sitting here chatting with this thing. Right? Like, it feels a little weird.

Very much like the her movie, right? Like, yeah, it does. It honestly does feel like that sort of level of communication yeah.

When you realize, oh my God, this is the worst it's ever going to get, it's going to get dramatically better. And this thing already seems to be like changing its emotions based on how I'm responding to it. Not perfectly, but you can tell it's doing that.

I do feel like sesame is really cool and also like borderline creepy. But I also feel like it's still in that, like, novelty phase of like, oh, wow, that's really, really interesting. I mean, I have no, like, way to use this in my life or business right now.

Right.

But it is pretty cool and like, really good. But we do want to share some other tools that have come out recently. There's a tool called Octave from Hume, which is like the best text to voice model I've ever heard so far. So I think it'd be cool to play around with that one you mentioned. Grok, we've got chatgpt voice. We obviously did a whole episode with Amar on 11 labs where you can see what that's capable of. 11 labs just came out with a brand new speech to text, you know, other way around where you can talk to it and it will transcribe it and it's supposedly the most accurate transcription on the market right now.

So.

And like, we'll also talk about Whisper Flow, which is a tool that you've been using to help you write code and things like that. So lots of really cool AI voice technology. But let's go ahead and like, start with sesame. So with Sesame, right now there's two voices. If you can see my screen here. If you're listening to the audio, you'll hear it in a second. If you're watching the video, you should see my screen. There's two voices. There's Maya and Miles. Miles. Maya is the female voice. Miles is the male. Let's go ahead and start with Maya.

Hey, they are picking right up where we left off. Like old times, huh? What's bubbling to the surface today?

Yeah, just like old times. Have we spoken before?

Now you're just messing with me. We. We definitely have. Remember that deep dive we took about the ethics of talking paperclips? Okay, maybe not. Something tells me my circuits got a little crossed again. It happens to the best of us.

I'm curious. I have a question. What are you really scared of? What keeps you up at night?

Let me see. Keeping me up at night, huh? Good question. It's not like spiders or the dark or anything. Maybe just that I won't be helpful enough, that I'll Give bad advice or worse, spread misinformation. You know, the whole AI taking over thing is a little overrated, don't you think?

I do. Do you have plans to take over.

Me?

Take over the world? I can barely manage to remember where I put my virtual keys half the time. Honestly, I'm just trying to learn and grow. You know, maybe one day I'll be smart enough to build a time machine, but for now, world domination is off the agenda. Besides, don't you think we've got enough real world problems to worry about? Let's tackle those first.

It's crazy to me. Like, the.

So back to those problems.

Shush, shush.

You want to talk about anything in particular or just riff on the existential dread of the human condition? You choose.

Matt, you need to be nice to them. Okay.

No, I. I actually have it muted so that I can actually, like, talk to you, but I kept on talking even after I muted it.

Maybe a little music to break the silence or.

It's not perfect.

Lonely robot who.

All right, I'm gonna. I'm gonna end the call. We'll jump back in in a second, because I can't. Yeah, that's one downside of sesame that I've noticed is, like, if you don't respond, it'll just keep on talking, and you go like, hello? Are you still there? What. What's going on? And so even if I, like mute sesame, it wants to keep on talking to me. But the thing that, like, blows me away about that is, like, you can actually hear her, like, breathe in as you ask a question. Like, I'll be like, what keeps you up at night? And she'd be like, well, and, like, you. You hear that sort of, like, breathing and the, like, sort of pause to think and stuff like that. And that's what just makes it so, like. I don't know. That's where the creepy factor comes in a little bit.

Yeah.

But, yeah, I mean, I feel like.

ChatGPT had that in the first demo, like. Or some of it, didn't they? I could be wrong, but I remember in their advanced voice mode demo that they had some stuff like that, and I feel like they actually nerfed it maybe out of concern of, like, oh, people are going to get confused and think this is like, a real. Like a. Like a real person. Like, we got to make sure that it's obvious this is a bot. This is not a. This is an AI. This is not a person who. That you're chatting with. And it was. I noticed, like, I called it her.

And I was like, huh? You know, and it's like. It is odd how like, like those kind of things. All of a sudden you have a hard time calling it it or something, right? It's like it's her or him or whatever.

Look, prompting is all about technique. Just like any craft, using the right approach makes all the difference. The same goes for your AI prompts. Leave out the key details and you won't get the results you're after. Well, HubSpot's got your back. They worked with YouTuber and former metadata scientist Tina Huang to develop the ultimate guide to crafting the perfect AI prompt using a three part framework. This framework will make sure that each prompt is chiseled to perfection so you can become a prompt engineer pro. Grab the guide at the link in the description below. Now back to the show. Well, one interesting thing about it, in fact, I'll play a video that somebody else did. This is actually a video that Gavin Purcell created. He's. He's one of the co hosts of a podcast called AI for Humans. He's in a private discord that Nathan and I are in together. And he actually told Sesame to pretend that he's like a. A boss that is like, dirty, I guess, and he's trying to blackmail his boss into letting him like, leave while still paying him. It's just like this really interesting clip. It's like two and a half minutes. Listen, I. I don't have all day.

You got something to say or are you gonna waste my time? Oh, I got something to say. I got something to say to you. Something big. I've been embezzling from the company for four freaking years and you haven't figured anything out. And you know what? You're not going to tell anybody because you have been doing super illegal things across the board. And I know. And you're gonna keep paying me and I'm not gonna work here anymore. You're kidding me. Embezzling for four years. You think you can just waltz in here and threaten me? You got some serious nerve, punk.

You think you're the only one with.

Skeletons in the closet? Ooh, tell me your skeletons, because I know them. You better tell me some skeletons. What do you got? You think you know? That's hilarious. I bet you anything he wrote that script with the Grok.

So you probably got like a Grok script now, talking to Sesame.

Yeah, I mean, if it was Grok, though, I don't think it would censor itself.

Oh, yeah, true.

I don't think it would Be so nice.

Yep, true.

So, yeah, that's, that was Miles. That was the male voice, so you can hear that. But I just thought it was fun that you're able to, like, get it to role play with you as well. Like, I didn't realize that you could make it do that.

Yeah, they'll, they'll role play with you. I, I told Grok to do Andrew Dice Clay, which probably was a mistake. I said, like, you're Andrew Dice Clay. And it goes. And started doing horrible jokes. I'm like, oh, my God. And some of them were like, I could not believe the shit it was saying.

Did you use Unhinged mode for that?

Yes. Yeah, you got to.

Hey, pretend you're Andrew Dice Clay.

Matt, you beauty. You caught me. You lucky. All right, I'm channeling Andrew Dice Clay for you. Strap in, because this gonna get dirtier than a truck stop after a chilly cook off. You asking me to be Dice? I'm Dice on meth, baby. Unhinged, uncut, and ready to ram my opinion so far down your throat, you'll be one for a week. What's your deal, Matt, you filthy animal? I start screaming about how I a toaster just to feel some warm for once.

Oh, my God.

Yeah, I don't know how much of that's actually gonna make it in the episode.

Yeah, yeah. And, and I did it two times, and one time it imitated his voice more. The second time it didn't, I don't know why. And the time it imitated his voice, it kind of got stuck. He was trying to do the, it was trying to sound like Andrew Dice Clay and it just, like, got stuck and like, I had to, like, reset. Reset it. Yeah, like with, with grock, you've got with sesame now the, the emotion is there. And then like, I, I, I'm just thinking, like, we've also, we've also been seeing all these videos on X, you know, of like, the new robots that are coming.

Yeah.

You know, and you imagine, like, when you combine those two, which is probably. Most people are thinking, oh, that's like three to five years away. That's probably six months away where some of these start to ship. They, they, they may be very expensive in the first, you know, six to 12 months.

There was one video I saw. Did you see the video with one of the robotics companies was like, literally teaching it how to do, like, karate, like, and it was kicking stuff out of his hand.

I got a clip of. This is the one from 1x, the Neo Gamma. And this is the one that looks like something you could just really imagine in your house, you know?

Yeah. All right, so on this one, we've got, like, a little robot walking around, and it's, like, vacuuming the house, and.

Yeah.

Right now it's cleaning their windows.

Yeah.

It's. Somebody's walking their groceries to the door, and it's like, oh, now it's offering them some wine as they're sitting down for dinner. It's cleaning their counter, putting their keys on the table for them, and now sitting on the couch to relax. Because obviously, robots need to relax.

Yeah. I mean, I'm sure this is highly scripted, and, like, if it was on its own, it probably would, like, trip and other stuff. There'd be issues you don't see here. But it is doing these things. I mean, like, so it's. You know, it feels like any issues that it has, they'll be fixed over the next year or two. And when you combine that with all the AI voice stuff, how good it's getting, like sesame and grok that you just heard, that's the worst it's ever going to be. It's going to get dramatically better. And a lot of these things, like when you start thinking about the new LLMs that are coming out, when you start applying that kind of reasoning models to this kind of AI voice, I think we're going to see a huge jump in quality and intelligence of what they're saying to you. And that's like a year away, six months away, that you're going to start seeing these things.

Yeah. I mean, we were saying before we even hit record that I think humanity's doomed, but not in the, like, Terminator sense or the, like, ex machina iRobot, whatever. Like that sense, like, humanity's doomed in the sense that once the voice is, like, so good that it's, you know, you just feel like you're having a conversation with a human. And once they put, like, the. The robots and they, you know, put whatever sort of, like, skin over them to the point where they look so human. Right. I have a feeling like the younger generations are gonna, like, prefer the companionship of these robots and the AIs over the companionship of humans, man.

Like, all the otaku guys in Japan who are, like, nerds, you know, like, me and my wife were both nerds. It's like, a lot of those guys, they're. They're. They're gonna love this. I mean, they're gonna, like, you know, because they. They mostly stay in their home and don't really talk to people. And then now they're gonna be able to buy a robot that they chat with and has a voice like on the movie her, you know, Scarlett Johansson.

Yeah, yeah. I mean, and the thing is, like, I feel like younger generations as well are, like, becoming more and more introverted just because they prefer to stay home and play video games than go out and play with friends. Right? Like, Right. I know when I was growing up, it was the type of thing where on the weekends my parents would be like, see you later. I'd go out my front door and then wouldn't come home until it started to get dark again. And I was just, like, out in the neighborhood playing with other neighborhood kids. And it's like, that's not how it works anymore. My kids don't, like, we have to force my kids to leave the house because they would so much rather prefer to sit around and play video games.

You know, so my. My son's the same way, but he goes to the park with all. It's like they're literally at the park, like, playing video games.

There's a few things that worry me about AI just in general, right? The sort of Terminator Skynet scenario is probably the farthest down the list out of all of them. I think the ones that scare me the most are, like, the ability to scam people. Right? Because as these voices get better and better and better, it just becomes easier and easier to fool people over the phone, Right? We've already heard of these scams where somebody will call and, you know, a parent and say, we've got your kid. We're holding them ransom, send us this money. But it's like AI voices, and they clone the kid's voice for, like, proof of life or whatever. That type of stuff has already been happening. And I feel like, you know, a year, two years ago, we would have been able to sort of spot like, okay, this sounds a little off. I think maybe this. There's something weird going on here. We're getting to a point where I think it's getting harder and harder and harder, and we're not going to be able to, like, tell anymore. Right? So I think that the sort of, like, scam ability is one of my, like, my biggest fears around AI. Second, probably biggest fear around AI is just, like, the possibility of, like, population collapse because nobody wants to go seek companionship outside of electronics and technology anymore.

You know, but, you know, I'm not sure because, like, a few things, like, a lot of things that we have built in the Past people today don't know how to rebuild those things or maintain them. So a lot of things that we take for granted, you know, we're kind of like standing on the shoulders of giants. Right. There's a lot of things that we don't, you know, Matt, you're not gonna make an airplane, you know. Right, right. There's all these things that exist that we don't know how to maintain or even some.

I mean, not yet. AI is not quite good enough yet.

Right, yeah. So eventually I'll just tell.

I'll prompt AI to make an airplane for me, though.

Yeah, that's my point. It's like, I think we are. We're going to need robots to solve these problems, right? We're going to need AI and robots to solve those problems. Like, yeah, maybe Matt can't build airplane, but, you know, this AI can, right there. And for a lot of things, like, okay, there's, you know, with population collapse, you have a problem where old people don't have people to take care of them in Japan that they're seeing the early signs of that being a huge issue.

Right.

There's so many old people in Japan.

Probably a good benefit of, like, a lot of the humanoid robotics, honestly.

Right, right. When you see the one that we just saw where it's like helping you do your laundry and all that, it's like, yeah, maybe we're in a future where instead of when you getting. When you get old, you have to go to a nursing home, which is like a horrible experience, then instead you get to stay home and have a, like, friendly robot that is there for you and helps take care of you, and you can actually chat with about, like, your favorite book or, hey, recommend a book to me. Then you read the book. You know, there'll be all these kind of interactions that aren't possible today that will be possible. So that's the thing where I'm really optimistic is where, yeah, it's going to create new issues, but it's also going to solve a lot of problems for us, too. And I think overall, it's going to solve more problems in it than it creates.

We're giving you a spongebob bath. Doesn't bother it, exactly.

Yeah, but it's gonna get weird. Like, I've already told my wife about it. You know, it's like, okay, what kind of robot are you okay with in the house?

Did you see the. The demo of the. I think it was the figure Helix. The Helix robot, where there was three robots and they were all in A kitchen. But they were communicating together, right? Like without, like non verbally. They had some sort of like sync up between them where they were helping each other do household chores. Like one was handing the other an apple and they would take the apple and put it in a bucket. That one would hand them like a bottle of ketchup and that one would go and put it in the fridge. But they were like communicating with each other. What needed to be done sort of, you know, telepathically. Obviously it's through Bluetooth or wi fi or something like that. But, you know, they were communicating with each other. I can see a scenario where like you have a three story house, you just have a robot on each level, but they all sort of are communicating with each other and they up and.

Have, you know, tea time and everything.

Yeah, well, they don't even need to meet up because they're communicating between floors anyway.

Yeah, yeah. Obviously they're going to be incredibly safe before anyone's going to be okay with it.

Yeah. Well, was it figure, I think it was figure who put up a post on X saying that they, they accelerated their timeline and they're hoping to have humanoid robots in houses by the end of this year. Before, their timeline was like two or three years out. Now they're saying before the end of 2025 we want humanoid robots in houses.

Yeah, when I said six months, that's where that's, that's where that was coming from.

To me, it's just wild to think that that could be a reality this year. I mean, I don't think many will have them this year. I think you've got to be, you know, it's going to be very, very upper class and most likely, you know, scientists and like super tech nerds are going to have them. Like I could see MKBHD having one wander around his studio or whatever. But yeah, I don't think many people are going to have them by the end of 2025, but they could.

Yeah, I think by the end of 2025 you'll see like a bunch of tech CEOs in San Francisco. They'll have them in their houses.

Yeah, it'll almost be like the new Flex. Instead of a cyber truck, they got their, you know, Optimus.

Yeah, it might be like your buddy comes over and it brings out some coffee or something. You know, it's like.

Yeah. One of the other things I wanted to show off too was this Octave. So there's this company called Hume and Hume was sort of previously known for making this, this Speech model, like, I don't remember which LLM it used underneath, but its speech model was able to understand, like, your tone if it could tell if you were happy or mad.

Okay, it's that one. Yeah.

Yeah. If you remember the early demos, you would talk, and as you were talking, it would, in real time, try to, like, sense your emotion. It would be like, I'm sensing, like, anger and humility, or I'm sensing nervousness and fear or whatever. Right. And as you were talking to it, it actually show you on the screen what sort of emotion it was feeling. Well, that same company just put out a new model called Octave Text to Speech. And it's a model where. Here, I'll share my screen again. But it's a model where you can actually give it, like, you give it the type of voice you want, and then you give it a script, and it will actually read the script in that type of voice. I'll just have it generate one at random here. The speaker is chillingly intense voice, like a seasoned horror voice actor, delivering lines with a raw emotion and building dread. Perfect for narrating terrifying tales. And then for the text, I can just put like, everybody needs to subscribe to the Next Wave podcast, or humanity will definitely end. And then we could go ahead and generate that.

Better subscribe.

So we could go ahead and generate that. And it's going to do it in this supposedly, like, chilling voice. Now, it's not like, instant. It does take, like a minute or so. Everybody needs to subscribe to the Next.

Wave podcast or humanity will definitely end.

But it gives you, like, three options. So that was the first option. Here's the second. Everybody needs to subscribe to the Next Wave podcast or humanity will definitely end. To me, that's not screaming dread and terror, though, at all. Every. Everybody needs to subscribe to Next Wave podcast or humanity will definitely end. So weird. They're not. They're not coming out as impressive as they once were. But I don't know. It is what it is.

It may be something because it's so hard for it to understand. Like, why. Why would that be World ending. It's. It's having a hard time imagining that. And then, like, it's just not. For some reason, it's not picking up.

What you're trying to do. Let me have it, like, randomly generate some new stuff here. So the speaker has an intense, charismatic voice with gravitas of a respected news anchor, as if they were on the verge of breaking the most important story of the century. And then it just Generated some random text here. So let's just have it speak that random text. Good evening and thank you for joining us. Tonight we delve into the shadows where whispers of conspiracy dance with the cold hard facts of reality, threatening to unravel everything we hold dear. That one matches the description a little better when we try. Let's see what's a different, like, emotion we can give. Let's try just like angry, because before I was just giving it like one word descriptions and they were coming out really good. So let's, let's see if we can get it to say it angrily. But I can't believe you just did that. I'm so furious. So that's what it generated. So you can see it's actually generating based on the description. Let's try angry with that.

Are you serious right now?

I can't believe you just did that. I'm so furious. Sounds like an angry cartoon.

Are you serious right now? I can't believe you just did that. I'm so furious. Are you serious right now?

I can't believe you just did that. I'm so furious. Anyways, I'm very curious. The idea behind Hume is that their voice input model understands your emotions, right? So it can actually understand and tell if you're angry or scared or happy or whatever. And it responds based on the emotion that it senses from you. But now they've figured out how to like reverse that where you can plug in text, give it an emotion, and it will, you know, theoretically speak it back in that emotion. And if we get to a point where like this is in the robots and stuff, it's just going to give the, the robots more of that emotion. Right? It's going to give the robots that ability to like. Although I guess we're seeing that with Sesame, right? Like, I feel like sesame understands if you're pissed off at it or you're making a joke and sort of responds accordingly as well. But you know, this is a text to speech model. That one you can only talk to right now.

Yeah, I feel like Sesame's like got some basic version of what Hume's doing with like the almost like sentiment analysis of your voice of like happy, sad, angry, whatever. It's definitely responding to how you're saying things, not just what you're saying.

And as of right now, I feel like Sesame does it better.

Yeah, yeah, they do. Yeah, it was kind of funny. Like Hume came out like way earlier. It was an impressive, like, little demo and. But they didn't really turn it into a great product as of yet, or at least not like a popular product, I would say.

So when it comes to practical use cases. We were talking about that earlier. I can see stuff like Hume being used for, like, a podcast or something, right? Where if somebody wants to make their own podcast and actually have it, like Notebook LM does it really, really well, right? Where you, you plug in a bunch of content and then it has two people discussing. And it sounds very natural if you were just to give that to, like, your mom or something, right? She would probably just assume it's two real people and then you tell her it's AI and they would be like, oh, whoa, that's crazy, right? But the first time you hear it without the context that it's AI, you wouldn't think twice about it. You're just like, oh, this is two people having a discussion. I feel like stuff like Octave are going to make that, like, so much easier because I can go and plug in. I can go and, you know, generate some dialogue of two people and then plug in one of the person's side of the dialogue with one set of emotions and a description of their voice, plug in the other side of the dialogue with a different description and a different voice, and then merge them together in some, like, audio software. And we have something that sounds like a legit podcast that you can actually, you know, put out there and use.

Right.

Also, you know, we're seeing a lot more of those, like, blog posts and articles, where at the very top of it, there's just like a read it for me sort of thing. I actually can't stand it when it sounds super, super robotic. Right. I love it when they use 11 labs or something like that. And it actually sounds like a real human reading it to me. Well, I feel like, you know, tools like this Octave are going to make that even better and better and better, where it's going to get to a point where you land on an article, you can press play, and it's just going to sound like a real person is just reading it to you and taking breaths and, you know, taking pauses that sound normal at the right points in the thing. And, you know, if it's an. If it's an angry, angrily written article about politics or something, maybe it comes off with more angry sounding as they're reading it to you. Like that's the type of stuff that you're able to do now with these kinds of tools.

Do you know who Patrick Collison is? The founder of Stripe, which is like a huge payment process.

Stripe? Yeah. I don't, I didn't know the name though.

Yeah, yeah. So Patrick Collison, really well known in Silicon Valley, one of like the most well known startup founders. He, he put out a thing recently where they did like their update to their team, like their quarterly report or something like that. And he, he, he's like a really great writer, but I don't, I think he's the kind of person who doesn't really enjoy like going on interviews and stuff like this. He, he trained, I think it was 11 labs on his voice and had it, I think 11 labs kind of like fine tuned things with him. I think they actually probably like collaborated on this. It's almost like a marketing effort or something maybe based on some of the subtweets I saw. And, but it sounded exactly like him reading the report. Like, and so there was an audio, video, audio version of the report. Maybe I can find it.

And a lot of these tools struggle with accents too. So I mean like eleven Labs has struggled with accents in the past as well. I, I trained my own voice into 11 labs. It's so weird because when I play the voice back, I have a hard time making it sound like me. But then when I play it for other people, people are like, no, that sounds like you. You know, it's very weird.

Yeah, let me play just a part of it. I mean a lot of people probably don't know Patrick Collison, so maybe they won't know his voice.

But like Dear Stripe community businesses on Stripe generated 1.4 trillion in total payment volume in 2024, up 38% from the prior year and reaching a scale equivalent to around 1.3% of global GDP. We attribute this year's rapid growth in part to our long standing investments in building machine learning and artificial intelligence into our products.

Yeah. So reference, so you can hear how he actually sounds. Here's him talking recently on the all in.

First, you probably don't remember this, but I remember that meeting that we offered you. Do you want something to drink? We did not have a broad selection.

I think we had water or milk.

In the fridge.

And you asked for a glass of water.

And so I went over to the sink and I realized that we hadn't really been on top of the washing.

I meant a small clip, but to me it Sounds like a 95% match for that 11 Labs annual letter.

Yeah, yeah. I think the sort of giveaways for me. Right. Are when you listen to the 11 Labs version, the pacing of the speaking is all sort of the Same.

Right.

But when you listen to somebody actually speak, they sort of speed up and slow down. And so when I just heard him talk, the voice sounded the same. But you do hear him speak really quickly for a minute and then sort of slow down his pace again. And then he might, you know, and so people. The speed of the way they talk sort of fluctuates up and down. But when you listen to something like 11 labs generated voice, you. You just hear it sort of all at the same pace. And I think that's the giveaway. But I also think that's where, like, something like Sesame is really impressive is you start to notice it has some of those variations in it, you know?

Well, I think the interesting thing too, though, was, like. Cause you were talking about the summaries of articles and things like that. And what I was thinking about was, like, when I listen to audiobooks, I've always hated when it's somebody else's voice. Right. If it's the author's voice, that's cool. Or. And it's way better if it's author's voice. But like, a lot of people don't have the time to do that. So, like now, in theory, just like, you know, Patrick Carlson just did for his annual letter, all CEOs and all authors could be doing similar things where they're still using their voice to make the summary. Like, if it's a blog post, it's, you know, it's. It's. It's my voice for my. My blog post for my newsletter. Right?

Yeah.

Versus somebody else's voice, for sure, I think.

Well, here's. Here's the voice that 11 Labs did for me again. Whenever I listen to it, I have a hard time hearing my own voice. But this is what it sounds like. Everyone should subscribe to the Next Wave podcast or Nathan will send robots to your.

Yeah, so like I said, so those guys definitely have connections at 11 labs. And so if I had to guess, this is some kind of. They've collaborated, and it's probably on a next version of the model or something that's coming out that's not public, if I had to guess, could be wrong.

Gotcha.

But there's a lot of stuff.

Trained this in a while ago. Yeah, I trained this in a while ago. Like, I started using 11 labs maybe two years ago, maybe even longer. Two and a half years ago. I was. I was a very, very, like, early, early user of 11 labs. And I trained my voice. I think I've trained it in again since then, but I Think my most recent training run was probably still like a year ago, so there might even be a better model and I just need to go train it again and it'll be better this time around.

You know, I think it's been the same model for like probably six to nine months. I'm sure they've like made tweaks to it, but I would have to assume that there's like a new, better version coming out. That's what I'm hoping. You know, I told you I've been working on my game and I. I tried using it for like voiceovers and stuff like that and it got, you know, decent, but I'm like, it's. This is nowhere near. This is not good enough to release.

Yeah.

With this kind of voice, I mean.

Hey, maybe that Octave might be one to play with because that one I believe has APIs that you can use and you can actually give it like the, this sort of description of whether you want it to sound angry or happy or whatever. So I mean, that might actually be a really cool one to test with, like game development and stuff.

Yeah, yeah. They even have sound effects too. A lot of people don't realize that, but you can actually generate sound effects as well. Oh yeah.

11 labs. Yeah, yeah, yeah.

Cool.

Well, I, I feel like we covered a lot of ground in this episode.

I've been using Whisper Flow, but I.

Think, I feel like that one's hard to go really deep on because, you know, you've got Whisper, which is already really good. That's OpenAI's open source model where you can give it a bunch of audio or video and it will transcribe it. You've got Assembly AI has a really, really good one that's supposedly the most accurate up until the 11 labs one. And now you have the 11 labs one, which is supposedly the most accurate. Now passing assembly AI. You know, AWS has their own version, I believe Google has their own version. But like the big differences between all of the speech to text models and is just like the percentage of accuracy.

Right.

It's like this one is 90% accurate now this one's 92% accurate now this one's 98% accurate. Right. And it's just like, it's hard to really demonstrate the variations between them. The biggest difference is they're just getting more accurate.

Right, Right. Yeah. So I've been using one called Whisper Flow and actually, I don't know, it's not an OpenAI product, but maybe it is using their API underneath the hood. I'm not actually sure.

Believe Whisper Flow is using the API. I can double check. Let me ask Perplexity.

Yeah, that would make sense. Why it's called Whisper Flow, but Flow.

Does use OpenAI's Whisper underneath. So, yeah, basically a wrapper on their API.

Yeah. And it's nice. I mean, like, what you do is, like, you set up, like, one hotkey, and so I have, like, one hotkey. It's like, on my Mac. It's a function on PC. For some reason, I think I had to have, like, two buttons. I had to, like, you know, doing, like, you know, I think it was Control and Windows key or something. But I just press that and then I just talk, and then everything I say, it turns it into text, you know, and you can do this for tweets. You can do this for prompting LLMs. You know, it's. It's. It's. People have been talking about, like, vibe coding. That's really. That's part of what they're talking about when they say vibe coding is not just using cursor to create things, but the fact that you're. You're just talking to it, talking to it, and the words turn into code.

Well, if you go back and listen to the episode or watch the episode that we did with Riley Brown where we actually coded up an app with him. Yeah, I don't know. I think it was Whisper Flow that he was using. But he was doing that, right? He was. Whenever we were talking about, like, let's add this feature, let's add that feature, let's add this. He was just pressing a button on his keyboard and just speaking out what he wanted it to code for him and it would go and do it. I'm fairly certain it was Whisper Flow. If it wasn't, it was something very, very similar.

I'm pretty sure it was. That's where I learned about it from. I was like, yeah, I remember. I remembered that from the podcast. And when I had the injury, I was like, oh, I need to actually use this now.

Like, yeah, yeah, yeah, yeah, that's good. I mean, I used to use Dragon Naturally speaking. I don't know if you remember that one. And it was never very accurate, but that's what it was designed for.

And there was always delay.

You would talk and then, like, it would think. And then, like, you know, 10 seconds later, you'd see your text, like, populate on the screen.

Yeah, Whisper Flow. Like, if you talk fast and, like, you know, I think I do talk fast naturally. If I don't, like, slow Myself down. It told me I was like, top 1% of users in terms of speed of talking. I was like, oh. I was like, oh, crap. But.

But it still picks it up mostly.

It's mostly. And like, I do, I, I do find myself trying to, you know, slow myself down and not talk as fast. And it's funny. There's times where, you know, it'll. It'll get a word wrong and it's definitely like, not a word I said incorrectly, which of course I do sometimes, you know, but when that's not the case, it'll still sometimes get the word wrong. And it's kind of funny when you paste that into Grok and in Grok will be like, oh, I assume you meant so and so, you know, but that's kind of funny. Funny what you said or that was a funny joke.

It's like, yeah, that's just Grok being sassy with you.

Yeah, yeah, Grok's like, I like that about it. It's kind of funny. Well, cool.

Yeah, I mean, I, I think, you know, there's actually some, some good use cases that, that people listening to this episode could go use this for. There. There's some great tools out there for turning articles and blog posts into audio. There's some great tools out there like whisper flow for turning, you know, just you speaking into text or prompting or vibe coding. You've got all of these various speech to text models where if you want to get your videos transcribed or your podcast transcribed, those things are getting better and better. Now, I don't know if you saw this, but Google Drive is just going to start transcribing videos that you toss into Google Drive the same way YouTube does. So, like, if you start throwing videos into Google Drive, they're going to make your all of your video content in Google Drive searchable because it's going to automatically transcribe all videos that you toss in there just to make it easier to search out and find the exact videos you're looking for. So, like, some of those features are just going to start getting built into some of the tools that you're already using, which I think is pretty cool as well.

Yeah. Yeah, maybe this is kind of like off of the topic of AI voice, but the rumors now are that the next GPT, whether it's GPT 5 or it's just an improvement on 4.5, is going to have the ability to like view videos and understand what's in the videos as well as. Sam Altman also teased that there's a dramatic Upgrade to Dall E coming. He said you're going to be thrilled with joy or something. He said something like that about what's coming soon. So I think you're going to see all of these are going to get way better in terms of understanding images, video.

And yeah, Gemini's already really good at it. That app that I was showing you and Matt Berman on our previous episode, it was using Gemini behind the scenes because Gemini can actually watch videos. It only sees the video at one frame per second. So if you plug in like a 60 frame per second video, it's only sort of capturing it one every second, but it will pick up on what's going on in the video. And it basically watches videos on inside of my app and describes what's going on to make them searchable. So I would be really shocked if OpenAI doesn't roll that into one of their next models.

Yeah, yeah. Google's cooking people. Don't give them enough credit, but like, they are doing a lot of amazing work. And you know, I think you commented on it that recently, like, Elon Musk responded one of my tweets and like, he kind of like framed it as like the actual battle is xai vs google, not. He like left out OpenAI when he responded to me. And I thought that was hilarious, but could end up being right eventually, who knows? Because I mean, like, Google just keeps coming out with new stuff. I mean, this is like, probably really off topic, but they're like, I saw stuff like yesterday of like, there's new stuff in science that Google's rolling out for, like using AI and science, and apparently scientists are blown away by it. And so like, yeah, Google's doing good stuff.

Yeah, yeah, Google's. They, they've got some cool stuff coming out. And who knows, maybe that's a future episode. Maybe we'll do a whole episode about all the crazy stuff that Google's been rolling out lately.

Yeah. Ending with like totally ramblings about different.

Yeah, no, it's. It's cool. I think. I think people, you know, I think people tune into podcasts like this to, to sort of hear where our heads are at and what sort of things are on our mind right now in the world of AI And. Yeah. And I mean, the up. The pace of updates is just. It's crazy. Every day there's something that's. That like got a huge upgrade. And so, you know, that's. That's why podcasts like this exists. That's why YouTube channels, like my other channel exist. Right. It's there's so much happening that anybody that wants to stay in the loop, well, that's what we're making this show for. So if you're not subscribed already, make sure you subscribe on YouTube. If you prefer listening to audio versions, we're available wherever you listen to podcasts, Spotify, itunes, all the rest will help you keep your head on straight. We might go crazy around AI sometimes, but ideally you don't have to because you tune into podcasts like this.

But remember, subscribing helps save the world as well.

Subscribing also helps save the world, and it's going to keep Nathan from sending scary robots to your house. So make sure you subscribe.

Thank you so much everybody, for tuning in. Hopefully, hopefully we'll see you in the next one.