AI Powered Voice Acting? - The FIRST LLM Designed for TTS

Hey everybody, welcome back to the Matvid Pro AI YouTube channel. Today we actually get to try a new type of AI model. I don't get to do this much on this channel, so I'm pretty excited. I'll admit I've already played around with it and it's definitely very interesting. I've got a lot to say about it. Today we're checking out Octave by Hume AI. It's the first large language model, but built for text to speech tasks. I've never heard of such a thing other than I suppose, GPT4 omni, but that doesn't really fit the bill. It's not built for text to speech, it's built to converse with humans. Now, traditional text to speech AI has come a long way in the past few years. Really high fidelity audio quality, we've got voice cloning and a lot of times traditional text to speech even sounds human enough to trick your brain. Now Hume here claims that unlike traditional text speech models that just read words aloud, Octave actually has the ability to understand the meaning behind the words it speaks and can change its delivery to generate more emotional human like speech. In fact, you can actually prompt this thing to do a certain type of delivery, which is very interesting. Basically acting instructions to control emotion and Delivery, sarcasm, whispering, etc. You can also design any voice you want with this model as well. Basically prompt to voice, which is something we've also seen with traditional text to speech models. Typically I'd show off all these little demo videos, but I really want to show you my hands on experience. But I do want to pull up some benchmarks here. According to Hume, this is a blind study where Octave actually outperformed ElevenLabs. Voice design, interestingly enough, naturalness is very, very close on this benchmark. Hume only edging out Elevenlabs by just a little bit. But in terms of description match, Hume takes the cake for sure. And in audio quality, it's a no contest here. Hume crushes elevenlabs, but we'll have to see for ourselves. Testing this product out. Don't want to go by just the Ben. We gotta test the vibes before we dive into the platform and give this thing a good test run. Full disclosure, Hume did sponsor me today to check this product out, but in no way does that mean that this video is an ad or endorsement. All opinions you'll see in this video are 100% my own. I test the product out beforehand and I make sure that I'm not just making a video for nothing. I want there to be real value for you guys, at the end of the day, I'll show you the good, the bad, and the ugly, no matter what. So while Hume did sponsor today's video, doesn't mean I'm necessarily gonna go easy on them. But like I said, I. I did do some testing of my own, so there are definitely good things to say as well. All right, let's dive in. First of all, it does seem like they have a small free demo on their website. The first large language model built for text to speech prompt to create AI voices, change emotions, and more. We can prompt the voice and the script here. This is going to be what's read out and this is how it's going to be read out. The speaker has a naturally charismatic, slightly nasal and very expressive voice. The like a beauty blogger from the Midwest who is sharing her latest favorite products and tips. One thing I'm going to note about this description here is it's very natural. It sounds like natural language. This is the way that you would talk to an LLM and it makes sense that we would prompt it as such because it is an LLM but designed for text to speech. Let's see what this sounds like.

Okay, so like, I've been trying out this new setting powder and whoa, it is so seriously a game changer for keeping my makeup in place all day. Have you guys tried it yet?

I gotta say, it's very solid sounding, especially for a first of its kind model. There's no doubt in my mind that it is state of the art. It flows very naturally. Sounds like a real beauty blogger's voice with headphones on though. I can definitely tell, especially in the higher pitched notes and tones. There's a little bit of that like AI reverb type sound going on. You might not be able to hear it through your regular speakers. If you're wearing headphones like me, you might be able to pick up on it, but on those higher pitched tones, little bit of fuzziness going on. Now let's completely flip the script. The speaker has a gravelly, deep and untrustworthy voice. They speak in booms that shake the ground and make you feel uneasy. They're likely a monster in a cave. I mean, if we're gonna test this new type of model out, we're gonna test it out. So let's see the difference again, we'll play V here for an example of the original voice and then we'll try this new one.

Okay, so like, I've been trying out this new setting powder and whoa, it is seriously a game changer. For keeping my makeup in place all day. Have you guys tried it yet?

Okay, so like, I've been trying out this new setting powder and wow, it is seriously a game changer for keeping my makeup in place all day.

Have you guys tried it yet? Gonna be real. I don't think that quite fits the bill for the voice that we described. However, since this is an LLM built for text to speech, it might be the script doesn't necessarily match the voice all that well. Let's see if these other voices sound any better. Okay, so like I've been trying out the. Okay, so like I've been trying out this. Yeah, that just sounds like a normal dude changing the script. Who dares enter the cave of Galaxan the third, you will be punished with severity. Oh, that hurt my voice.

Who dares to enter the cave of Galaxy 3rd, you will be punished with severity.

Now that was way closer to what we were expecting. There is definitely a tie to the way that the voice speaks its words and the script itself. And I almost guarantee it's a factor of it being a large language model developed for text to speech that might lend this model specifically for the use case of reading out a story or something like that with different characters. This might not be the text to speech model for reading a webpage out or transcribing your book for school so you can listen to it instead of read it. Interesting how different types of models can lend themselves for different use cases. Let's try these other voices out.

Who dares to enter the cave? Galaxan the Third.

Yeah, that one's not as good.

Who dares to enter the cave of Galax on the third? You will be punished with severity.

Yeah, Voice 3 here does not necessarily lend itself too great to the description we wanted for, but I think it lends itself better to the script because to be honest, these words here, the way it's written doesn't sound like a monster in a cave. It sounds more like an evil, you know, more sophisticated warlord or something like that. I don't know, maybe I'm taking crazy pills, but my first thought here is that because it's a large language model developed for text to speech, it really reads your script and the script can really steer the way the text to speech is going to sound at the end of the day. Alright, so I've moved on now into the actual app. I've signed in. This is the text to speech playground. This actually has another capability which we talked about in the opening, the acting instructions. So we can Instruct the large language model to speak and act out the voices in a certain way. Also, want to make it pretty clear here to you guys, you're still selecting a certain particular voice as a base. So, for example, I've got one that I made earlier. Green little goblin. This is what he sounds like.

Oh, dear Traveler. Is that some gold I smell in your pocket? Us goblins love gold. Would be a shame if it went missing.

He.

I kind of like the way he reads that out. Missing at the end of it.

Hey.

There's three parameters that can be adjusted to tune or change the way that the spoken voice is gonna sound at the end of the day. First, we've got the bass voice, which we prompt. We've got the acting instructions, and then finally, the speech itself or the text that's going to be read out. So let's come up with, you know, a situation this goblin's in. In fact, I might as well put the brand new GPT 4.5 to the test. Write me a monologue for a goblin who is frustrated and upset, trying to integrate into normal human life. He's on the verge of tears. Now, as it's writing the script, I do notice that in parentheses, voice trembles. Right. So that would be acting instructions in line for the actor that's reading this out. I wonder how the model is going to handle something like this. Just being in the middle of the text, it's probably going to, in my mind, just read out, voice trembles. And if that's the case, we'll remove it, put it in the acting section. We'll copy this, bring it back in here. Acting instructions. Frustrated emotions. Boiling trembles. Sad and exhausted. So, again, we're gonna keep in mind and see if it reads out these things in parentheses that are obviously not supposed to be read out. Oh, it looks like in my original testing, I exhausted the credit balance. So, yeah, they do have a free limited amount of credits when you first sign up. By the way, that's something important I should mention. Yeah, it looks like you get 10,000 free credits, which isn't too bad to test it out, honestly, I'll spend three bucks here to show off the rest of the plat platform. All right, Goblin man, speak to me.

It's always the same, ain't it? Voice trembles. Be normal, they say. Blend in, they say. But how's the goblin supposed to blend, huh? Look at these ears, these teeth. I smile at a kid, they scream, I wave at a neighbor, they grab pitchforks, just stroll right in, shake hands, smile Chit chat about the weather. Me, I gotta hide my claws, bite my tongue, talk proper. Good morning. How's your day? But it's never good enough, is it? I'm always too green, too short, too goblin. Yai stops pacing, growing more emotional. Last week, tried to shop for groceries.

Okay, so clearly these little parentheses, like in text instructions are not the way to use this thing properly. So I'm going to go ahead and remove them. Stuff like this obviously is going to be in the acting instructions, which I did. I mean, I put all that in the acting instructions. So we'll remove all this and see how things flow. Other than that, I'm really impressed with its ability to change emotion mid sentence. I gotta be honest. I mean, traditional text to speech, even without the LLM integration, is not bad with emotions sometimes, but changing emotions midway through and having it sound human is definitely a difficult task. And this seems to be nailing that, more or less. All right, let's give it another shot with cleaner text.

It's always the same, ain't it? Be normal, they say. Blend in, they say, but how's a goblin supposed to blend, eh? Look at these ears, these teeth. I smile at a kid, they scream, I wave at a neighbor, they grab pitchforks. Humans got it easy. Just stroll right in, shake hands, smile, chit chat about the weather. Me, I gotta hide my claws, bite my tongue, talk proper. Good morning. How's your day? But it's never good enough, is it? I'm always too green, too short, too goblin. Y last week, tried to shop for groceries. Simple, right? But the clerk, he called security, said I was lurking suspiciously. Lurking? I was picking apples. Do you know how hard it is to pick apples when you're barely taller than a pumpkin? I just want to belong, to fit in somewhere. Maybe go bowling without everyone staring. Maybe have someone say, hey, goblin, come sit with us, instead of whispering behind my back. Is that so much to ask? I thought leaving the caves behind meant finding a new life, finding friends, finding home. But maybe I'm just fooling myself. Maybe goblins ain't meant for human lives, no matter how hard we try.

Wow. Okay, I have a lot to say about this. First off, like I mentioned earlier, definitely switches through those emotions quite well, but I think that's coming at some sort of a price here. It seems like as it changes sentences, the voice isn't super consistent in terms of its pitch. It can kind of sound like a different goblin every other paragraph or so. So I'm wondering if, again, we change the structure here and consolidate Everything. If it will fix that issue. Also for the Goblin Y part, I'm gonna change it to Goblin knee. Trying to fix that with most text to speech models. Actually, not just LLM based ones. Sometimes you'll have to change things like that. Let's refine the acting instructions. I'm gonna put clear and easy to understand voice acting from an animated film. I really don't know if that kind of a prompt is gonna be good for a LLM designed to do text to speech. But hey, we're experimenting. It's a new type of model. Let's see.

It's always the same, ain't it? Be normal, they say. Blend in, they say. But how's a goblin supposed to blend, huh? Look at these ears, these teeth. I smile at a kid, they scream, I wave at a neighbor, they grab pitchforks. Humans got it easy. Just stroll right in, shake hands, smile, chit, chat about the weather. Me, I gotta hide my claws, bite my tongue, talk proper. Good morning. How's your day? But it's never good enough, is it? I'm always too green, too short, too goblink neee last week, tried to shop for groceries. Simple, right? But the clerk, he called security, said I was lurking suspiciously. Working. I was picking apples. Do you know how hard it is to pick apples when you're barely taller than a pumpkin? I just want to belong. To fit in somewhere. Maybe go bowling without everyone staring. Maybe have someone say, hey goblin, come sit with us. Instead of whispering behind my back. Is that so much to ask? I thought leaving the caves behind meant finding a new life, finding friends, finding home. But maybe I'm just fooling myself. Maybe goblins ain't meant for human lives, no matter how hard we try.

Okay, definitely more solid still though, I'm seeing a little bit of a change up, a little bit of a waving, at least in the tone of the voice as it reads through the script. But to be fair, maybe it's just the voice that I made. Maybe I didn't prompt it that well. So we could try maybe one of the preset voices they have. Other than that though, I am happy with this result. I think it does narrate it naturally. It definitely sounds like a monologue. I like the way that it expresses the emotions as it reads through the script. Since Hume also did comparisons with 11 labs, I wanted to do my own real quick to give you a taste of traditional text to speech. I think it's gonna be useful.

It's always the same, ain't it? Be normal, they say. Blend in, they Say, but how's a goblin supposed to blend, huh? Look at these ears, these teeth. I smile at a kid, they scream, I wave at a neighbor, they grab pitchforks. Humans got it easy. Just stroll right in, shake hands, smile, chit, chat about the weather. Me, I gotta hide my claws, bite my tongue, talk proper. Good morning. How's your day? But it's never good enough, is it?

Yeah.

Okay, I'm definitely getting a clearer picture. Traditional text speech has definitely come a long way in terms of emotion, but it doesn't have the capability to actually understand the words you're writing where the LLM based text to speech does. And that definitely affects delivery. It knows how the script is going to flow and can design the audio accordingly. It's not perfect. This new large language model text to speech definitely struggles keeping the voices consistent. That's where traditional text to speech seems to still hold the crown. But man, that realism, that emotion, that actual acting definitely shines and comes through the LLM based text to speech. Without a doubt. Alright, now I want to try this wise wizard character. This is a character that is a preset voice that Hume AI made in.

Ages long past when the world was young.

We can assume here that this has a good prompt and this voice has been tested extensively, so we can remove it as a factor in our testing. I. I had Chat GPT generate a nice monologue rant from an old wizard complaining about new wizards these days, which is gonna be fun. Let's actually go ahead and ask ChatGPT for acting instructions. Give clear voice acting instructions. Alright, here's our acting instructions. Clear and easy to understand voice acting from an animated film. Use a gruff, exaggeratedly crotchety voice with frequent dramatic pauses emphasizing sarcasm and playful irritation. Alright, I'm excited for this.

Wizards these days, Pfft. Back in my day, spell casting meant something. You spent 20 years memorizing incantations, risking your eyebrows every other Tuesday. And for what? Glory? Respect. Now look at EM Wand waving like they're conducting an orchestra. Swish and flick. What is this? Wizard aerobics class? And their robes sparkles and sequins and glow in the dark thread. Back in the good old days, robes came in two colors, Dark grey and darker grey. And they were itchy. Oh, so wonderfully itchy. Built character. Nowadays it's all comfort spells and silky lining softies. Don't get me started on their newfangled magic mirrors. Taking portraits. Okay, they called.

I want it to be more ranty. I'm gonna stop it there. Ranting. Very annoyed. Another roll of the dice.

Bah. Wizards these days. Back in my day, spellcasting meant something. You spent 20 years memorizing incantations, risking your eyebrows every other Tuesday, and for what? Glory. Respect. Now look at him. Wand waving like they're conducting an orchestra. Swish and flick. Ah. What is this? Wizard aerobics class. And their robes. Sparkles and sequins and glow in the dark thread. Back in the good old days, robes came in two colors. Dark grey and darker gray. And they were itchy. Oh, so wonderfully itchy. Built character. Nowadays it's all comfort spells and silky linings. Softies. Don't get me started on their newfangled magic mirrors. Taking portraits. They call them self spells. Self spells. Ha. Who wants a thousand Moving.

It was starting off so good, and then once it got to this point, it was like it died out. We want this to be high energy, man. Maybe it's the frequent dramatic pauses that are doing it. It really seems to be sensitive to prompting here.

Ah, wizards these days. Back in my day, spellcasting meant something. You spent 20 years memorizing incantations, risking your eyebrows every other Tuesday, and for what? Glory, Respect. Now look at him. Wand waving like they're conducting an orchestra. Swish and flick. What is this? Wizard aerobics class. And their robes. Sparkles and sequins and glow in the dark thread. Back in the good old days, robes came in two colors, Dark gray and darker gray. And they were itchy.

Why they switch up.

Oh, so wonderfully itchy. Build character. Nowadays it's all comfort spells and silky lining. Softies. Don't get me started on their newfangled magic mirrors.

Taking portraits have issues towards the end. Like, it starts out nice and high energy, but it's. It can't seem to keep it up as it goes through. So if you want more control over how things are acted out and how things are spoken, you can make a project and not just use the playground. So we're gonna take this same wizard dialogue and see if we can kind of perfect it. Wise wizard. Essentially, if I hit enter between these sentences, it turns into a new section. So for this first section, acting instructions. Angry, high energy, upset at the world. So now can we only just play that section?

Ah, wizards these days. Pift. Back in my day, spellcasting meant something.

Reminiscing, frustration for the second line.

You spent 20 years memorizing incantations, risking your eyebrows every other Tuesday, and for what? Glory, Respect.

That's pretty good. Okay. Projects are definitely the way to go. So now we're no longer reminiscing. We're insulting, scolding, frustration Again, more fond reminiscing here. Talking about the itchy robes. Let's export this and see if this sounds better.

Ah, wizards these days. Piffed. Back in my day, spellcasting meant something. You spent 20 years memorizing incantations, risking your eyebrows every other Tuesday, and for what? Glory? Respect. Now look at him. Wand waving like they're conducting an orchestra. Swish and flick. What is this? Wizard Arabic's class and their robes. Sparkles and sequins and glow in the dark thread. Back in the good old days, robes came in two, dark gray and darker gray. And they were itchy. Oh, so wonderfully itchy.

Ooh, I like the reading out at the end. It's definitely got that emotion down and the nuance in the way of reading the conversation as it flows. The big drawback here for me is just that it sounds like almost like a slightly different wizard each time. So the consistency there is the one thing that I think is holding this back a little bit. Other than that, really impressive. Final thoughts here. It's a new type of text to speech model. It's one that can actually understand the text that it's reading out like a real human actor. And I think there's real benefit and real value there and I think they've already captured some of it for sure. There's a clear difference between this and traditional text to speech. No doubt about that. If you can get the voices to be consistent, this is definitely the way to go for more emotional voice acting style stuff. But. But in my testing it's really difficult for the voice to be as consistent as you want it to be. It does kind of sound like slightly different characters sometimes as it switches through. Now this is something that I think could definitely be improved in the future. I would love to see an Update or a v2 version of this large language model. Text to speech. I really think it could be the future of text to speech. On the issue of consistency, I did ask Hume about it and they said that they can expect this to be resolved in the upcoming weeks and actually not require a V2 model upgrade. They are aware of this issue and they plan on shipping fast, which is good. They want to improve quality over time without, you know, waiting months to ship a new model. They want to try to keep it going week by week and I respect that. It is impressive for a V1, a first iteration of a brand new type of text to speech model. Impressive that Hume was able to make something Pretty state of the art as I said. Typically does not happen with brand new types of models. Now whether or not I'd recommend this really depends on your use case. If you need emotion, if you need something that sounds very natural, if you need something especially that you can instruct as well, this is probably going to be the best way to go. But you might have to do some post work there in terms of adjusting the pitch, maybe to make the voice align good as it reads out the whole thing. Don't think it's quite there in that regard. If you just want to transcribe like your textbook for school so you can listen to it and not read it, I think this would work for that as well. But traditional Texas speech is probably gonna have a more consistent voice for that. And you don't care if it sounds like AI because it's just for learning for school. Now I know we didn't talk about it too much in this video, but I do want to go over pricing. They actually have some really competitive pricing on this website. It's cheap text to speech and they have plans starting at the most entry level physically possible, like three bucks a month all the way up to enterprise level. Extremely competitive, cheaper than Elevenlabs, cheaper than Play ht. So not only is it state of the and highly emotional, but it's priced very well. So yeah, it's competitive, it's impressive. I really like the fact that they took a chance on a large language model based text to speech variant and I think that at the end of the day this is going to pay off, especially long term if they stick with this architecture. No doubt in my mind that a large language model that can actually understand how to attenuate and read out a script is going to be much more beneficial than an AI that's just taking text in and translating it to an audio file. Let me know Guys, what do you think about this? Do you agree with my sentiment on it? Are you going to test it out for yourself? It does have a limited free amount of generations here. Love to see what you guys come up with and what you think about it. Thanks so much for watching folks. I'll see you in the next video and goodbye.