BARK: BEST FREE Text-to-Audio Model 🤖🎵 | High-Quality Speech & Emotions in Multiple Languages 🎙️😂

If you're looking for a free multilingual hyper realistic text to speech synthesizer, I have a treat for you. There is a new text to audio model called BARK by a company called Sono. This is a transformer based audio generation model. As you may know, ChatGPT is also a transformer based text generation model. BARK is a lot more than just a simple speech synthesizer that we have seen so far. It can generate hyper realistic multilingual speech as well as other audio effects including music, background noise and simple sound effects. The model can also produce nonverbal communications such as laughing or crying. In this video I'm going to show you how to use BARK for your own text to speech synthesis. And they provide a very simple prompt template with the help of which you can control the output. So let me show you a few examples of what the output will look like or actually sound like. So here are a few examples. Look at this text. The model is called Bark like Clifford the Big Red Dog. Pay close attention to these double hyphens and you will see it really impacts the output of the model. So here is how it is going to sound like.

The model is called Bark like Clifford the Big Red Dog or BARK as in tree bark.

Well, I don't know about you, but to me this looks like a real human voice, not a computer generated voice, which is simply amazing. Okay, let's look at a couple of other examples. So for example, this is going to sound like a TV commercial.

Tired of Doom's fooling Twitter? Worried about the inevitable rise of our AI overlords? Try Soma, the miracle drug from Brave New World, brought to you by beloved Megacourt Jackson and Jackson.

This is like absolutely amazing. It does sound like a TV commercial. And I think there's no really no hint of AI generated sound the way like we are used to with bark. You can also control some non speech sounds. For example, you can control the tone of the voice as well as add laughter to it. So here are a couple of examples. In this case, if you see this sad in the brackets, it's a special token that will control the tone of the voice. And then these are excise or laughs. These are some non speech sounds that it is going to add. So let's look at an example.

My friend's bakery burned down last night. Now his business is toast.

They're just using this special token you can actually listen to the sound and the tone of the sound or the voice is sad in the beginning. Let's look at this one. I got a face for Radio and like what do they say? Voice for print. I don't know about you, but this is actually pretty amazing. And later in the video I'm going to show you how you can actually run this on your own text. Now you probably noticed that if I play the sounds male voice or even for the female voice, there are different voices and seems to be different characters. Right? So you can actually control which voice to use. I'm going to show that later in the video. As I said in the start, it's multilingual. So it has support for Chinese, French, German, Hindi, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish and Turkish. And I think they are adding some more languages as well. So let's play a couple of these. I don't really understand let's say French. So I don't know how accurate this is going to be, but let's play it.

Bonjour. Bonjour.

Duit manger classe. Well, this did sound French to me, but I'm not like sure how accurate this is. Although the author says that this works pretty great for English at the moment, but the other languages will improve over time. Now the great thing about this tool is it's not just multilingual. You can actually change the context. So that means that text, you can actually mix up two languages. They call it code switching. So let's say in this case if you see so in the first part of the sentence is English and then, I don't know, this is Spanish I guess. Right. And in the second case it seems to be German and then English. So I'm going to play both of them and just pay very close attention to the accent that is being selected.

It was as if he had never existed. My grandmother used to say, il amor es como in agua no puer de sustena fluentus marnos.

Okay, now let's listen to the other one. Hey Martin, es is wikilken. Das dummet danner linken han voga meerwenkanz. It's super cool, but maybe you could type faster if you use the right hand tool. The way they have designed the system or trained the system is that depending on the first language in the conversation, it will select the accent. So let's say if it's in English, the first part is English. So the action the accent selected is going to be English. And then even for the Spanish they're going to use that accent. Right. In the second case, if the first part of the conversation is German, then the accent is going to be German. So the English Part is going to be narrated in German accent. Okay. So before running it on your own text, I want to show you a couple of other cool features that I haven't seen in any of the other tools. They provide a very simple and easy to use Python interface. And this is exactly what we're going to be using in Google Colab, or if you want to run it on your own machine. So I'm going to talk about the installation in a little bit, but essentially it's a Python package where you need to import a couple of things. So first is the sampling rate. So all the audio samples are generated at a sampling rate of 24 kilohertz. Then you have the generate audio function, that's the function that is used for generating the audio. And then there is a preload model function that will load the model. Right? So what you need to do is you need to provide a simple prompt that is going to be basically the text that you want to convert to speech. So in this case there is a prompt. Hello, my name is Swano and the rest, right? Then you simply pass that prompt to this generate audio function. You get an audio array and then in order to generate the audio, you pass that audio array onto this audio function from IPython display and along with the sampling that you choose. Okay, so let's look at this quick example and then I'm going to show you some amazing features, features that this library has.

Hello, my name is Suno and. And I like pizza, but I also have other interests, such as playing Tic Tac Toe.

If you give the same prompt and generate the audio, it might sound a bit different each time. So because it's a random initialization, so it might sound a little bit different, but we can actually control that by selecting the speaker. I'm going to talk about that in a little. Here's the part I really wanted to show you. So you can actually use this to generate music, right? In that case, you're going to be using this special token which is music notes. So if you look at it, if you put this music note in front and at the end of your text prompt or a part of it, this will generate music based on whatever prompt or text you provide.

In the jungle, the mighty jungle the lion Barts tonight.

Now, if you notice, it actually added some instruments in the background, which is pretty cool. In one of my previous videos I showed you how you can clone your voice or somebody else's voice. Now, Bark also has the ability to do that, but in this case they are simply Limiting it to some predefined voices because of the privacy concerns and stuff like that. However, there is a way around it. And I'm going to show you in another video how you can clone your voice or somebody else's voice using bark with bark's quality. But that's for another video, so keep an eye out for that video. But in this case, you can actually select one of I think the 9, 10 speakers that they have. Right? And that's why like, if you recall when I was playing different voices, the voices were different. So that depends on the voice that you select here. So with the same audio generate underscore audio function, you will pass on this extra parameter called history underscore prompt and that will. You can select your speaker and that will generate a different audio. So let's play this.

I have a silky smooth voice. And today I will tell you about the exercise regimen of the common sloth.

Okay, I'm going to show you in Google Collab how we can play around with this part. Here's another awesome feature. You can actually select a part of the prompt that you provide and select the narrator as well. So in this case, for example, the first part is narrated by a woman voice and then the second part is based on a men's voice. So let's listen to this.

I would like an oat milk latte, please.

Wow, that's expensive.

It sounds pretty awesome, I must say. Okay, so in terms of installation, installation is very simple. All you have to do is just create a new virtual environment on your local machine if the hardware supports this. And then you can do pip install the GitHub link that will install the package. Right? That's as easy as gets, right? Or you can simply do it git clone and then PIP install inside that directory. All right. In terms of hardware required requirements, so they are saying that it works both on a CPU and a GPU bar requires running greater than 100 million parameter transform models. So on a modern GPU and Pytorch nightly bar can generate audio in roughly real time. I yet to have tested on my local machine because I have a cpu so it might be much slower because it's on CPU and funds time might be 10 to 100 times slower. Right. But the great thing is there's a Google Collab available, so we're going to test that out. Right. And the second thing is they also have a playground that is going to be available soon, so you can actually sign up for early access. I already signed up, have yet to hear back from them. But I think this is going to be available soon now in terms of licensing. So Bark right now is for non commercial use only and it's not because of Sono but because of the backend that they're using. So they're using Incodec from Meta or Facebook. That is why that has like a non commercial license. Because of that you can't use it. But it seems like the Sono will be releasing a number of models themselves which will be available for commercial use. So that's a great news, I think. And based on the quality that we're getting from park, it does definitely is going to be hyper realistic. Even though the Sono models there are a number of things that you can actually use for non speech sounds. So for example Laughter that's used simply put in the brackets, right? Anything in the brackets that is that are used as special tokens, right? And it's going to have some non speech sounds included, right? So for example, if you want to have a throat clearing sound, so you will just put it this. If you notice we already talked about the hyphen that's for hesitation, right. And you can also include the music notes for sound lyrics. And if you want to emphasize on a word you can simply capitalize that. So capital words are going to be emphasized more. We're going to look at all these things in a Google Colab. You can access Google Colab from here. They also have a hugging space so we can use that as well. So I'm going to open both of them. So for the hugging face space you simply provide the text, select the type of speaker that you want. Right? So there are different, I think 10 speakers for each language that they have. So you can. Let's just select this one, right? And then just click submit. Okay, so I'm gonna. Let's just select two, click submit and it's going to start running. So right now I am the first one in the queue and it's going to take a little bit of time to actually process my request. While we were waiting for that, let's look at the Google Colab as usual. Just make sure that you go to file and save a copy in the drive. Prefer to make a copy of it, right. And then the first block is installation. So let's run this. This will take a little bit of time because it has to download quite large files, right? So that will download the files and install it. And then here is how you can actually use it. Right? So simple import those three functions I talked about. This will Also take some time because I think it's. It will download around 4 gigabyte file 1 and then there is around 4. Yeah, like two 4GB files. So this is going to take some time as well. Okay, so our hugging face output is ready. Let's play this. Hello, my name is Suno and I like pizza. But I also have other interests such as playing Ficto and I had noticed that for these demos you actually need to click the play button twice. I don't know what's wrong with it, but you need to do that. So this will install the library and you can import the models. Okay, now here are the examples that we already saw, but I want to do do something different. So here's my own prompt. First I want to show you how long it's going to take for it to actually run and process it if you're using Google Colab. So by default I'm using the speaker one and I'm not putting any non speech sounds in here. We're going to play around with those in a second. So let's see how long it's going to take for this to run. So I simply click run and you see that it took a few seconds and now I think it's generating the sound. So it's still ongoing. It's taking around 1.1. One point? Yeah, around 1.1 iterations per second. So it's, it's reasonably fast, not terribly slow given it's a free Google collab. Okay, let's see how it sounds like.

Hey, subscribe if you already haven't, press the notification button to never miss another video.

Okay, let's see, let's randomly choose another sound or speaker. Okay, let's run this again. All right, so here's the second. Got the foot.

Hey, subscribe if you already haven't press the notification button to never miss another video.

It seems to be of woman voice. Right? So we can actually control it here. If I type in man, right? So I think this is supposed to give me a man twice. Right? So we can run this. All right, so here, hey, subscribe if you already have it. Okay, next let's look at this list. So I'm going to do a couple of things. Let's say let's put a throat clearing in there. So let's say we're going to put it here. Okay, I'm going to change the speaker to let's say three. All right. And let's run this again.

Hey, subscribe if you already haven't and press the notification button to never miss another video.

Okay. For some reason it completely ignored this token and it went with a woman voice. This could happen I guess. Okay, let's look at emphasis on a specific word. So in this case the subscribe is all in caps. So let's see how this sounds like.

Hey subscribe if you already haven't.

Okay, I guess I could say it does put an emphasis on it. And now I guess like you already know what to do. So please subscribe to the channel if you like the content. Don't forget to comment if you have any questions. I hope you found this video useful. It's one of the most hyper realistic text to speech converter that I have seen so far and given that it has all these other abilities that you can use, these non speech sounds, the filler words and the ooze that makes it really human like not robotic that you can actually get from other tools like 11 labs or why start AI? So I hope you learned something new. Let me know if there are any questions. Thanks for watching. See you in the next one.