How to Clone Any Voice With AI | Tortoise-TTS Tutorial
LARRUZ walks through cloning a voice with Tortoise-TTS, an open-source model. You need three 10-second WAV samples at 22 kHz with no background noise, upload them to a Google Colab notebook, run the cells, and pick a speed preset. The output quality depends heavily on your audio samples.
Chapters
In this video I'm going to show you how to clone your voice or somebody else's voice using this cool and free AI tool. Just keep in mind to use this tool responsibly, all you have to do is to provide some samples of the voice that you want to clone and this tool will do the rest. I'll walk you through a step by step process including how to get the best possible training data. Watch the video till the end for all the tips and tricks. So let's get started. In this video we are going to be using this open source tool called Tortoise Text to Speech. The code is available on GitHub free of cost. You can run this on your own machine. The installation instructions are provided in the repo, but we are going to be using Google Colab to run the code in the cloud. The author of the package has removed the original Google Collab, but there are some other good people on the Internet who have made it available, so we are going to be using that one. But if you want the best results, it's better to install it locally and the installation instructions are pretty straightforward to follow. Now how do you add a new voice and how do you clone it? So here are the all the instructions that you need. So first you need to gather audio clips for the speaker that you want to clone the voice. In my case, I'm going to be using my own voice and I'll show you a software with the help of which you can record voice segments. Then the audio clips are supposed to be in 10 second segments. They need to be at least three segments. But if you want better quality, it's always good to include more data points. You need to store the audio files as WAV files and they have to be in the 22 kilohertz sampling rate. Most of the audio system record in 44 kilohertz, but we need to modify it into 22 kilohertz. And then like if you are running it locally, then you need to create a directory subdirectory called Voices and you put your data in there. But we're going to be doing another cloud. So it's going the last three steps are going to be a little different. Now with the instruction out of the way, we need to collect some data. So for audio recording I'm using this free tool called Audacity. I'm going to put a link in the description of the video. It's a very powerful audio processing tool, but we're going to be using just the basic functionality. Now in order to record your audio, you first need to select your microphone. So come here, click here and then select the microphone that you want to use. So I have this toner microphone. So I will select this. If you recall from the instructions, the audio is supposed to be in 22 kilohertz. So I'm going to come here and set the sampling rate to 22 kilohertz. In order to record an audio segment, all you have to do is click on the record button and then it starts recording your voice. So in this case it is recording and you can see like how many seconds of data you have recorded. So let's say I will go up to around 10 seconds and then you can simply record the stop the recording, right? In order to export it, will you go to file, then export and you want to export it as a wav file. So here I have a directory, I'm going to store this as audio, underscore segment, underscore one and you need around five segments. But in my case I'm going to go around 10 segments. You can also listen to what you have recorded. Simply click on this play button and then it starts recording your voice. So in this case it is recording. So you want to make sure that you listen to it and the quality of audio is good. So I recorded about 10 different voice segments. Now let's look at some instructions that you need to follow before recording any of the audio clips. The quality of the clone clip is going to be dependent on your input. So the authors have provided a guidelines to follow. So for example, they're saying that avoid clips with background music noise or reverb. These clips were removed from the training dataset. Similarly, word speeches, these generally have distortion caused by the amplification systems. You also want to avoid clips from phone calls and then white clips that have excessive stuttering, stammering or words like oh like right. So you don't. You want to avoid those. When you are providing the data as an input, you want to keep in mind what your output should sound like. So for example, if you want to hear your target voice reading an audiobook, try to find clips them reading a book. And it doesn't matter what you are saying in the clips. You just want to have a diverse text so that the model can learn from it. Here's the notebook that we are going to be running. I would recommend you guys to go and make a copy of it. So let's say it's going to create a copy and we are going to be running the copy. So you don't want to be sharing your data with the author of the notebook. First step is just go to runtimes. Make sure the runtime is gpu. So there are some other options, but you want to select gpu, save it, then click Connect. It's going to be very simple process. All you have to do is run a few cells. You don't need to know what code is running. Okay, so once it's connected, you can actually see the system ram, GPU and disk usage. So first come here and click display button in the first cell. This will run the code and install some libraries that are needed for running the model. Just wait for it to complete. Once the execution of a cell is complete, you will see this green sign. So let's go down and execute the second cell. So simply click play button. This downloads a large file, so be patient with the progress. Okay, so this took around four minutes. Let's execute the next cell. I'm going to actually execute this one. When we run this cell, it will ask us to choose the files that we recorded. Let me upload all the files. So all you have to do, select all of them and click Open and it will start uploading the file one by one. Okay, so all the files are uploaded. In this next block, we want to define the text that the clone sound is supposed to say. So just put a plug for my channel. So if you found this video useful, give it a thumbs up and do not forget to comment. All right. And it has a preset option, so you can set it to fast. Fast is the default one. It will process it pretty quickly. Then there's a standard high quality. High quality would take a little more time to process. So let's run this cell and I'm going to run the final cell. So this is where it will generate the audio and then you will see something like this. So let's see what it comes up with. So the results already. Let's see how it sounds like.
If you found this video useful, give it a thumbs up and do not forget to comment.
It doesn't exactly sound like me, but I think there is some resemblance and we can actually play around with the preset. So instead of high quality, you could go with the fast. Let me play this again.
If you found this video useful, give it a thumbs up and do not forget to comment.
Okay, let's try with with the fast. So run this block again and then run the final cell. Okay, so I end it with two settings. The first one is fast and the other one, the Second one is ultra fast, so let's see if it has any impact on the quality of the output sound.
If you found this video useful, give it a thumbs up and do not forget to comment.
And let's see with the ultra fast preset.
If you found this video useful, give it a thumbs up and do not forget to comment.
Okay, I think there are definitely quality issues. There's maybe a little bit of resemblance, but keep in mind that I only provided around 10 samples and the audio that I have recorded is in a room, so it's not really a great quality audio. So make sure that the audio samples that you provide, they're good quality samples. Now, one great feature in this library is that you can actually go control the tone of the voice. So for example, if you want to have a sad tone of the voice, you need to put something like I'm really sad in brackets and then it will speak the same thing but in a saddle voice. You can do the same for like a happier tone or somebody's enthusiastic. So you will simply put, I'm very enthusiastic. I hope you found the video useful. Please comment and share your thoughts and don't forget to like the video if you it helps with the algorithm. Thanks for watching. See you in the next one.
