YouTube video (Two Minute Papers) in which Dr. Zsolnai discusses and demos Nvidia’s AI voice-cloning technology. The presenter explains that the method can clone a human voice from roughly 30 minutes of recorded samples, demonstrates synthesized lines (using a cloned "Jamil"), compares this work to prior methods (e.g., Tacotron), and notes improved quality, multilingual generalization, and ease of training. The episode mentions an Nvidia early-access program for the tool and is sponsored by Cohere AI.
Questions this video answers
November 3, 2022
Approximately 30 minutes of voice samples.
Cohere AI
Dr. Zsolnai
Nvidia (Omniverse / Nvidia voice-cloning tools)
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to clone real human voices using an AI.
How?
Well, in an earlier Nvidia keynote, we had a Look at Jensen Jr. An AI powered virtual assistant of Nvidia CEO Jensen Huang. It could do this.
Synthetic biology is about designing biological systems at multiple levels from individual molecules.
How it also uses hand gestures that go really well with the explanation. These virtual AI assistants are going to appear everywhere to help you with your daily tasks of the car, recommend shows nearby, and even drive you there. These Omniverse avatars may also help us order our favorite burgers too. And we won't even need to push buttons on a touchscreen. We just need to say what we wish to eat and the assistant will answer and take our orders. Perhaps later, even in a familiar person's voice. How cool is that? And today I am going to ask you to imagine a future where we can all have our toy Jensens or our own virtual assistants with our own voice.
All of us.
That sound really cool. So is that in the far future? No, no, not at all. Today I have the amazing opportunity to show you a bit more about the AI that makes this voice synthesis happen. And yes, you will hear things that are only available here at two Minute Papers. So what is all this about? This work is an AI based technique that takes samples of our voice and can then clone it. Let's give it a try. This is Jamil from Nvidia who was kind enough to record these voice snippets.
Listen, I think they have to change that. Further details are expected later.
Okay, so how much of this do we need to train the AI? Well, not the entire live recordings of the test subject, but much less only 30 minutes of these voice samples. The technique asks us to say these sentences and analyzes the timbre, prosody and the rhythm of our voice, which is quite a task. And what can it do afterwards? Well, hold on to your papers, because Jamil, not the real one, the cloned AI. Jamil has a scholarly message for you that I wrote.
This is a voice line generated by an AI. Wonder if you fellow scholars are going to notice. The first law of Papers says that research is a process. Do not look at where we are, look at where we will be. Two more papers down the line. The third law of papers says that a bad researcher fails 100% of the time, while a good one only fails 99% of the time. Hence, what you see here is always just 1% of the work that was done so what do you think about these voice lines? Are they real or are they synthesized Now?
Of course it is not perfect. I think that most of you are able to tell that these voice lines were synthesized. But my opinion is that these are easily good enough for a helpful human like virtual assistant. And really, how cool is that? Cloning a human voice from half an hour worth of sound samples. What a time to be alive. And note that I have been really tough with them. These are some long scholarly sentences that would give a challenge to any of these algorithms. Now, if you are one of our earlier fellow scholars, you might remember that a few years ago we talked about an AI technique called Tacotron which can perform voice cloning from a really short few second long sample. I have only heard of simple shorter sentences for that. And what is new here is that this new technique takes more data, but in return offers higher quality. But it doesn't stop there, it does even more. This new method is easier to train and it also generalizes to more languages better. And these are already good enough to be used in real products. So I wonder what a technique like this will look like just two more papers down the line? Maybe my voice here on two minute papers could be synthesized by an AI. Maybe it already is. Would that be a good thing? What do you think? Also a virtual CAROJ for you to read your daily dose of papers.
Hmm.
Actually, since Nvidia has a great track record of putting these amazing tools into everyone's hands, if you are interested, there is already an early access program where you can apply. I hope that some of you will be able to try this and let us know in the comments below. This episode has been supported by Cohere AI.
Cohere builds large language models and makes them available through an API so businesses can add advanced language understanding to their system or app quickly. With just one line of code, you can use your own data, whether it's text from customer service requests, legal contracts, or social media posts, to create your own custom models to understand text or even generate it. For instance, it can be used to automatically determine whether your messages are about your business hours, returns or shipping. Or it can be used to generate a list of possible sentences you can use for your product descriptions. Make sure to go to Cohere AI papers or click the link in the video description and give it a try. Today, it's super easy to use.
Thanks for watching and for your generous support and I'll see you next time.
