Daniel Bourke works through Udacity's AI Nanodegree voice track, covering speech recognition pipelines and the shift from traditional modular systems to end-to-end deep learning. He explains how data quality, model size, and GPU compute drive accuracy gains, and notes that end-to-end deep learning powers smart speakers like Alexa and Google Home.
Chapters
What's going on, guys? Welcome back to another episode of Learning Intelligence. It's your boy, Daniel Bourke, and we are here today, we're learning, we're doing the next lesson in the artificial intelligence voice user interface course. I've been going through just the intro to speech recognition. Where's the mouse? I should have got that ready. Intro to speech recognition, there we go. Plenty of glare on the screen because the sun has turned right up, but it is time for me to make a phone call. I'm a little bit late, I've got 1 minute late, but that's alright, we'll get onto it. A friend on here. Reached out on LinkedIn and wants to know if I would be willing to assist him in writing some books on AI. So, I'm going to call him now. I'll see if I can record the phone call on the video. If not, well, I'll just tell you how it goes anyway. Let's see what happens. Hello? Hey, how you doing? Yeah, good, thanks. How are you? Excellent, thank you, mate. Just as a heads up, I make like a weekly video. Do you mind if I like record myself on the phone to you and potentially put it in a YouTube clip. Didn't want to be, uh, recorded, but that's, that's perfectly fine. What was the presence of the phone call about? Well, essentially someone had been reading my LinkedIn profile and then reached out to me asking if I'd be interested in a potential copywriting role, as in, as in being a writer for an upcoming book they want to do as part of their, as what their company works on. And so it's a company that works on machine learning, whatnot, and they're partnered with B2B, and the founder is looking to work on a book per se to give to people in the B2B world so they can understand how to use machine learning models and the ethics behind them and whatnot. So that's a really cool opportunity. Finally, I'm going to send some of the writing I've done in the past to Daniel Bourke and see where it goes from there. If I find out anything more in the future, it might turn into nothing, it may turn into something. I'm not entirely sure, but I'll let you know. And just as a question, like a heads up to you guys, have you read any of my stuff? Like, if you have, leave a comment below. And if you haven't, would you like to read something from me or do you prefer video? I love writing, I love making video, but if you want to see an article or a video of any sort, don't Don't forget, leave a comment below. I'm gonna get back into some, some voice user interface learning, and we'll catch up soon. I've been hounding LinkedIn over the past few videos or so, but that's just another example of how these opportunities, they just, they just arise out of nowhere. It seems like they arise out of nowhere, but really it's, it's the putting yourself out there, guys. I can't recommend it enough. These opportunities, I'm gonna have to start saying no to some because I want to keep working on, on what I've been working on. I mean, I know I'm trying to get into the field and whatnot, so I should be saying yes to a lot of things, but ultimately what you say you say no to is going to decide where you end up. So that's my thoughts on the topic. If you're putting yourself out there doing awesome work, people will find you. Make them find you. Alexa, 25-minute timer. 25 minutes starting now. Alexa, stop the timer. That was a really good study session. Let me show you what I've been working on. So, as I said in the previous clip, I'm going through the voice user interface classes in the AI Nanodegree, and I just finished this class here. So, introduction to speech recognition. Let's dive in. I'll show you where I'm up to. I've gone through all the lectures so far. I've kind of, like I always do, as I go through them first and then, like, in a fast pace, and then go back through them later if there's anything I don't really understand. But I've jumped back into number 13, which is the voice lab with voice data. So, essentially, we're going through and exploring LibriSpeech Corpus Quiz, or no, sorry, the LibriSpeech Corpus of Data, which I believe is an open source corpus of data of people reading audiobooks. So, I'm doing a few questions around that. There might be some code and whatnot. I haven't dived too far into it, but essentially, the next step after I finish this lab is the voice user interfaces capstone, and I'll quickly go through that in the Jupyter Notebook. If we look up here, we've got a bunch of tasks that we're going to have to do. So, here we go. Project: speech recognition with neural networks. Essentially, we're building an end-to-end deep learning speech recognition network. Now, what does that mean? Well, just like how Alexa works, how before we set a timer. Alexa. Yes, I'm here. Alexa, don't worry. So, how does that happen? How does a smart speaker register your voice? It has to be listening all the time and then how How does that turn that voice into a command to do an action? That's what I'm going to be working on, is creating my own deep learning voice recognition system to translate something into text. So, there's a few steps in that pipeline, and I'll quickly show you what's going on. We have speech, and with speech it gets turned into features, and then features get turned into an acoustic model. Acoustic model gets turned into phonemes, which is essentially little sound bites that make up words. Then the words get passed to the language model, then it gets passed to text. So, there's a fair few steps here for what seems It seems like it could be a relatively simple process, but believe me, it's anything but. And last thing to finish off this video is this video here. I'll link it in the description so you can see it too. It's Deep Learning for Speech Recognition by Adam Coates from Baidu. And now, this is what the project is based on, this speech here, this talk he gave at a conference. And it's published 2 years ago now, or just under 2 years. It's essentially what kicked off the whole speech, deep learning speech Recognition Revolution. And now I want to show you something on the graph. Now, voice user interfaces have been around for a while. It's always been a dream. Oh, look at this message from my brother. Thanks, Sam. Love you too. It's always been a dream for people to be able to talk to a computer. I've been seeing Samantha. She's an operating system. You're dating an OS? What is that like? I feel really close to her. Like when I talk to her, I feel like she's with me. Mm-hmm. Now, this is one of my favorite takeaways from the talk. So, what we have here is accuracy versus data model and size. Now, I'll show you traditional models first, traditional voice recognition systems. So, they start off pretty well, right? They keep going, but as more data and the model size increases, they kind of teeter off here at the end and just, they don't really improve that much past a certain point, even if you add more data. Whereas, deep learning models, maybe, maybe they don't start off as accurate. So they start here, but they take a while to get, to get very good. But then as the data size increases, deep learning starts to take off, starts to surpass traditional models. We go past there. And now what the VUI capstone project is working on is an end-to-end deep learning model. So this one here, this, this first iteration of a deep learning model was only subbing out little pieces of this pipeline with deep learning at a time. But what the talk was about was introducing end-to-end deep learning, and that's where we're at now. I mean, it's 2018 and we— this is, this is the state of the art now. And so how does that model go? Well, it starts off just like, just like our first deep learning model, may not be as good, but then It just rockets off the charts. Gets better, better, better, better. As you add more data, right here, the model keeps going up and up and up in accuracy. And now what's model size? Well, model size can depend. It means, if you look at a neural network as a series of layers, model size could be more layers or increased connections between those layers. And now what are the two best ways to improve a deep learning model? Well, number 1, of course, is to get more data or get better data, and that's what we've had in this little x-axis here. Alright, we've got a bit more memory. So, that's a good metaphor for the second way to improve your deep learning model, is to throw more computational power at it. And now, how can you do that? Well, you can either make the model size bigger and then add more GPUs. Now, what's a GPU? A GPU is a graphical processing unit. And that's, that's where we've had this inflection point over the last couple of years. It's the introduction of more data, better data— that's still one point, by the way— and more computing power through GPUs, through cloud computing and whatnot. And that's how we've got here. That's how we've got to up here, the state of the art. That's how Alexa, that's how Google Home, that's how all these new voice recognition, voice-to-text systems work, is from using end-to-end deep learning. And now, will there be another way to improve it over time? Can we make this graph go— whew— up through the stratosphere? Probably, but not as of yet. At the moment, end-to-end deep learning is the state of the art. And so, that's what I'm going to be working on with the Voice User Interface. This capstone project is building my own end-to-end deep learning speech recognition system. And of course, once I'm done or throughout the project, I'll update you and my code will be on GitHub once it's finished. But check the links below for anything that I've mentioned throughout the show. I'll put it all through there so you can learn as well. Leave a comment if you have any questions, and as always, keep learning. We'll see you next week.
