Udacity Artificial Intelligence Nanodegree NLP Concentration Progress | Learning Intelligence 23

This week we're jumping into the natural language processing section of term 2 of the Udacity artificial intelligence nano degree. Let me run you through what I've been up to today. It's been a pretty productive day. If we look here, I was reading this book on a pile of books. And actually, before I— I'm really keen to read this one. I've been listening to the audio version of it. Actually, all these are good books. If you want to see what books I'm reading, check out mrdbourke.com/books. I'll put the link in the description. Anyway, been reading this book on deep learning. It's called— I can't even touch the screen. It's called Deep Learning with Python, I believe, or Deep Learning in Python, something along the lines of that. The link will be in the description anyway. And I'm taking some notes here. It's written by, I think I mentioned it in a previous video, it's written by Francis or Francois, I'm not entirely sure how to pronounce his name, Francois Chollet, maybe, I think it's French. But he is the author of Keras, which is a massively popular deep learning framework. And a deep learning library as well. And he works at Google. So he knows his stuff. And the way this book is written, I've already, I've been studying deep learning and AI for almost a year now. And I'm picking up different, I'm putting together different pieces of the puzzle by reading this book. So that's why I think it's so important, guys, to try and learn. Although there's some great courses online, like I'm doing the best courses in the world at the moment, in my opinion, but it's always good to learn from different perspectives. That means blog posts, books, podcasts, wherever you can. So, just create your own mental models and don't take any one word that one person says to be gospel. Compare that to someone else who's an expert in the field and then you can draw your own conclusions through it. We go into the Udacity classroom. I went through this lesson today and that is very orange. So, give me a second. I'll just turn off Flux so we can see this screen a bit better. There we go. So, I went through this lesson here which is Intro to Natural Language Processing and that was really fun actually. There was a member here or a staff member from IBM IBM. There we go. I'll show you quickly. Is this going to load for me? From your experience, what are the most important challenges in NLP that need to be solved? One of the biggest challenges is understanding and maintaining a context throughout the conversation. So Arpan, who you just saw, or Arman, I'm not entirely sure. That's Arpan, I think, and that's Arman. Arman works for IBM, which is where we've been going through, or what we've been using for this project. Now this is called Bookworm, and it's one of the projects in the NLP section on the AI Nanodegree Term 2. And we're going through here using IBM's Watson, which is kind of like IBM. So AI models are best run in the cloud, guys, unless you have a giant computer which can run all your, all your deep learning models and all that sort of stuff. I don't. I have a laptop with a CPU, not a GPU. So I use cloud models. So IBM, Google Cloud, Amazon Web Services, you can, you can use a state-of-the-art APIs from them to build incredible AI models. And so what we're doing with this project here is using IBM's Watson to ingest, I think it's Star Wars, a corpus of text of Star Wars dialogue, and then derive some meaning from that. But I'm still working through that and I'll show you a bit more once I've progressed through it. And the last thing, or last, two last things for this clip, this book. I've been, I was recommended it by Arpan, which is uno Arman, I'm sorry if I'm getting the name right. The guy that works for IBM, really smart dude. I really love to listen to him talk. And he recommended this book, The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. And you know, I love anything to do with AI, so I think, and I love reading, of course, so I think this book, I've got the trial or the sample chapter on Kindle, so I'm going to read through that maybe later tonight, maybe in the next few days or so. I'll let you know how it goes and if it's worth picking up or not. Last but not least, if you want some free credits on IBM's platform and another free learning resource, I just found this one out today. I think it's— there we go, cognitiveclasses.ai. You can get $1,200 worth of IBM Cloud value by signing up. Don't use my code because I've already used it. Sign up for a free account on cognitiveclasses.ai. They've got a bunch of other courses. I haven't even checked them out yet. I've already got my courses that I'm supposed to be doing, so it's, it's best for me not to get distracted. I think we're gonna get a storm here tonight. It's coming. Skies are cloudy, rain starting to fall. Who else studies really well when it's raining? Let me show you what I've been up to. Today was all about the feature extractions class in the Udacity NLP concentration on the AI Nanodegree, and it's really cool. So let me, let me just read out here what feature extraction is. So transform text using models and like bag of words— oh, you can't read that— TF-IDF, Word2Vec, and GloVe to extract features that you can use in machine learning models. I won't go through all those in detail, but I'll just, I'll just share with you some of my quick notes that I took on them. We come over here, we start with bag of words. So essentially, bag of words treats each document like an unordered group of words, and then it picks out each term, and each term can be treated as term frequency. So how often does the word little occur in this sentence. We got one, house, one, and then we have— we go on to— you can work out the relationships of those between those words by using the cosine similarity or the dot product. I think that's really cool. That's a really entry-level way of finding it out. And then TF-IDF is another way of adding weights to words to which signify their relevance in documents. So you compare the term frequency, which is back up here, how many times it occurs, with the inverse document frequency. So how many times does a particular term, let's say little, occur in, in the one document and how many times does it occur in all of the documents? And then we've got one-hot encoding, which is essentially creating one-hot vectors for different, different words. So house obviously has a 1 there, zeros for everything else. Same with lamb, zeros for everything else. If you've seen one-hot encoding before, Similar setup to computer convolutional neural networks. And then word embeddings is probably one of my favorites, actually. It maps the dimensionality of the relationships between these two words. So, if you see here, woman and queen are fairly close because a queen is a woman, right? And man and queen are— man and, sorry, man and woman are related because they're both humans of different sexes. But man and king are closer than woman and queen, or as close as woman and queen. So that's like working out the different relationships between the words. And then you can go further on by using models like Word2Vec to really turn that into numbers by creating a word vector using some networks here by feeding in a word at the start. We're getting too deep on this stuff, guys. If you want to check it out, I'd highly suggest looking up some blog posts on Word2Vec or GloVe, I believe, is, is another major one which uses co-occurrence probabilities. So, for example, given the context solid, how likely is the word ice to appear versus the word steam. And now if you think, if you're given the context solid, the word ice is going to appear much more than the word steam. So as always, what it's trying to do, what we're trying to do here with the feature extraction is get that extracted data out of text because computers, they don't speak like us. We are diverse creatures. We can say whatever we want. We can even sing if we wanted to. So, they have to extract the features out, extract the patterns out. What it is, is it's just like a convolutional neural network or computer vision task. It's drilling down, getting the features out. How are these words related to each other? How can we turn them into numbers? And how can we eventually model them to work out something significant? Which is what tomorrow's class will be on, which is modelling. I then treated myself with a little video here. I'll put the link in the description, which is MIT self-driving cars course. Sasha Arnoud, I believe, Arnoud, director of engineering at Waymo. That's a really cool video. So it's a great insight into how to build a production, or not exactly how to build it, or what it takes to run a production-level AI-driven, get it, AI-driven, did you get that pun? AI-driven company, which is really cool. And I think Waymo is now there, they've had 4 million purely self-driven miles. I think that's incredible. Like, no one, no one in the driver's seat. Like, the car is driving itself. I, for one, can't wait for self-driving cars. Can you? Put a comment below if you think self-driving cars are a good thing. I'm not the biggest fan of driving. I don't mind getting driven other places. I love Uber. I think self-driving cars are going to be really cool when they come about. Before we get to this week's shoutouts and wrap up Learning Intelligence 23, I want to tell you a quick story of something that happened yesterday. Let me, let me just sit this Sit this down and get on a, should I do it on a squat or kneeling down? Yeah, let's do a kneeling down one. Okay, so what happened? Well, can you even see me properly? Maybe I shouldn't lean down. I'll get the chair. You probably couldn't see me, but oh well. So yesterday or a couple of days ago, last week actually, I went for lunch with my godfather who used to work for LinkedIn. So that's a really good contact to have. And oh well, I love him, so it's really good. Relationship to have regardless of where he worked. I went to lunch with him and he was meeting up with someone and he said, "Give me a second, I'll introduce you to this person I'm meeting." And so, long story short, we got into talking and I just— he was asking me questions about what I'm doing. I told him I'm studying AI. I told him I've done a bit of web development in the past with my own website and social media marketing and whatnot, just little bits and pieces. Long story short, I think I've said that twice already, so trying to make this long story even shorter, he wanted to meet up with me and we met up yesterday and we had a great conversation. We had lunch and essentially it was just talking about what I'm studying right now and what I, what my ideas are in terms of where AI is going. So I think the lesson in that, guys, is that if you are studying AI, just be aware that this is a massively growing field and there are lots of people that will want to share your knowledge. And at least if you're sort of— I'm still new to the field, right? And so what I'm trying to say is don't be afraid to talk about what you're learning. And I think there can be some imposter syndrome, and I get that as well. I've had the benefit of having a lot of practice driving Uber telling people what I study every week. So, I see someone and they ask me, "What do I study?" and I say artificial intelligence, even though it's— I'm still about only 6 to 12 months into this field. At the start, it was hard for me to tell people that because I felt like I wasn't the right person or I wasn't capable or I wasn't really a developer. It does take a lot of practice and even me, someone who I look, I appear confident on these videos and trust me, that's how I am in nature, but it still takes a lot of practice to be able to break into that and break through that barrier of imposter syndrome. What I can say is that if you've ever written a line of code, if you've ever dabbled in AI, if you've ever watched a deep learning lecture, you're in the field. Don't be afraid to have a conversation with someone about AI. And I'm gonna— the follow-on from this is that I'm going to meet up with this person in the future and keep discussing the stuff that I'm learning. That's what these things can learn to— just lead to, sorry. Just having a conversation with someone, you never know what they're— who they are or what they're into. They might not be into AI at all, but that's okay. You probably won't be into half the stuff they're into. The takeaway message from this is relationships are key in whatever field you're in, and as AI starts to get more and more popular, it's fundamentally important that we all start talking about it more and more. That way we can share our knowledge, and that way we can all help to build a better world and make sure the, I don't know, the dystopian world that some people think of AI is going to take over. Because I think it's such a great field to be in, and I thank you all for watching my videos. And together, along with everyone else in this community on the internet and everywhere else, we can build something great. That's a quick little story. Don't forget relationships, and don't forget to speak your truth. Let's get to some shoutouts of the week. Shoutouts of the week! Woo! Okay guys, so these people either reached out to me via YouTube comment, email, Twitter, wherever. You can contact me on the internet anywhere you like. Daniel@mrdbourke.com, aka www.mrdbourke.com, or Twitter @mrdbourke. Otherwise, leave a comment. Everyone else can see it. That's probably the best place to do it. Saroba. And now, this is a reply to the question I had last week, which was, what are your thoughts on the singularity? Will we reach artificial general intelligence by 2045? Sarab, I hope I'm saying your name right, Saraba, Saraba, Sarab, Sarab, I think it's Sarab. Sarab, great answer on last week's singularity question, and it is true, the amount of data that we're getting these days is expanding dramatically, and computing power is only getting more and more accessible, and that's what allows people like you and me, we can work on the latest frontiers in AI, from our bedrooms and we have an internet connection. That's— I think that's amazing and it's only going to get— this— we couldn't have done this stuff 5 years ago, 5, 10 years ago. So, I think it's only going to get more and more prevalent in the society, well, in the world really. There's data flowing out everywhere and I can only imagine, I'm sure you can too, what's going to happen by 2045. So, maybe not that date specifically, but just within the next 20, 30, 50-year time frame. It's going to be crazy, guys. I think we're going to be talking to our children when we're 40, 50, 60 years old, 70 years old, 80 years old, being like, we had to use our iPhones to connect to the internet. Who knows what it's going to be like in the future. Thank you, Surabh. From the YouTube comment space as well, we have Deepak. So, Deepak, thank you so much for your comment, and thank you for your kind words on my Deep Learning Nanodegree review video. And as for your question versus the ML nano— Machine Learning Nanodegree on Udacity and the Deep Learning Nanodegree on Udacity, I had a look quickly. I haven't done the Machine Learning Nanodegree, but what it seems is it's a bit more broader in terms of scope in the, the world of machine learning. It does have some deep learning projects in there and it is a longer timeframe. So the Deep Learning Nanodegree goes for 4 months, whereas the Machine Learning Nanodegree goes for 6 months. Nearly, nearly messed up my fingers there. And it costs a little bit more. So you will look at some machine learning stuff versus deep learning machine learning stuff, machine learning content as well as deep learning content in the Machine Learning Nanodegree, whereas the Deep Learning Nanodegree is solely focused on deep learning. What are the two major differences between them? While machine learning is and deep learning are quite similar, but deep learning, uh, you do need a lot more data. That's how I think of it. D for deep learning stands for data. Deep learning, you can never have enough data. You just keep throwing it at it. Back to what we just said before, so much data is becoming available. That's why deep learning is taking off. So if you're looking at problems that you want to work on that have a lot of data available, I would say look at the deep learning nanodegree first, maybe. But if you want to get into the general machine learning, maybe you don't have as much data for the problems you want to solve, machine learning may be somewhere you want to look into. This week's question of the week. Now, in light of the Waymo video that I shared throughout this episode, what is your favorite AI company and why? Waymo looks really cool. I love their philosophy, I love their attack at everything. I love Google's approach to almost any problem. They just get stuff done. Comment below, What's your favorite AI company and why? And I'll shout out the best comment in next week's video. And thank you again to Surabh for your comment on last week's question of the week. Next week I got invited to a Google Cloud event. So there's going to be a bunch of Google Cloud Platform trainers coming to Brisbane and they invited me along to come and check out what to build next with Google Cloud Platform. So I'm really excited for that. I'll bring the camera along, I'll film what I can, I'll write about what I can. So So, be sure to tune in to next week's video. I'll put something there. Maybe it'll be a separate video. Who knows? I'll try and work out how to best share it with you guys. Otherwise, we're always going to be jumping, following the machine, not the machine, the AI Master's Degree. Next week, I'm continuing on with the natural language processing classes in the AI Nanodegree. Thank you so much for watching and keep learning.