Coursera Deep Learning Specialization Progress | Learning Intelligence 15

What is going on, learners? Welcome back to Learning Intelligence, episode 14. Fun fact about the number 14: I have 2 dogs. Divide 14 by 2 and you get 7, and I have also have a dog called Seven. So, fun fact to begin the video with. But today's been an epic day. I've been working through the deep learning course on Coursera as I was in the last video, and check it out, I just passed week 3 programming assignment. So I got 100 out of 100 points, and that was my second submission actually. Where is it? There was the first one. So I got 90 out of 100, which is still passing. You only need 70 out of 100 to pass. But then I went and resubmitted it, fixed my error, and got 100 out of 100. And that's one of the things I like about these courses is that you can resubmit the projects and resubmit the quizzes and implement the knowledge that you've learned from your failures or from your, I don't know, from the answers you got wrong in the past one. I think that's a really good way to learn. 'Cause I remember in the past I'd submit projects and I wouldn't really get feedback on them or wouldn't really know where I'd go wrong or You'd submit an assignment, for me in my case anyway, at university, and it would be 2 to 3 weeks minimum before you get feedback on it. And this, I know code is a lot sort of, you can put in test cases and mark it almost instantly in some cases. It'd be really hard to mark an essay with a computer yet. But when natural language processing gets better, you could probably use that to mark essays. Who knows? But I want to show you 2 more things for the introductory clip. First of all is the Deep Learning book by Ian Goodfellow. Actually, Ian Goodfellow was in the interview of the Coursera course. So, at the end of each week, there's an interview with a hero of deep learning, and today was Ian Goodfellow. You may or may not have heard of him. If you want this book, by the way, go to deeplearningbook.org. It's free. All the chapters are free. I'll put the link in the description. But Ian Goodfellow has invented GANs, and if you haven't heard of GANs, I won't go through them entirely in this video, but a brief overview is it combines game theory deep learning. So essentially it creates two networks, two neural networks to work against each other to, to produce an output. But if you're interested more in GANs, I'd look up one of Siraj's videos on GANs because he does an incredible job at explaining it. And the third thing is one of my articles got published in the FreeCodeCamp Medium publication. You may or may not have heard of this, but Medium has some incredible articles and I wrote an article The next step towards artificial general intelligence: StarCraft 2. Look, there's me. Look at that face. I wrote an algorithm the other day and my face got detected as a cat, so— wrecked. But yeah, this is my opinion on why I really love reinforcement learning and game playing. And look, it's even Christmas-themed because that's coming up in a few days. I got the Christmas tree explaining what what deep learning is in a nutshell. So, I'll put the link in the description to that as well. If you could go give that some love on Medium, get it, get it out into the universe, I'd really appreciate it. I do love writing, I do love making videos, so if you guys could give some love to that, thank you so much. But as for the rest of this week, I'm going to be going hard on the deep learning course on Coursera. I want to get at least 50% done by the end of this week, so we'll check in with the progress. Something I forgot to show you in the first clip, so really it's, it's 4 things I wanted to show you in this first clip, it's been a big day, was what I did for the project. So, The idea was to build a neural network to classify these blue and red dots. And you may not be able to see, but there's some blue dots here, red dots. And the idea was color these zones, which color is predominantly of the dots. So you can tell this one is predominantly red, see all the red here. This one's predominantly blue, red, blue, blue, red, et cetera. And so the neural network that you built by the end of the project classified them with an accuracy of 90%. And now let's check out the logistic regression model, which is essentially just a straight line. So yeah, see, it said this is predominantly red, this is predominantly blue. And the output was only a 47% accuracy. So that's a really good example of how much of an improvement you can get with just a single layer neural network. And deep learning uses multiple layers. So over 1 is classified as deep learning, and, and from what I've gathered. So I think some of the biggest deep learning nets, like very deep learning, is classified as about 16 different layers. So you can imagine as the input goes through all those different layers and all these massive math— mathematical computations are made on it, you can get some very, very accurate outputs and predictions on the other end. All right, check this out, guys. I just passed both assignments for week 4 of the deep learning specialization, and now I'm testing my models on my own image. So ideally it will predict 1 if it's a cat picture or 0 if it's a non-cat picture. So essentially it's a cat version of hot dog, not hot dog. And I need to tweak my model because look at this. Cat picture. That's clearly a photo of me. Predicts a cat picture. Another photo of me. Predicts a, you guessed it, cat picture. That's a beautiful car though. And cat picture and cat picture. So clearly there's something going on with my model, but the algorithms that I wrote, the code that I wrote, managed to pass all the test cases that they had for the assignment. And the same, same goes for the other assignment I completed today. So I managed to get 2 done today. It's been a big day. It's been, it's been good fun. So this was number 1. Essentially, you build your deep neural network step by step, and this is an overview of it. You start by initializing the parameters, so you have the, the weights and the biases. And you come through here, you build the forward propagation network, and then you come down, calculate the loss, Feed it through the backward propagation network, update your parameters, and this is a, this is a loop depending on how many layers of your neural network you have. And then finally, you make a prediction on the dataset of what you think the, or what the algorithm thinks the next likely output is. And because of this, because I passed these two, check it out guys, we passed, or we, we've successively, successively, successfully Completed course 1 of 5, Neural Networks and Deep Learning. So you know what that means. Tomorrow I'm going to be going on to part 2 of the specialization. So improving deep neural networks, hyperparameter tuning, regularization, and optimization. I can't express to you how much I'm loving this course, right? So if you, if you haven't done the Deep Learning Nanodegree before, I would highly suggest, and if that's, that's too big of an expenditure for you, I would highly suggest going— the Deep Learning Allegory is from Udacity, by the way, amazing program. But this one, if you've never done anything to do with deep learning, I'm finding this has a much more foundational approach. So, it starts from the ground floor. And now, look, I might be a bit biased because I've already done the Deep Learning Allegory, so I do have some foundational knowledge of what deep learning does. But from the brief time I've spent on this course, I've only been enrolled for it for 4 days and I've finished module 1, it has definitely much more of a ground floor approach to building up your knowledge and your, your ideologies of deep learning. So, tomorrow I'm going to start in Part 2. I'm really excited for that. And the first one was supposed to be 4 weeks, but I got it completed in 4 days. It's only about 4 to 6 hours per week, so I was doing that per day or so. And this one's— Part 2 is about 3 weeks in length, but I should be able to get that done by the end of the week. So potentially from the end of this video, we'll be halfway through the deep learning specialization. So we started on week 2 of part 2 of the deep learning Coursera. Today I started on part 2 of course— Today I started on part 2 of the deep learning specialization on Coursera. I've made it through the first week of classes so far. I'm up to the programming assignment, which I'm about to do next. But I thought I'd share with you 2 of my favorite things that I found during the lectures, and one of them was on bias and variance. So essentially, if your model has high bias, it's a set— it's underfitting the data. And if your model has high variance, you can think of it as overfitting the data, like being too close. And the way to fix that is with high bias, you can train the model for longer or add more complexity to your model, maybe more layers or something like that, more hidden units. And to fix high variance, so overfitting, You can use more data or you can do add regularization. And regularization was something I liked and I found an analogy that regularization is akin to, say you are raising a child, right? And you want to give the child enough freedom that they decide who they want to be on their own. At the same time is you want to, you want to give them some constraints so you make sure they don't go too far off the rails. So, for example, if you want your If you want to raise your child in the perfect balance, you might say, okay, you can have this, this drawing pad, you can use your iPad here and there. However, you also have to do your homework, get good grades in school, or exercise at least once a day or something like that. So, regularization helps to reduce the overfitting of your model by making sure that it fits just right. So, not too, not too, not too much, which is overfitting, which is what you don't want, and not, not too little, which is underfitting. So there's a balance there that, that can be found, and that's, that's with all, all deep learning models, right? If you find that it's— if your model's underfitting the data, well, it may have high bias. And if you find that your model's overfitting the data, well, it probably has high variance. And there's some, some quick and easy ways that you can fix both of those things. So I just finished the assignment for week 1 of part 2 of the deep learning specialization on Coursera. And it was really great actually, we went over, so this part of the course is all about, say you've got your deep learning algorithm ready, it's all about optimizing that algorithm, making it better. So yeah, your deep learning model, sorry. So say you've got your model there, and this part of the course is going over, I think week 1 was to do with initialization, regularization, and gradient checking. And now we're up to optimization algorithms. So these are all ways to, to improve your deep learning model. So it's one thing to, to have a good deep learning model that works and produces data, but how do you make it better? So for example, initialization is something that we covered in the assignment, and you build each of these step by step in this, this course. Actually, I really enjoy that. It sort of breaks it down. It really takes it back to first principles. So initialization is— an example of that is is choosing the optimum weights to start your values at. So if you start— if you initialize your weights of your deep learning model to start at all zeros, well, then it's not going to make any progress because it's just going to continually produce zeros. So what I learned in the initialization phase is that it's good to, to randomize the weights when you begin with. And then as your model progresses forward through the forward propagation layers and then through the back propagation layers, those weights get updated. And then there was something really cool as well, was an interview with Yoshua Bengio. If you haven't heard of him, he's also a deep learning professor much like Geoffrey Hinton, and he's someone you should definitely look up. And my favorite thing from what he said was his advice on deep learning is to play around with a lot of toy problems because toy problems allow you to— it increases your research time. Oh, sorry, speeds up your research time so you can do more problems over and over and over again. So he said instead of trying to tackle the biggest thing you possibly can, start with, with the smallest thing you possibly can and then perfect that and iterate over and over and over again. And that's something I need to work on. So future videos, I will be making deep learning models from scratch. I'm going to finish this course first and then we'll, we'll get into those videos sometime in the future. Today was all about optimization algorithms. So we started off with this initial dataset. This is called the moon dataset. So you see how the data, the blue here, is shaped in like sort of a moon, and so is the, the red, the red dots are shaped in a moon. And what we did was we built a 3-layer neural network to model that data. You see here we built, use the gradient descent optimization, and with the learning rate, they all have the same learning rate, 0.0007. And you can see it's pretty noisy there, and ended up with 79.6% accuracy. And fairly, fairly okay split of the data there. And then we used momentum optimization, and that got the exact same results as the gradient descent optimization. And then finally, the Adam optimizer blew the other two out of the water, 91.6% accuracy. So round it up, about 92% accuracy. Awesome modeling there. Each of those algorithms we built from scratch, and I found that to be really good because you can implement these I have implemented the Atom Optimizer before with TensorFlow in the past, and it is, it is built into TensorFlow, which is a deep learning framework. But building it from scratch is giving me a deeper understanding, and that's what I'm finding with this course. It really takes it from a ground-up approach. So the Deep Learning Nanodegree that I did a couple of months ago was kind of like diving into the deep end of deep learning, but now I've sort of, I'm doing it backwards. I started implementing the models in deep in the Deep Learning Nanodegree and building projects and whatnot. But now I'm, now I'm getting that base, baseline understanding of deep learning through the Coursera Deep Learning AI course. So if you learn in a fundamental way, I would suggest the Deep Learning AI course and starting from there. But if you like to dive straight into things, you've got a good, good understanding of how, how different neural networks and stuff work, the Deep Learning Nanodegree is probably for you. So next, We're learning about hyperparameter tuning, batch normalization, and programming frameworks. This is week 3 of course 2, and I'm excited. There's an assignment at the end of this one on TensorFlow, so let's do it. Oh, PS, you like my banana singlet, if you can see it? If you do, leave a comment below or share with me your favorite fruit. Yeah, what fruit should I put on a singlet next? Alrighty, so we're about halfway through the lectures on week 3 of course 2 on the Coursera deep learning specialization. And this week goes through— I had it here, there we go— hyperparameter tuning, batch normalization, multiclass classification, and introduction to programming frameworks. Now I'm really excited for that last one. I'll go over that in a sec. But interesting point Andrew went over here about tuning your hyperparameters. In terms of importance, he ranked learning rate as being number 1, and then you got theta and the number of hidden units and the mini-batch size as being second in importance. And then you've got the number of layers and the learning rate decay as being third most important. And the beta, the beta 2, and the epsilon— he, he uses the Adam optimization algorithm to set these automatically, which go to 0.9, 0.999, 10.8. And now if you're not sure what hyperparameters are, just imagine— what can they be? A set of criteria to help improve your model. That's it. So it's like a, it's like a guideline. So when you go bowling, uh, when you first start out, you might have the bumpers up to help you help guide your, the ball down the, down the lane. And then as you get better and better, you might slowly decrease the amount or, or change the amount of bumper you use. And that's the same with a deep learning model. But a bit more basic. With a deep learning model, you might start with some hyperparameters that are like a safety net, like just trialing it out, see how you go. And then as it gets better and better, you'll slowly tune those hyperparameters, which will be equivalent to, to lowering the bumpers on that bowling lane. And now it's time to wrap up Learning Intelligence 15. It's been a great week, guys. Getting, getting some great progress through the deep learning specialization on Coursera. As I said, this video will probably drop just before Christmas or that time, or New Year's around there, probably before New Year's, more around Christmas. So, Merry Christmas if you're watching, and of course, thank you for watching. But my goal is to have the course almost entirely finished by Christmas. I think I'm just on track, or if not, just a bit before. I'm certainly ahead of schedule on what they recommend. Some of the courses, they've said it will take up to a month, like doing 4 to 6 hours a week. Instead, I'm doing 4 to 6 hours a day. So it's definitely accelerating the process. It's really good fun. And if you haven't been able to tell from the previous videos or the other clips, I'm really enjoying this course. And if you're looking to get into AI or deep learning, in comparison, I'll do a full review of this video— of, sorry, of this course once I've completed the whole thing. But so far it is probably the best resource I've found for learning deep learning or AI from a ground floor approach. So if you're new to deep learning at all, new to neural networks, this is probably somewhere you want to start, the, the deep learning specialization on Coursera. And of course there'll be a link in the description. But otherwise, it's time for some shoutouts of the week. In no particular order, these people reached out, commented on my videos, sent me an email. You can send me an email too if you want, daniel@mrdbourke.com. Otherwise, leave a comment if you want my help on anything. I'll try answer as best you can. If you want to see a video in the future, leave a comment below. Nonetheless, thank you to Rutaraj, Snunhan01, Nexus Wars, Zakia, Larry, and Michael. You guys are amazing. I really appreciate you guys reaching out. It means the world to me that you're interacting with my videos. I hope I'm keeping to bring you, or I hope I'm continually bringing you value as much as I can. What's in store for next week's video? Well, we're going to keep going through with the deep learning course on Coursera. I'm really liking it. As I said, I want to try finish it through, through before Christmas, and we're just going to straight line it, do that 4 to 6 hours a day. And of course, I'll be bringing you along with me. Thank you so much again for watching. I'll see you in next week's video. If you want to see anything else, leave a comment below. Don't forget to like, subscribe if you want to, and always keep learning.