How I'd start learning machine learning again (3 years in)

3 years ago, without knowing anything about machine learning, I had the audacity to start my own AI master's degree. And what's that, you ask? Well, 2017 Dan, take over for a second. Well, what do we have here? I finally got around to writing up a curriculum for my self-created artificial intelligence master's degree, and it's on Medium. It's all out there in the world. It's practically a one-stop shop for all of the courses I've done and, and why I'm starting artificial artificial intelligence and why I decided to essentially create my own master's degree rather than go back to university. I'm using online courses to, to pave my own way. My friend Dave gave me that haircut and I mean, thank goodness I've grown some hair since then. And while I was studying my AI master's degree, I decided to, because I was doing it in this room here all alone, I decided to document the process online in a, in a video series called, I don't know what I'm going to call this one yet. Maybe learning intelligence or building intelligence. I think learning intelligence is a good one. So, today I went over lesson 7 and 8, almost finished lesson 8. So, the first time, it was all new to me, so I didn't really understand it and I had trouble. I don't know, I went through and I was sort of getting demotivated because it was so difficult. But now going back over it a second time, it's starting to sit more. What's up, y'all? Welcome to Learning Intelligence episode 7. It's been an awesome day of learning today. What's going on, learners? Welcome back to Learning Intelligence, Episode 13, Episode 14. It's quarter past 8 at night and I just finished Week 2 of the Sequence Models course. Can you hear that? What do we have here? And 9 months into studying my own curriculum, because I was sharing my work online, someone reached out to me and I was lucky enough to get offered an internship at a local machine learning company. On the way to the internship. A little bit scared. I'm heading off to day 2 of my internship at Max Kelsen, which I'm really looking forward to. And 2 weeks into that internship, I got offered a role as a junior machine learning engineer. I just got off the phone with the CEO of Max Kelsen, Nick, and I just essentially got offered a role. So my first day is tomorrow. It's happening, guys, getting into the industry. Today's research day. The ML team have dedicated Mondays to reading papers and looking up new ways of doing things. And it was the best job I've ever had until now, of course. About 14 months ago, I was in this exact kind of position. It was my first day starting an internship as a machine learning engineer. And that led into a job. And now coming full circle, today's my last day. 6 months after leaving, I launched a machine learning course with my good friend Andrej to answer the question I would get most often. I want to learn machine learning. Where do I start? Hello everyone, nice to meet you all. My name is Daniel Bourke. We're coming at you live from my bedroom, which also doubles, or used to be, the parents' garage. And the reason of this stream is mostly just to answer any questions that you have. My good friend Andrei and I just launched a machine learning course on Udemy. It's got a lot of, a lot of stuff there. And 6 months after launching the machine learning course, well, That's where we are now. So if you add up all of those timelines, you get about 3 years. So if I had to start over again and start learning machine learning from scratch, what would I do? If you come to this video for a list of courses, it's not that. It's more of a philosophy around learning itself. So I've got some points here. There's an article to go along with it. If you prefer reading, make sure you check that out. The link will be below. This video doesn't contain the exact same things. It's more of a riff an extension of what's in the article. Let's get into it. You can consider each of these as kind of like a little bit of advice to myself, my past self, and of course it's going to be all biased on my own experience. So, take what you want. If you're experiencing this too, you can critique my points and help out other people who are on a similar journey. So, number 1 is I would avoid, or past Daniel, this is a note to you, avoid credential-seeking mode. And so, it's very easy these days to kind of get obsessed with completing a whole bunch of online courses and getting certificate after certificate after certificate after certificate. Ta-da! The point here is, is that certificates don't necessarily equal skills. But if you develop skills, they may lead to certificates. And I kind of got stuck in this when I was going through my own Artificial Intelligence master's degree. It was a list of courses and I kind of got stuck in this, I was over-optimizing for just completing sets of courses rather than developing the skill of being able to solve a problem, which is more important than getting a certificate. And so, this is what I tell people a lot of the time, even if they're going through my course, it's like, I don't care if you complete it or not. I'd rather you just go through the whole thing, don't even bother about finishing it, go through it, Take what you need, learn what you need, and then use it in your own problem. And so, this is a big thing. It's like a note to my past self is that rather than watching videos on like 1.75 speed just to— I seriously would do that, watch videos on 1.75 speed to get through them quicker, to get to the end. Really knuckle down and don't finish or don't claim the certificate unless you really understand what's going on. And so, of course, I value speed and being able to go through things and learn things fast. But there's no point to it if you're just burning through course lectures or looking for the magic course to teach you the thing that you want. I wouldn't finish something unless I could replicate it myself, if that makes sense. So, rather than— you don't want to be that person who when it comes up to solving a problem of your own, all you've done is you've optimized the skill of being able to complete more courses rather than work on your own project. So yeah, avoid credential-seeking mode. The next one is take the leap. You know, someone asked me the other day, how do I avoid tutorial hell? And I didn't really know what tutorial hell was, so I looked it up and I kind of, you can kind of think about it and figure it out. So my definition of tutorial hell is you want to work on something of your own But you kind of just get stuck in this mode of just seeking a tutorial every single time for someone to guide you through it. And so, where Take the Leap comes from is eventually you're going to have to get to a point. It's like when you're riding a bike. You start out, you have training wheels, and sure, you're riding along, you're having a lot of fun, but then eventually you want to go out on your own. You want to break away from those training wheels because they're slowing you down. You can't really— I don't know if you remember riding as a kid, but when you're kind of falling left and right, those training wheels, they add more friction, slows you down. Eventually, your parents took those training wheels off and they said, "You're off on your own." I remember it, it was someone, I think it was my mum, running behind me holding the back seat, and then as I went off by myself, I was like, "Oh my God, oh my God, I'm by myself now. I don't have anyone." And so what happens then? You might fall over, you might skin your knee. I mean, some blood might come out, you might shed a few tears, but you get back up. You have a faith in your abilities, a true belief in your abilities. You know, that's the kind of world that I want to live in. A world where people have more of a true belief in their own abilities, their own abilities to adapt when necessary to solve a problem, to avoid tutorial hell by taking the leap. I'm doing a lot of physical activity here. Take the leap. So this is where your skill set is, and this is where you want to be. This is tutorial hell. What you have to do is take a leap. You have to take that leap of faith. You have to jump. You have to take the training wheels off and give it a go. What's the worst that's going to happen? Skin your knee? Hurt for a little bit? That's alright. It's only going to make you better. What happens after you get back on the bike and you ride it a few more times? Well, you get better. You ride a bike like it's like you're breathing. And often the benefit of taking the leap leads into you building something of your own. That's where you really learn. That's where the real knowledge is. Not, no, there's no magic course. We've said that before. There's no magic course because if something can be taught, well then that knowledge is commoditized. Where the real knowledge happens is when you're learning, when you're tinkering on building your own thing, going end to end. You know, once you get more and more involved in the field of software engineering and any type of technical endeavor, you'll find out that a lot of it comes from self-teaching. You know, when I was working in my job, I was really surprised that there was— I was— because I didn't go and do a computer science degree, I was working with people who had done computer science degrees and they weren't necessarily taught how to code. So I'd been studying in my bedroom for like 9 months and I could code things at the same skill level as them because all I'd focus on is code. Of course they had skills that I didn't have But in terms of practical knowledge, we were almost on the same playing field there. And you'll often hear stories of people online, like of software engineers and whatnot, they taught themselves how to code because they wanted to build games and build programs that they wanted to use themselves. So there's the advice for taking a leap. If you want to avoid tutorial hell, work on your own project, work, build something that you would want to use yourself. The third point is to use less tools. And now, this isn't to bludge the usefulness of the plethora of software engineering tools that are out there, but oftentimes I would find myself— this is a reflection of myself, remember these are mostly notes to the past me learning— would be I would seek the use or seek— I would try to find out the newest thing, like the newest framework or the newest tool or the newest programming paradigm and all that sort of stuff. At the time, it seemed like the right thing to do, but then I kind of realised after a while that this seeking of the new tool was me avoiding the difficult task of mastering the tools I was already using. And so, a more concrete example would be, the best programming advice I've ever received is to learn the language not the framework. You see this with like JavaScript especially, and sometimes Python and whatnot, is that there'll be just frameworks galore for, for different things, and they're sort of abstracting away from the programming language itself. Of course it makes things easier, but in the end it's kind of like taking— you take enough shortcuts and you end up taking the long cut. When you're learning something like programming, think about the analogy of the chef. The chef has a kitchen full of different tools that they could use, but what do they use most? It's the controlled use of fire, so a stove or something like that, and a knife. From those two tools, everything else branches out. So if you're learning programming, learn the programming language, so for me it's Python, That's why over here, after 3 years of, of learning Python, I got this book to start learning more. Truth be told, I haven't read through much of this. There's another one called, I think it's called 90 Specific Ways to Write Better Python. And I figured, is this is going to be, if Python is my knife, as the equivalent of a chef, I need to be really good at just this programming language, not all the frameworks around it, just this programming language itself. And for machine learning, the analogy could be because it's all math underneath, improve your math skills. So another book, really recommend this book. So this is, this is the machine learning engineer at the current, at the recording of this. This is the fire and the knife. Math is your controlled use of fire, and your knife is the Python programming language. Of course, take what you want. You can code in different languages or whatever. They're my tools of choice. So past Daniel When you see something that's new and you think it's the right thing to do to learn that, ask yourself, do I really need to know this new thing? Or are you just avoiding the difficult task of mastering the tools that you already have? The penultimate one, that's a fancy word for second last, is if you want to learn something, teach it. I heard a saying the other day, it's like, you remember 10% of the things that you read, I mean, you've probably had this. How many times do you read something and just forget most of it? So, maybe that number could even be lower. I mean, the numbers don't really matter, but 20% of the things that you watch or listen to, so that's like watching lectures or instructors or something like that, 70% of the things, of the exercises that you do, so that's putting the knowledge, the things that you've learned into practice, which is why taking the leap and building your own projects is really helpful. Because it helps cement that knowledge. And then, something like 90% of the things that you teach. And so, the action here that you can do is I like the concept of spend as much time thinking as you do doing. And what I mean by that is, because you're smarter doing than you are thinking, is that say you spent an hour watching lectures, spend at least a minimum of an hour trying to re-implement what you've learned in that lecture. So, that would be the 70%, remembering 70% of doing things. Or if you spend an hour reading a book, you would want to spend at least an hour reflecting on what you've done and trying and doing the exercises there. And then if you want to really step up your knowledge, this is where it's going to take a lot more time and effort, is to— if you've spent an hour reading some sort of new topic, maybe it could be 3 hours or something like that. You could create a piece of material to teach someone else that knowledge, but do it in your own style. And developing that piece of material may take a lot longer than actually just doing it. So, it might take 3 times as long. And I can attest to this because when I was building my machine learning course, I'd done all of the— I'd done all of the exercises myself at some point. But to kind of reverse engineer everything that I'd learned and put it in a way that someone else could understand and go, ah, why did I know that when I first started? Why do I do that automatically? Why do I create a training and a test set automatically? If someone who's new to machine learning was, was just starting out, would they know to do that? And the benefit of all this is that if you do create some sort of material or work on some sort of project to to upgrade your knowledge or to use your knowledge, you can share this and it becomes part of your portfolio. So this is, you could be, this could be a sub-point to this. If you want to learn something, teach it. And as a bare minimum, share your work. That way, if you go to apply to a job or something like that, you can say, hey, here are the things that I've done. And the final point, probably the most important one, I should put this one first, but the ordering doesn't matter. Is to have fun with it. You know, when I get stuck in learning something, I have to remind myself that this is all part of the dance, the dance of learning, from jumping from concept to concept, from taking the leap. That's all part of it. I mean, learning in itself is inherently hard. You have to think about it. You have to create new cells or new pathways in your brain that didn't exist before. And so, you can either treat that as a hard task, and of course, I'm not denying that it is at times, but if you flip the switch and look at it from another angle, you go, "Wow, this is actually a challenge that I can take on." And then so, you turn it into a game, you turn it into a dance. And so, when it gets difficult, you go, "Oh wow, I just have to learn this thing, and then I can learn that thing." And then, oh, you go, "Daniel, what happens if I learn something and I figure out that that's useless?" Well, that's the most important kind of knowledge, the knowledge of what doesn't work, the absence of knowledge. And so, you're like, oh wow, well, that's just another stepping stone into this little pond that I want to cross, this little pond that I want to take the leap over and improve my skills. If you made it this far, you're probably going, Daniel, you know what, you're talking a lot of philosophical nonsense, but I think those are important, right? I like to talk about things that can be applied to anything, not just one specific Task. Take those points how you want, see how they apply to you in your own learning journey. Of course, they're biased in my self-learning, but you might go, "Give me some specific steps," and well, okay, let's do that. So, this is— we're not going to go through very— we're not going to go into them in depth, but this is all in the article, "How I'd Start Again." So, I like this point here. I wrote it myself. First of all, more important than any resource that you can find online is to get rid of the "I can't learn it" mentality. That's bullshit. The internet, you've got the internet. The internet has given rise to a new kind of hunter-gatherer. This is where it comes into the fun part. There are resources out there galore. There is no shortage of resources that you need. This is why whenever someone asks, "What's the best course?" There's infinite amount of good courses out there. They're all teaching basically the same thing. It's code or math. And so, Treat the, the internet as your, as your field of different fruits that you can pick from, and you're the hunter-gatherer. So you have to get out there and you have to find the resources that you need. If you want some that are, that are kind of channeled or kind of tailored, this is kind of the path that I would take again if I was starting from scratch. Now the main goal that I would have if I was starting completely from scratch again would be to build more things that get into people's hands. I would take the leap. I would go end-to-end more, because what you'll see a lot is with learning machine learning, you can get kind of obsessed with just improving the model. But as, as you get more and more into it, you realize that improving the model may look good on paper, may look good in your Jupyter Notebook, but it might not look good when someone else is trying to use it and they can't because the model is too big, it takes too long to do inference, the problem that you're solving is actually not a problem someone has, you're just, you're just over-optimizing this model like you're over-optimizing for, for just doing certifications rather than solving problems. Now, if I was a beginner, I would lay some foundations. I would spend about 3 to 6 months doing this, probably as a minimum. Why? Because knowledge takes time. Learning things worthwhile takes time. Again, I'm an advocate for speed. I believe, yes, you can You can learn something fast, but don't consider something learn— uh, don't consider that you understand something until you can replicate the materials that you're, you're learning from. So I'll get machine learning concepts. I'd get your mind ready. So you could use a great resource, is, uh, I think it's elementsofai.com. Learn some Python, learn some NumPy, Pandas, scikit-learn. At the same time, I would read Part 1 of Hands-On Machine Learning with scikit-learn. Now this is This curriculum is very code-focused. It's very practical, very hands-on. So you're writing machine learning code as quickly as possible. That's how I designed my course. You may prefer to learn all the theory and that stuff first, like the mathematics behind it. I personally like to— I'm a very visual learner, so I like to see things working, and I'm biased towards building an end-to-end solution. So something like an application that someone can hold in their hand on the smartphone or something like that and use. At the end of this, I would build a milestone project. You could use Streamlit for this, or you could use pure Python if you're adhering to the use less tools philosophy. If you want to go advanced, so after the beginner, 3 to 6 months, remember there's a plus there. If you need to take longer, take longer. You can go through— I would go through all of Fast.ai's curriculum. I go through a couple of deeplearning.ai specializations. You can pick and choose which ones you want. I would definitely do all of fast.ai as a bare minimum. And then this is where the end-to-end solution comes in. I would go through the full-stack deep learning curriculum. So that's what it means by full-stack. It's kind of like a full-stack web developer, which means you're building— not only are you building the machine learning component, you're building the components around it that a user would use. So someone who you're building something for. You might say, well, Daniel, I just want to go and do research. Well, when you're building things, it turns into research. You want to research different things. This is where Archive comes in. Archive is a resource for reading a whole bunch of technical papers. When you're building things and your, your solution hits a stopping point that it's not providing the it's not providing what you want it to, then you go and research different things to improve your end-to-end solution. So building things turns into research and vice versa. At the end of this, now along here I've got kind of this, this is not like a, how do you say, fully structured, for lack of a better word, fully structured. So this milestone project can actually be whenever you want. But this time, your Milestone Project 2 is going to be deployed. What that means is that it's a, it's a full solution that someone else can access by going into their web browser. So this is where I've got FreeCodeCamp here to do some web development. If you need to learn math, Khan Academy is great. You probably want to choose one of the cloud providers to at least get semi-provisioned with, so when you need to deploy something to the web, you can use a cloud service. And then Missing Semester is actually a phenomenal resource, the missing part of your CS degree, because as I said, a lot of software engineering knowledge is kind of self-taught. This kind of contains all of the— well, not all of the, but a large amount of the things, of the tools that you kind of have to pick up on your own that aren't necessarily contained in courses. So this is like a course containing the tools that aren't taught in courses but that you, you basically use all the time, like the shell, Vim, Git, etc., etc. At the same time, I would read Part 2 and do the exercises. That's really important. Remember, we discussed earlier in the video that you remember 10% of what you read, 20% of what you listen to, and 70% of, of the things you do exercises for. Then if you really wanted to, you could, after you've gone through this, you could write a blog post about what you've learned and teach others. So this is an excerpt of the 2020 machine learning roadmap. If you want the specifics for all of these plus some course recommendations, as I said, I would go through the blog post there. If you want to see it being done, you can check out my own AI master's degree, which— that's a fail, doesn't work, but I'll fix this link once I go back to this blog post. But just search Daniel Bourke AI master's degree. That's the curriculum I went through. But if I was starting again, I would follow something more similar to Jason Ben's How I Learned Web Development, Software Engineering, and ML. So this is a really great post to see an example of how you could create your own curriculum using online resources. One more thing I would add to the example curriculum/list of what to learn is there's a lot of resources actually out there for of what to learn, not so many on how to actually learn, which kind of perplexes me. So, I thought about this like when I'm teaching or when I'm studying something, it's like, okay, I know kind of what to teach, how should I teach that? Or if you know what to learn, how should you learn that? We've kind of discussed a point here is to have fun, treat learning as a dance, jump from concept to concept. But if you want something more like specifically tailored on how to learn, I'd go through Coursera's Learning How to Learn course. That's really good. It kind of teaches you different ways of thinking. But otherwise, just practice, practice, practice, figure it out. You'll eventually create your own style. Whoo! We've covered a lot. And now, I mean, no matter what I've said here, at the end of the day, it's up to you to keep learning, keep learning, keep learning. This is the most important thing. And the number one rule: keep learning. You remember my catchphrase from last night? Yes. What is it? Keep learning. Keep learning. And another one I'm going to add here, it's to keep creating. So, remember the balance. As much as you learn, use that to create as well. Don't forget, any links that you need will be in the description below. If you have any questions, leave a comment or just reach out to me. My email is daniel@mrdbourke.com. If you've got any feedback or advice, again, this is all biased towards my own experience in the field so far. But it's been a lot of fun, and who knows, maybe we'll do this again in the next 3 years. I can't wait to see what happens then. I'll see you next time.