Ask a Machine Learning Engineer Anything (live) | May 2019

All right, all right, what's happening? Welcome to the channel. If you're new here, I do a— oh wait, we're getting some updates here. We're just going to wait till we go up and we are live. Beautiful. Okay, hello everyone, my name is Daniel Bourke and I am a machine learning engineer slash online content creator based in Brisbane. Uh, the purpose of this live stream is to answer any of your questions relating to machine learning, to technology, health, life. I don't have all the answers, but I can share with you what I've learned. Um, if you're new to the channel, thank you so much for joining. And, uh, it's nice to meet you all. So what I'm going to do, how we're going to structure this is probably for the next hour or so, I'll run through some of the questions I most often get or ever received recently via email or any other sort of platform. And then I'll, I'll get to the chat. I'm going to, I'm going to try and get through as many questions as I can. So feel free to post in the chat whatever you want and we'll have some fun. Alright. Happy Sunday everyone. Oh, and, uh, first of all, I haven't posted a video since last month's livestream just because I've been working out some things to do with health and to do with, with other stuff. So in terms of, I kind of got diagnosed with Ross River virus, which is, uh, like a mosquito bites you and then you just have these really sore joints and chronic fatigue. So, to deal with that, the prescription was rest. So, that's kind of what I've been doing for the last month is just getting plenty of sleep, lots of nutri— lots of good food, and essentially, I don't want to say taking it easy because I like doing hard things, but taking it easy for my book. So, that's why there hasn't been a video for the last month or so, but don't worry, I'm starting to get better, starting to feel 100%. What's dangerous is when the brain thinks you're 100% but you're not fully ready. So, taking care of health first so I can create as best I can, make work that brings value to everyone. That's why there hasn't been a video for the last month. Alright, let's dive into it. Question and answer time. Feel free to ask anything and I will do my best to answer it. What's up y'all? How we doing? How are we? Seven, Yash. Andre Pratik. New hairstyle? Yes, I did get a haircut, Yash. I've got a mohawk. Check it out. What do you think? Shaved the back and sides, got a bit of length on the back here, and I've got a scratch. This is from jiu-jitsu, and I've got another one on my eye. Had a jiu-jitsu seminar yesterday, so that was really fun. All right, let's, let's jump into some question and answer. So Seven says, should I buy a desktop with GPU or laptop to begin machine learning? You only have one choice and it totally confused me. Help me. Yeah, you're right, Seven. When I first, um, when I first started, I was like, wow, I can't use my computer to do deep learning and machine learning. And I would say, here's what I personally do. I don't even use my laptop doesn't have a GPU, right? My laptop just has a CPU. I use cloud computing services to, to, I've rented GPU off Google, off Amazon, off Paperspace, off somewhere, wherever's cheapest or has enough power that I can use. So do you need an inbuilt GPU to your desktop? Not necessarily, but if you don't have access to cloud services, it may be handy to, to have GPU on your, your desktop. Now generally desktops have more powerful GPUs than laptops. It all really depends on what kind of work style you want to do. For me personally, I like having a laptop that I can move around and then I just connect to the cloud and rent a GPU off there. As for trade-offs, that's probably the biggest one, right? It's like if you have a laptop, the GPU might not be as powerful as something you can rent on Google Cloud, or in a desktop. All right, thank you all for joining. Good to see you all. If you're new to the channel, again, welcome. We've had a little boost in the last month or so, um, so yeah, I really appreciate you guys, uh, joining in here. Okay, so we have— where we got— Aran says, hey bro, I'm one year bachelor degree in CS and I find the programming very hard while learning. Can you suggest to learn a fast way, please help. Alright, so Arun, a bachelor degree in computer science and you find the programming very hard. Arun, think of it this way, right? Programming is, it's a language, right? That makes sense, Daniel Bourke, yeah, I understand Python is a language, but so is English, right? And when you were young, let's say 3 years old, you were learning to speak English or whatever your native language is. And of course, 3-year-old you was having trouble. You couldn't pronounce words, you couldn't string a sentence together. You would try to communicate a message to someone, but they wouldn't understand. And so you would get frustrated. It's the same with programming. And this is what I have to remind myself. It's the same with math. So math is the language of nature, right? This is what you can do to sort of remind yourself when you're learning a new language. Of course it's going to be difficult, right? So when you're 3 years old learning a new language, you're practicing all the time. You're going, you're trying to say a word, you're trying to say a sentence, you're trying to point to the horse and go, that's a, that's a, that's a— but programming is the same thing, right? You're trying to write an app or you're trying to write a function, but you don't really know all the ins and outs. You sort of know but you can't really write it fluently. That's how I— that's how what I tell myself when I'm struggling to learn programming or a new concept is that it's just a language, and what you have to do is continually use that language, continually practice it. And so even after 1 year, imagine how well, how good you were speaking English after 1 year of speaking English. Not that great. I'm the same. I've only been coding for about 2 years, and My coding is probably the level of a 2-year-old being able to speak. I know some things, but I'm not as good as other people, like adults who've been speaking for 20 years. So it's the same thing. That's what I do to remind myself when I'm having trouble programming or learning another language like math or Chinese or Japanese or something. So, watashi no nihongo jyozu janai desu. My Japanese isn't very good. Andre says, if we're getting started with machine learning but we're familiar with Python, what do you recommend first? Coursera certifications, books, or YouTube videos on machine learning? Andre, that's a very hard question to, to answer. What I can tell you, what I did, I did Coursera and books and YouTube. Well, I've actually done all three of them, so What would be the best of those? First of all, I need to recommend this book. I've been reading it the past week or so. I think if you can see it, it's the 100-page machine learning book. If you know some Python, I really recommend checking this one out. There's also the Hands-On Machine Learning. I don't have a copy of it, but if you Google Hands-On Machine Learning with scikit-learn and TensorFlow, that's another really great resource if you, if you do know Python. And so alongside that, this is what I really wish I got into sooner, was building some of my own projects. And what do I mean by that? I mean something that isn't necessarily within coursework or within a YouTube video tutorial. It's, it's 100% okay to start with something like a YouTube video tutorial of doing something, of, of going through a machine learning problem, and then copying it word for word or line of code for line of code. But where the real learning happens is when you start to build on top of that, working your own sort of your own flavors, your own, your own details. That's where I definitely learned the most, is set something up for a foundation and then build on top of it. So if you know some Python, I would say maybe check out a book like this, right? This is a 100-page machine learning book. You can read it, you could read this in a day if you really wanted to. Or the hands-on book by, uh, I can't remember his name. Anyway, Hands-On Machine Learning with Scikit-Learn and TensorFlow. I really rate that one alongside potentially a Coursera course if you, if you've never done some machine learning code. Because Coursera would be really helpful for that. And then where you're going to really take your learning to the next level is by creating your own projects. And now don't, don't hold yourself back trying to build the next latest and greatest startup, right? Fair enough if you want to go for that, go for that. But start small and then see where it goes, see where it leads you. And then another really valuable piece is that even if the project doesn't work out, you have a story to share and you have something to show someone and you can go, hey, I've done that. That's this, this thing that I built, that was me. And this is what I did wrong and where I can improve next. All right. So that's, that's definitely something I wish I got into earlier is creating my own projects. Thanks for the question, Andre. Hello everyone. Welcome. I see a few more people have joined. Happy Sunday to you all. Happy May. It's a beautiful sunny day outside where I'm living, so I hope you're all having a great day. Whoa, we're getting some heaps of questions coming in. Christopher, AI engineer interview tips. Christopher, again, that'll vary depending on where you're interviewing for. I personally don't have much hands-on experience with AI engineering interviews. Because my interview, or the traditional ones where you see like coding on a whiteboard or something like that, or working on a problem, my interview was sitting across the table from, from someone who works at where I work now. And then we were just having a conversation for about an hour. I told him what I was into. I told him I'd been studying online. I showed him my online AI master's degree. And then he said, "Do you want to come in on Thursday?" I went in on Thursday and practiced using some pandas to manipulate datasets, and then the next Thursday I went, went in there, and the next Thursday after that I got hired. But if you're looking for engineer interviewing tips, I highly recommend the book Cracking the Coding Interview. I've seen that recommended a lot of places. I've only read a little bit of it, I never got to put the knowledge into practice, so I can't really give you firsthand experience from that. Prateek says, but thank you for the question, Christopher. Prateek says, how to start machine learning, like with some prerequisites or your recommendations? Now, Prateek, this again, these questions, guys, I want to just preface them, right? It's all, all I can ever give you is my own experience, right? And so the best skill any data scientist, machine learning engineer, or just a general life skill is self-reliance, right? So take everything I say with a grain of salt because it's going to be biased by my experience. So take the best advice from people you see, including myself, and you don't necessarily have to follow it. Just take it all, the best advice, combine it together, and then you can start to form your own decisions. That's how I operate and that's how I encourage you to operate. Now, how to start machine learning, Prateek? If you've never done machine learning, if you've never coded Python, I highly recommend getting into coding Python. I recommend from scratch, maybe the best place would be like something like DataCamp or some kind of book. Like this, it's really hard to, to sort of grasp, or how do I say this? I want to recommend the Fast.ai machine learning course, but it's probably good to have some Python experience before then. So where can you start if you're a complete beginner? Get some Python experience, right? A couple of months of that. DataCamp's probably a good place for that. This is, or YouTube, just go for some Python basic Python videos, get familiar with how to write a function in Python, how to use Pandas a little bit and whatnot, and then I'd start to really, really dive into machine learning problems as soon as you can. That's how I would go back and start again. That's how I'm recommending my brother, who's learning machine learning, he's in the other room now, learning data science, is Python, get some Python experience, and then start to specialize into machine learning problems using Python. And now, of course, I'm biased, right? We work— I work at Max Kelsen, I'm a machine learning engineer. We use nothing but Python, hence why I'm recommending Python. But for me, I learn best when I'm doing something practical, which is why I recommend don't let the theory and all that sort of stuff hold you back. Start working on problems as soon as you can. That way you'll start to, to get an idea of what a machine learning problem looks like, which is most important. Because at the moment society's kind of like, they have this, have, if we have a problem, let's just solve it with machine learning. But as you'll soon find out, machine learning isn't necessarily always the best option, right? So start diving into problems as soon as you can. But if you don't have any coding experience, start with Python or R or something like that. I'm going to be biased and say Python. Alright, so Rico says, why developer always use Mac? That's a good question. I used to use Windows when I was a kid, but then I got a job at Apple and I've used Macs ever since. I really like Macs. Hari says cloud computing is too costly for day-to-day research and experiments. Oh, Hari, it depends, really depends, right? So I can rent a $10,000 GPU for about $1 an hour. Now, if I'm not going to be using that for 10,000 hours, it's probably more cost-effective to just rent it on Google. Now, it really depends on what your use case scenario is. If we're— so we're thinking of buying some GPUs where I work because we use them a lot. But if you're really only just wanting to run an experiment for a couple of hours, it's probably more cost-effective to rent them. Douglas says, "Great hair, First." Thank you very much. "You want to learn robotics, how to start?" I don't know too much about robotics, to be honest, Douglas. I would say check out Udacity. I think they have a robotics Nanodegree. Udacity courses are world-class. And robotics is kind of one of those things where it's another thing where you have to really get practical to start learning. It's not like if you want to build a robot, you can't— it's hard to sort of really test that with using your computer. You need to see it in the real world. You can simulate as much as you want, but then until it actually gets into existence, you won't really know how it's going to run. So try that out. I don't know too much about robotics, so again, remember Take my advice with a grain of salt. Best to do your own research and that sort of thing. Jason. Hi guys from Korea. Jason, nice to meet you. Thanks for joining. Michael says, after Andrew Ng, what's next? Depends what Andrew Ng course you've done. If you've done the machine learning one, I'd recommend the deep learning one. And then there's another one that's just come out, which is Hands-On TensorFlow by Andrew Ng. Now, if you've done all those, I would definitely recommend trying to build something of your own, trying to build your own project or something like that. So that's, uh, that's, that's what I recommend. If you've done all of these courses, start trying to build something. Now we go here. Oh my gosh, I've lost my place in the chat. Here we go. Rafael, do I ever read ML papers on your free time? Yep. So of course there's way too many, way too many machine learning papers to keep up to date with them. But if something, if something relative to what I'm working on, in this case the last month has been natural language processing, I'll read the paper. If it's, if it's interesting enough to me, I'll check it out, see what sort of techniques they're using. I just recently read the UMLFit paper. That's a great one. We're doing a lot of stuff with language modeling and transfer learning in natural language at the moment, which I think is awesome. I'm probably going to release a video using transfer learning for NLP in, let's say, the next couple of weeks. Once I— I've got pretty good at it, so I'll share with you guys what I'm— how I'm doing it. It's actually using the fastai library, so really, really great library. Or the, the docs could be— no, sorry, the, the code base could be improved a little bit. But in terms of getting an AI project off the ground, fastai is an amazing library. And here we go. So what I'll probably get in these sort of questions— Rajendra asked how to start with data engineer. If you could be as specific as possible I'll try to answer these questions my best, but things like how to start, I mean, there's definitely hundreds of different ways, thousands of different ways you could get started. I can only ever share what I did, so, or recommend what I did or what I would do instead of what I did. So, things, questions like how to start, I'll probably breeze over them fairly quickly, but other like specific things I'll dive nice and deep into. So Shafaf Hussain, what is needed to create an AI startup? Okay, so this is something that I'm going to be trying to do within the next couple months. So stay tuned for that. No, but it's the same as any other business or same as any other startup, right? What is the purpose of creating a business? What is the purpose of building a startup? It's to solve some sort of problem. And now, what's really valuable is having a problem, a very specific problem that you're going to go after. And why? Because you might not necessarily end up solving that problem, but at least then you sort of you know what direction you're heading. Now, of course, if you want to do an AI startup, you'll need some knowledge about AI, you'll need some knowledge about the industry you're going to go into, you'll need some, some time, you'll need some money, right? But again, the most important is to have a problem that you're going to go after and then aside from problems, so probably as equal as, as the importance of having a specific problem, is either you having the skill or you being in a team that all have the skill to solve that problem. Does that make sense? So, have a specific problem and then have a team that can solve that. And then, sort of, which one's more important out of the two? I would argue that if you have a good enough team, you could almost attack any problem. So if it's just you on yourself, you be good at what you do. And if you're looking for a team, look for someone that is better than you at what you're not good at, right? So you can balance it out, or multiple people that are better than you at what you're not good at. So you can bring your skills, they can bring their skills. And you can work towards solving a problem. Suraj, alright, Suri Jan says, firstly, thank you, recently found you, but your stuff has kept me, helped me a lot. Oh, that's awesome. Thanks, man. I really appreciate that. What keeps me motivated? That's a really good question. I've kind of worked it out. I've actually kind of worked it out. In a nutshell, what keeps me motivated? I like to create things and I like to learn things. So, I believe there's some kind of Japanese word for it, but the purpose of my life is to create to learn and learn to create. So, it's like a circle, right? If you imagine learn here, create here, create to learn, learn to create, which is kind of why I have a YouTube channel about learning things and why I write articles and why I keep learning things and why I keep making things is because it's just fulfilling that circle, right? And then, of course, it's adding value to society, adding value to humanity. Like, to me, I've got enough money to pay for rent, I've got enough money to pay for food, that's enough for me. So, it's like, what's next? And then there's things like you could go for fame, you could go for power, which I don't really— they don't really interest me. I just like working on hard things and having a good time and then sharing my story with others. So there's that circle, learn to create, create to learn. And that's kind of what I'm doing and I'm really, really lucky to be able to do that. So what keeps me motivated is to just keep that circle going, right? That's my motivation. There we go. Seven says, "I have to make a software project this semester, it can be anything, website, app or anything else? Do you have any ideas? Oh, I got lots of ideas, Seven, but I would say that is a decision that's totally up to you. There's lots of ideas out there. The best thing you could probably do is write down a list and then start crossing out the ones that don't really appeal to you too much, and then narrow down the list of— maybe you start with 25 and then you're going to narrow it down to 10. And then narrow it down to 3, and then have a nap, and then narrow it down to 1, and then stick with that and ignore the rest of the list. Uh, Majik says— Majik, Majik, Majik says, hi, greetings from Poland. Nice to meet you, Majik. Thanks for joining in. And if you're just joining in, um, I'm doing a Q&A. Any questions you want about machine learning, about health, about life, hit me up. Anthony Arrington, what do I think of the Udacity courses? I got nothing but extreme value. So, Rafael doesn't agree, but yeah, I really enjoyed the Udacity courses. Some people have different opinions, but doing those courses fundamentally were one of the stepping stones for me getting my first role. So Udacity, Coursera, both great free resources. But again, courses only build foundational skills. What you can do is to get those foundational skills by completing some online courses and then use those skills to work on your own projects, and that's where the real learning will happen. But to get a foundation, you've got to start somewhere, right? And so, for me personally, I really liked having a structured course to follow, to learn more on my own time. I didn't mind paying for it because I'm a big fan of paying for things because that means you're committed to it, right? How many people say, "Yes, there's free resources out there," but then maybe read one free article or watch one free video and then let the rest of it go? So that's why I'm a big fan of paying for education. You invest in yourself, guys. That's always a good investment. And here we go. We got Rishrak. Rishrak asks, learning through MOOCs or self-learning through projects? Again, another one is use the courses as foundation, right? Build a foundation if you don't have it and then start working on your project. Something something that might not work. I wrote an article recently called "How to Work on Your Own Machine Learning Projects," and you can, you can read that and check it out. It's on my blog. So again, MOOCs for foundations, your own projects for building upon those foundations. Arun, how long do you usually take to complete a Kaggle project? Arun, it really depends, right? So Sometimes you could do it in a day, but will you get the best result? Maybe not. Sometimes it takes months, right? These competitions are open for about 3 months at a time. The timeline varies, and often you'll see people start from the very start and then work right up to the deadline. So it totally depends. Do you want to just complete it and just say that you've done it, you've made a submission, or do you want to be top of the leaderboard. One will take longer than the other. Ronnie says, how do you find people to work with on ML projects/Kaggle competitions? Ronnie, I don't really have a good answer for that because I— how do I find people? Well, I work with a machine learning team, so we, we kind of bounce off each other and work on our own projects through that. But in terms of finding, what would I do to find someone? If it was real life, I would go to meetups. So data science meetups, machine learning meetups, AI meetups, that sort of thing. Or online, I would check out the forums, check out Kaggle forums, check out Reddit, check out LinkedIn or something like that, and then get to know people. So there's a few ways you can do it in real life. Meetups online. You've got a bunch of different forum spaces that I'd get into and start messaging people and seeing if they're interested to join. Okay, we got— Priemang says, why don't you join MS program? I'm not sure what an MS program is. If you're talking about master's program, I've already done one and it's my own AI master's degree. You can find that on Google if you just search Daniel Bourke AI master's degree. I thought about going back to university, but I've had enough of university this decade. So maybe in the future, but probably not. Abhishek says, what would be the difference between math PhD data scientist and online self-taught data scientist? Well, it really— so I'm starting all these answers with it really depends. No, I'm going to start sharing my real opinion. Difference between math PhD. Okay, here we go. I've got a great example for you. I work with someone who is doing a data science master's degree and I'm self-taught online. So here is the difference. He is way better than me at the theory, right? So way better than me about the math and all that sort of stuff. But in terms of practical and applying data science to business problems, I'm way better than him. So, it really depends on what you want to do. Do you want to be extremely good at theory and extremely good at taking a problem to its sort of base roots? Or do you want to be able to create business value as soon as possible? So, I'm biased. I want to create business value as soon as possible and not necessarily write papers or do academic work. I want to create businesses. I want to create value for people. I'm not saying that, that writing papers doesn't create value, but that's just the way my brain works. I want to use the skills I've learned to apply them to a business problem, not necessarily a theoretical problem. So that's kind of where, where we differ. I'm the self-taught, more practical. He's the data science master's student. More theoretical, but we kind of help each other out by sharing our knowledge with each other. Here we go. So Rajendra asked how to start with data engineer. Currently I'm a DevOps engineer. If you want to become a data engineer, practice doing things that, that data engineers do, right? So building data science pipelines, building data engineer pipelines, on cloud services, on other kinds of platforms. But the best way to become the thing you want to be is to practice that thing. How much are you paid in your country as an ML engineer? It's somewhere between $50,000 and $100,000 Australian dollars. So that's how much I'm paid. Suraj, Suraj, Suraj. "Hey, good morning." Good morning to you too, Sarav. Thanks for joining in. Seven says, "Have you watched Avengers: Endgame?" Yes, I did, and I really enjoyed it. As you can probably tell, I've got the Hawkeye haircut going on. So, this was— this haircut was inspired by Hawkeye. I saw that, I'm like, "Yeah, that's cool. I'm going to get that haircut." Definitely Not Chris says, "It's hard to study at home. How do you do it so well?" Chris, again, lots of practice. I set a timer. I don't do anything but study during that timer. That's the best way I can. And if it's hard, I remind myself studying new things is always going to be hard, right? That's why it's a barrier to entry. Not everyone can do it, right? Because concentration is a skill that's been diminished in modern-day society because we have devices. I don't— My phone is in my drawer, right? That's probably one of the other best habits that I have is, is removing my phone from my sight because devices are sort of built these days to distract people and make you spend more time with them. So one of the best things I've done is just to delete all social media off my phone, remove all distractions and just do the work that's required of me. But again, I'm not perfect, right? I'm not a robot. I'm still, I'm still a person like you guys. I still have, like, well, not like you guys, like everyone else. I still have days where I'm like, I don't feel like studying, right? I'm just— but then I remind myself, what's the alternative? The alternative is I could not do the work. I already know what happens then. I don't achieve the things I want to achieve. Alright, so I write down a list of things that I need to do for the day. Today I've got livestream. Livestream is one of those things, and then I just work through the list. And if I don't complete the list, well then, that's life, right? I do my best. And so we got— where are we up to? Oh, we're going down here. Okay, so we got Chetan Palmer asks, what IDE do I use? I use Jupyter, nothing but Jupyter, or a bit of VS Code, right? VS Code. I never used PyCharm before. And here we go, what are you spending your time to learn now from Nexusforce? And, uh, A lot of things, right? At the moment, it's, it's been PyTorch and NLP. They're the two main things I've been learning. Otherwise, a lot of, a lot of marketing, a lot of how things to— what's it called— admin things on how to start a business, all this sort of stuff. Whether I should incorporate what I'm doing on the side into a business, but more on that in the future. In terms of machine learning and data engineering and that sort of stuff, most of my time in the past month has been towards NLP, PyTorch, and deploying machine learning models. So like things like building Docker containers, hosting them on Kubernetes, making, making your model available as an API endpoint so a customer can access it, that sort of thing. So, well, I've kind of wrapped that up really quickly, but it's kind of There are 3 things: NLP, PyTorch, deploying machine learning models. Dongwon Kim says, how do employers recognize your online ML certificates or nanodegrees? Um, I'm not entirely sure how, because I've, I've never actually heard how someone recognizes it. But thing is, don't treat certificates as competency, right? Certificates are good to build the foundation. I'm going to sound like a broken record, but you still need to develop the skills, right? If you can, you can go through a course and get the certificate by just clicking through lectures and, and copying code to submit and all that sort of stuff, but that doesn't necessarily mean you have the skill. Where the skill will come from is by Creating your own projects, sharing those sort of things online. That's, that's where, in my opinion, what, what an employer will really be looking for. They'll be like, great, you've got the certificates. Have you done anything using those, or are you just collecting more certificates? So think about that when you're going for certificates. Really awesome that you're doing learning, but what are you doing with that knowledge that you're getting? Uh, Bassem says, if I'm building a startup, can I use image classification code online, or should I build everything from scratch? I understand TensorFlow a bit and improving every day. Bassem, you can— I, I don't think it would be wise these days to build any kind of image recognition model from scratch. Why? Hear me out. I think if you're building a startup You should definitely take advantage of the amazing computer vision resources that are online. The thing is, you need to figure out what application you're using those resources to solve, right? And how are you selling that to customers? Because in the end, you do not have a business unless you are selling, unless you have paying customers, right? People get confused, right? They Of course, it's great to solve all these problems, but at the end of the day, your business will not survive without paying customers. So what can you do? Well, maybe you're going to start by building an image classification application. I personally would never write a computer vision algorithm from scratch these days because of how good the open source resources are. So you could even use something like Google's AutoML Vision. For your app. I honestly think this is a viable way to start a startup. But what you have to do with that service is to market that, to sell it, to offer value to people in such a way that they will pay you for it, or investors will give you money to keep building it up. So I personally prefer having paying customers from the start. That's how I will build my business. Is trying to generate money straight away rather than sort of taking on investment and not creating a profit for years. So that's just my biased opinion. But in terms of building a model from scratch, even where I work, right, I've been like, when we get a new customer problem, a new client problem, it's very rare that we'll start building a model from scratch. We will use the open source models and pre-trained, pre-trained models to apply to a business problem first, and then if it's not performing how we want it to perform, then we go into custom models. So in terms of starting your startup, feel free to take, take whatever's online and use it as yours, right? As long as there's sort of the copyright is all taken care of. I'm assuming you want to use something maybe open source or something like that. Just make sure that you can use it for a business case. And then if you can find a problem to use that to solve, well, then there's the basis of your startup. And if you find that your product or offering is not performing how you want, that's when you could go into something custom. So by all means, I totally agree with, uh, with you. Well, I totally— I'm a de— I'm an advocate for using off-the-shelf tools to start something, and then as you progress, potentially switching to something custom. All right, we need to get some water. How's this going? Is this, uh, Give me some feedback in the chat. Are we in the direction you want to be going? Are you happy for me to just keep answering questions? Because that's what I'm probably going to do for the next 20 minutes, half an hour or so. We'll see how we go. But thank you all again for joining in. I really appreciate it. I really have fun with these. I hope you guys have fun too. And again, call me out if my answers are not up to scratch. I'd love your advice. I'd love your feedback on how to improve these for for the future. I mean, that's again, one of the best skills that a data scientist or machine learning engineer can have is a willingness to be wrong, right? Because you've got to run a lot of models. None of them will ever be 100% correct, right? But some are very useful. So be prepared to be wrong if you want to be a machine learning engineer. Drew Staples says, in what order did you learn the necessary machine learning libraries? What libraries and frameworks do you use at work? What order did I learn them? I believe I did Python, then TensorFlow, then Keras, and then PyTorch or scikit-learn. So you imagine all the most popular ones, I've learned them. Doesn't necessarily matter what order, in my opinion. Use the one that's best for your use case. Now, most, most recently I've been using pure Python and PyTorch. And fastai. So they're, they're the three that I'm working on at the moment, and that's because they're most suited to the problem that, that I'm working on right now or at work. And I'll probably continue to use PyTorch outside of work just because I'm really enjoying it, right? I'm really— I'm getting better at it. I'm really enjoying it. And fastai is a great— is written all in PyTorch, so it's probably, it's probably a great place to to keep going, to keep following. That's sort of a really important point, right, is to not get too distracted. Well, this is my weakness, I get distracted by shiny new things, right? You just saw that my thought just changed and I changed my direction of conversation. But pick one. Python is probably your foundation, and then PyTorch, TensorFlow, Keras. TensorFlow and Keras are kind of merging, uh, I've seen that with TensorFlow 2, so they're kind of integrating, right? So they're trying to make TensorFlow a bit more like Keras from what I've seen in TensorFlow 2. But really, Python, scikit-learn, and TensorFlow, Keras, or PyTorch will get you 99% of what you need. And fastai is built on top of PyTorch, so if you know PyTorch, you kind of know fastai. That was a very convoluted answer, but that's kind of where I'm at and what I'm using at the moment. Long story short, use the one most suited to your project or the one you're most familiar with. Jason James, hello, is machine learning only just glorified statistics? That's one way of describing it, yeah. Yep, you could describe it like that. It really depends on how you— I'm saying it depends too much, but that is one way of describing machine learning. You combine math, you combine stats, you combine compute power, you combine code, you get a lot of data, and you've got a great recipe for machine learning. Priful says, are you still practicing intermittent fasting? I started intermittent fasting after watching your video 3 months ago and it has been really good. I've seen significant changes in my body fat percentage. Yeah, Priful, I intermittent fast 2 Daily. So my current schedule is I don't eat anything before 12 PM most days, some days later, some days earlier, but I never— like, I get up and I never really have breakfast straight away. I either work on something or do some physical activity, have, have something to eat around 12 PM, and then I stop eating completely at 8 PM and never eat really any later than 8 PM ever. So I use the 8-hour window. It's really worked for me, and the foods I eat in that 8-hour window are whole, unprocessed foods, as in meat, vegetables, plants, fruits, nuts, oils— not seed oils, so avoid sunflower oil, all those other kind of seed oils— mostly olive oil. That's, that's probably the number one oil I eat. But yeah, I eat like a caveman basically. But yeah, I still intermittent fast every day. Rush, Rush Rook Ryan says, although I'm a computer science graduate, but having a transition into data science, I find hardest now is the lack of statistical knowledge. Usually I needed to have insight to the results. Rakush, Rakush, I'm not sure. What, uh, what the question is there. But again, it's always going to be hard to, to transition into any new field. So again, treat it as, treat it as a learning journey, and anything you're learning, you're investing in yourself. Hashit says, hello, I want to start as a data scientist. I have a bachelor degree in computer application. Any suggestions? I want to go down the networking path, so I was really confused. What would you suggest? Again, just start practicing on data science problems, right? So pick one, one resource. If you know some Python, try out Fast.ai's machine learning course and see how machine learning problems pan out, right? And then as soon as you've got some experience, try something like Kaggle, or try finding an open source data, data data resource. People get at me for saying data wrong. Data, data resource, and then build your own project on top of that. Oh wow, we went, uh, right to the bottom of the chat again. This chat like auto scrolls. How do I rate Udacity? Nanodegree. I personally found a lot of value out of it. It's one of the best online courses that I got. And yes, it is expensive compared to others, but I'm a big believer in paying for, for your education, right? If you can't, that's fair enough. You have to do what you can. But if you can, investing in your own education, it's like investing in your health, investing in your relationships. Investing in yourself, you will never go wrong. That's the most important skill you can develop as a person, is self-reliance. That doesn't mean you never ask for help. That just means that when it comes to shove, you can, you can rely on yourself to figure out a solution to the problems that you're facing. Harsh, how to get ideas for deep learning projects and how to release it? How to get ideas, man? Like, think of— have you— you've never had ideas? Is there something that pisses you off? Can you use deep learning to fix it? There's an idea. What else is there? Is there something that really motivates you? Can you use deep learning to make it better? There's a million ways to think of ideas. What is really important is to not— how do I say this— to not sort of put a— okay, your ideas, they can be up in the clouds, but at the end of the day, you still need to be grounded in practicality. So this is where it's really Really important to be able to think of incredible things, like for example Elon Musk thinks of going to Mars. But he knows that to get to Mars he has to get a good team of people, he has to know about physics, he has to know about different kinds of science, different kinds of engineering, right? So it's one thing to have this amazing idea, but it's very also important to have the practicality to back it up, because otherwise it'll just be this idea that's high in the sky. So the best, the best work comes from when passion— like, so Elon Musk's passion is to go to Mars meets practicality. So the practicality is he, to build his mission to go to Mars, his business SpaceX sells their services to launch satellites into the atmosphere. That, that one earns the money to keep researching on different space technology. And 2, let's them practice sending things into orbit, right? So that's a beautiful way that Elon Musk's outlandish idea is getting work towards through a business problem, right? So that's what you can do. So if you have an idea for some kind of wild idea, think about the technologies that you will require to get to that idea. Deep learning may be one of them. And then think about, okay, can I create a business to solve the problems using deep learning to get to this overarching goal, right? So that's just one, one example. Anthony, I'm going to start out my new career as a cloud engineer and see which direction I want to— AI, ML, DevOps. "I like how you're sharing your journey. Thank you for answering my question." Oh, thanks Anthony. Really appreciate that. Chetan says, "Yeah, I agree. Nobody asks for certificates or educational qualms. I fucking spent a lot of money and my time to get those degrees. They want to only discuss is my experience and skills." Yep, 100%, right? As I said, always skills is what you want to build. You do— why do you do a course? You're not doing it for the certificate, right? You're doing it for the skills. That's the main thing. Build skills, not a pile of certificates. Hi Daniel, I want to ask, which course did you do for computer vision for ML/DL? Did you opt for opencv.org, 3 course in CV? How is it? Please review Udacity CI or AI course. Udacity's— I did a lot of theirs for computer vision. Otherwise, there's another one called— what's it called— pyimagesearch.com. pyimagesearch.com is another great blog by Adrian. Adrian, let me check it out for you. pyimagesearch. This is a really great blog for computer vision. Pi Image Search. Here we go. There you go, piimagesearch.com. That's a really great blog for computer vision, so check that out. Hasar got— Daniel, my brother, I got my first job as a machine learning engineer for a startup based off Switzerland, and you're only 16. Yes, Hasar! See, that's a, that's a story, guys. That's, that's what I love to see is people getting after it. I mean, not necessarily because first of all, congrats to Hasarth, everyone congrats to Hasarth. Let's get some, let's get some emojis in here. What it means is that what that shows, 16 years old, right? And age, it doesn't really mean anything, but Hasarth, I've spoken to him for, I'd say, going on a year, right? And he has just been learning, been getting after it, been sharing his work online. That is a great example of how you can do that too. If you're building skills, you're sharing your work online, you're talking to people in the industry, you're adding value, right? That is how you can get a role in, in the data science or machine learning community, or you can potentially start to build some of your own business or something like that. That is the 3 main things that are going to be required for any role, for any business is, of course, skills. You have to have something that you can offer. 2, you have to be able to communicate those skills. 3, you have to know people in the industry that you're in. So those 3 things, you combine those 3 things, and that is the foundation. So congrats, Haasath, keep getting after it. I love seeing that. I love seeing that. All right, where we go? Um, what do we got? Um, Mani Nal asks, am I graduated in CS? No, I'm not. I've never studied computer science at university, only online. Um, what do we got here? I keep losing my place in the chat. Machine learning algorithms or a neural network deep learning, which is better? Really depends on the problem you're working on. Um, some things deep learning showing massive gains in a lot of things, but again, if you don't need deep learning Traditional machine learning is also very great. Is any degree— Sanket asks, is any degree required to become an AI engineer? No, I don't have a degree. It depends. Some workplaces may require you to have a degree, but a lot of them don't these days. If you have skills and you're able to communicate those skills, that is worth more than a degree in my eyes. So focus on skills, not certifications. Again, certifications may be required for some places, but in the end of the day, if you have skills and you can demonstrate those skills, that is what is most important. Harshit, you're doing awesome answering questions. Thank you for all the suggestions. Do you have a course anywhere from where I can learn and be somewhere near as good as you? Good work, buddy. Harshit, I don't personally have one myself. I'm planning on making many more videos in the future about trying to teach you the skills that I've learned as the past year as a machine learning engineer. If you want anything specific, let me, let me know. Send me a message wherever you want or leave a comment somewhere and I'll try my best to create a video for you or a blog post or something like that. But as for courses, I would Google Daniel Bourke AI master's degree. They're all the courses that I've, I've done or have done some of, and that's where the foundation of my skills came from. And Shafaf, when can I start reading and implementing ML research papers? Whenever you want. Yeah. Um, I wish I started earlier, to be honest. A lot of— if a paper is well written, you— and you read it a few times, the first time you probably don't get it, but if you read it a few times, you'll probably get to the point where you can kind of figure out what they're using, what they're doing, if it's well written. Sometimes you get bad ones that are just like, this is terribly, terribly communicated, which is not very often, but there are some like that. But I would say start reading. If you're reading ML papers, start trying to figure out, even if you're not coding it, I would highly suggest coding it, but write down like the concept or the framework of what it is. And if, and if you're stuck, email the authors, ask them, where's your code? Can you show me your code? If they can't, that's alright, right? Some of these things might be private, they might have to be private. But I would email and say, "Hey, I'm trying to implement your paper, but I can't. Is there any resources you suggest to help me, or is the code on GitHub or something like that?" Or, if you're really lucky, maybe you'll go, "Hey, I've tried to do it. It's not working. Can you review my code?" I'm sure if you do that enough times, I've had emails back from researchers saying, "Oh yeah, sure. Here you go, Daniel. Here's the GitHub repo." or direct me to another resource. So this is what it's about, guys. It's about being in this community, being communicative, trying to do things even though they might not work, right? So when can you start reading ML papers and implementing them? As, as soon as you decide you can, right? Your skills might not be there, but the effort could be, and then you do it enough times, you'll start to get better at it, right? Tara Prat, Tara Pat says, "Hey, Daniel, from Thailand." Nice to meet you. Welcome. Thank you for joining. Ameer Saad, "Where did I go to school?" I went to school about 5 minutes up the road from where I live at a school called St. Patrick's College. and I went to university at the University of Queensland, and I learned machine learning and AI at the School of the Internet. I still do. Jagal says, PyImageSearch is great for a specific task solving, but what if someone wants to use the heuristic of visual data for deep learning architecture? Needing CV for research. I would say, Jagal, get into CS2, 3, CS2, 3, 4, 1, N, maybe 2, 3, 4, 1, N, which is Stanford Computer Vision. So I'm Googling this right now for you. It's open source. This is probably the best resource for doing like research-based Stanford Computer Vision. I can't type and talk at the same time. Stanford Computer Vision. So CS231N. If you go just Google Stanford's computer vision, you'll find probably the best computer vision course on the internet. So that's for Jagal. Calvin Wu, what do you think of the rise of Bayesian methods in machine learning? I know there are a lot of people making it scalable, but have you made it work in practice? Calvin, I've— to be honest, I'm not the best person to be able to like talk to you about Bayesian methods, but for some reason I think in the last month I've kind of been inadvertently working on a Bayesian method. And what do I mean by that? Well, I've used Monte Carlo dropout at at test time, at evaluation time, to predict the uncertainty of a prediction. And I don't know if that is pure Bayesian, right? Because again, I'm not well-versed in the theory behind Bayesian methods. I'm more of a practical practitioner. So what I did, I'm going to explain it to you pretty quickly, and then you can tell me if it's Bayesian or not. So what I did is I can send you the paper as well. It's called— wait, let me just— if you Google Monte Carlo dropout, Yaren Gal, Monte Carlo dropout, Yaren Gal dropout as a Bayesian approximation, that's the paper, right? I'm going to link the paper in the chat. And so what essentially it was, was it was measuring model uncertainty by applying dropout at test time. And what does that mean? So usually when you make a prediction on the model, you don't apply dropout at test time, but then to, to kind of work out, uh, what's it called, Bayesian uncertainty, Bayesian approximation. So to work out how uncertain our model was about a certain prediction, we applied dropout at test time and then made 100 of the same predictions. and then of those predictions we took the variance, right? So if the model— think about it this way, I can explain it really well with a coin. I don't have a coin on me, but say you had a coin, right? A normal coin is heads and tails. You flip that 100 times, you're going to get heads 50 times, tails 50 times. Makes sense. Now treat that as a binary classification problem. What if you had a two-headed coin? So on the coin it has both heads. If you made a model to predict what that coin would land on 100 times, what would you expect your model to predict? Remember, it's a two-headed coin. So you would expect your model to predict heads 100 out of 100 times, right? Makes sense. So you flip the coin, heads. You flip the coin, heads. You flip the coin, heads. But what if your model started predicting tails for 20 of them and heads for 80 of them, right? If you could take the variance of those predictions, your variance is going to be higher than if it predicted heads for 100 out of 100 times. Why would, why would you get different predictions? Well, because you're using a slightly different model to make your predictions every time because you're applying dropout at test time. Now again, I'm summarizing this pretty, pretty broadly here, but that's essentially what I may have been working on. Now the takeaway— what I— well, sorry, may have— what I have been working on in the past week as an actual problem. So what our goal was, was to use dropout as a Bayesian approximation to figure out predictions where our model was most uncertain and so that we can pass those predictions where uncertainty is the highest to a human classifier rather than letting our model make the predictions automatically. Now, I'm going to make a video and write an article about this explaining it properly. Apologies if I haven't done it the right way, but essentially if you look up Yaron Gal, it's, it's implementing his work and also Uber have a blog post on Bayesian uncertainty. Uber model uncertainty, I think. Yes, engineering, engineering uncertainty estimation in neural networks. So this is from Uber. This is another blog post. It's really great. So again, I don't know if I'm fully explaining it well enough, but the— in our experience, adding that Monte Carlo dropout method for Bayesian approximation essentially increases your inference time by 100x because you're making 100 predictions instead of just one. And so is that related to Bayesian at all? I'm not sure, but Is it just a coincidence that the paper has Bayesian in its headline? I'm again not sure. I'm not well versed on the theory. I've only seen it, I've only applied it in a practical example. So I hope that answers your question in some sort of way. What do we got here? We'll answer 2 more questions and then it's time for me to wrap up. I've got a bit of a sore throat. I had a big day yesterday at a BJJ tournament, so I need to, I need to rest and eat up and eat some food. All right, so two more things. I've read a lot about Kaggle being useless for building a portfolio. It doesn't offer real-world experience. Praful, it— everywhere you read is going to have a different opinion, right? Again, what is the number one thing that you're trying to learn? It's skills, right? So people who are saying Kaggle is is not a great place to build portfolio or to build experience, are they offering an alternative? Right? Because their alternative will be get experience in a role. What if you don't have a role? Right? So you have to do something before that. So Kaggle, courses, you need— Right? It's people say get experience. It's like, okay, what experience? If you're not in an actual role, you're not going to have that actual experience. So you need to do the next best thing. Which is Kaggle, building your own projects, gaining skills, sharing your knowledge online. That is the next best thing, right? Apart from being in an actual role working on those actual things. So all these people who say, oh, it's not great for building portfolio, not great for resources, it's like, okay, what's— you got an alternative? If they're not offering an alternative, all right, well then their argument isn't that great, isn't that very valid, right? So just focus on learning skills. Doesn't really matter where you learn them from, to be honest. Um, Uran Developer, hey man, I just tuned in so I didn't know if you already answered this question, but I'm currently learning from Andrew Ng course. Sarah, what do you think would be a good follow-up for that course? Uh, do all of Andrew Ng's courses and then start working on your own projects. That's, that's my, um That's my advice for that one. All right, we want to— we want one more question and then we'll wrap it up. And Shafaf says, can startups who want AI, ML, AI in their products use Google Vision APIs or Amazon ML APIs? It's just a black box and market their product. Don't really need much of a technical ML background then. Shafaf, yes, of course. I— that's the whole reason Google and Amazon make those ML APIs, right? So if you want to bootstrap something, get it off the ground, get it working, I highly recommend like trying an API, right? Because Google and Amazon are experts, right, at what they do. So if you have a problem that you think one of their APIs can solve, and you can package that up into some kind of startup and and sell your service to other people, even if you are just using— well, I'm not saying just, right, because it makes it sound like not, not actually as good. Um, I, I totally agree. Like, Uber uses Google Maps API, right? Lyft uses Google Maps API. So don't let the fact that you're not building things from scratch hold you back. I actually think it's, it's better to start because Google— trust me, Google are going to be able to build a better computer vision algorithm than you are to in the beginning, right? That's just a fact. They're Google, right? So they're open, they're leaving their best— well, probably not their best stuff, but world-class APIs available for you and me to use. So start with something like that, and then if you find it's not suiting your problems, well, then maybe you want to look into building your own custom stuff. Alright, I lied. Maybe one more time. Do you recommend taking Andrew Ng's machine learning course in Octave? No, get Python, use Python. That's, that's what I think, and I'm biased, right, because I use Python. But we're going to wrap it up. Thank you so much for joining in. Thank you all so much for your patience and joining the channel. If you're new, I really welcome you here. Otherwise, if I haven't, if I haven't answered your question in the stream, I'm definitely going to be hosting more of these in the future. So, so stay tuned. In the meantime, if you want it answered earlier, send me a message on Telegram. You can go t.me/MrDBourke or my email address is daniel@mrdbourke.com. Whatever, whatever suits you. Otherwise, yeah. I keep saying otherwise, and it depends. Two things I say far too often. Be direct with your language, guys. Have a great day and enjoy the rest of May. We'll see you next month, and I promise there'll be more videos to come soon. Peace out. I've got to work out how to end this thing. Such a streaming rookie.