Data Engineering on Google Cloud Certification Progress | Learning Intelligence 48

Oh yeah, check that out. That was, that was very strange actually. My headphones died as, as I took them off, so good timing. All right, we've passed Part 4 of the Google Cloud ML Data Engineering certification. This one, the part we went through was all about how do you deploy a machine learning model, right? How do you get it into production? That, that is what I'm focusing on at the moment, is, is not, not so much that I want to do it now, actually working on it, we're doing it at Max Kelsen. Long story short, if you build a great machine learning model but it lives in a Jupyter Notebook, it's not necessarily helping anyone. What I personally want to do, and what, what probably a lot of you guys want to do as well, I hope, you should want to do this, is get your code out there into the world and into into production. What do I mean by into production? That, that could be anything, right? But it means, it means to, to get someone using it. So like, I don't have my iPhone on me because that's in my drawer because I've been studying. So when you use your iPhone and you use an app or something, chances are Facebook, Instagram, you know, the common ones, they all use machine learning. But how cool would it be to have one of your own out there that, that use machine learning to, to to do something incredible. Like for me, I've got an idea of using machine learning to help people learn about nutrition by taking photos of foods. So, I want to— me and my brother have worked on something similar, so that'll be a future project. But we've got a certificate on the way. By the time this video goes up, it'll probably be here. So, high five for the certificate. We'll get it back up. There we go. How shiny does that look? Haha! I'm gonna go take a break and then we're gonna get into part 5. Let's do it! There we go. Sup, brother? What's happening? Not much. What game is this? It's like a knockoff version of Super Smash, but for Xbox. You know how much I love movement. Been breaking up study recently with, with skipping. I find if I do like a Pomodoro or something, so like 25 minutes of study and come out outdoors— usually it's a bit sunny out, but it's, it's late in the afternoon now. It's— I don't have a watch on, it's like 4:30, 5 PM or something. 100 skips, and then that I find gets everything flowing, gets the brain going, because you sit down for too long and get a bit stagnant. So you move around get the board to jump in, and then back into it. Now, wonder if I can get 100 straight. I'll probably just jinx myself. You can! Woo! Little bit fast at the end. I think that's 120. Alright, heart rate's up. We'll go back in, we'll get into part 5. Do a couple Pomodoros. I've got a few left for today. I think we'll do 8, 8 or 10. We'll see how today goes. You see, the thing about machine learning models, as you probably know, they love data. So the Titanic problem that we did had about 1,000 rows. Now, what if your dataset has close to 1 billion rows? Rows, so a million times more. Well, that with pandas, right, that's what we use for the Titanic one, or that's what gets commonly used with smaller datasets. The DataFrame or the table you work with stores in memory, so right, the RAM of your computer. Now, storing a billion rows in RAM might not be the best option. So, What can you do? That's when you need distributed systems. That's when you need cloud platforms. That's when you need giant machines. I'm still puffed from those skips. If you haven't got a skipping rope, get a skipping rope. And that point there, the more data thing was, it was a simple one, right? But that's what I like about this course is that it broke it into, there was 3 steps to improve machine learning. I've got my little notepad here. And so number 1 was more data. That's what the example I just had. What if you had a billion rows of data, not just, 1,000 like our Titanic little baby sample. Number 2 is feature engineering. So we didn't do any feature engineering in the Titanic example, but what is feature engineering? I think the camera is slightly moving. That's right. I'll just hold it. Feature engineering. How could you combine 2 features into 1? And what do I mean by feature? Imagine a column. A column is a feature. So imagine Titanic. It's an easy example. So you have a column for age, cabin, class. What if you combined age and class. So you had people who are, let's say, over or under 50, right? This is a rudimentary example, over or under 50 and in first class or second class. So you've combined the age column with the class column. Now that's just one, one example, but that's another way to improve your machine learning models is if, if the features you get don't sort of give you the best results, you can create your own. And then the final one is hyperparameter tuning. So more data, feature engineering, and then hyperparameter tuning. So if you do build a model and you want to test a whole bunch of different settings on that model, if you imagine, if you imagine you've got a different combination of, let's say, learning rate, let's say optimization rate, let's say the number of batch size that you use. What if you wanted to test a whole bunch of different combinations? And that is where Cloud AutoML comes in, You can do like Hyperopt, which is a library for hyperparameter tuning. I mentioned that in the Titanic video. Or grid search that searches a whole bunch of different parameters. And now that's enough talking. That's 3 ways to, to, to blow out your machine learning— well, to improve your machine learning models, not blow them out. That's what we want to do, improve. More skipping. Oh yeah, I think it's starting to rain. We better make these quick. Skipping in the rain. I think there's a song about that. Skipping and singing in the rain. You keeping count? Messed myself up. And again. Whoo! Alright, one more Pomodoro. You notice how I stuffed up? And I lost concentration. Similar to when you study, focus on the one thing. That's why I set the timer and don't do anything else except— just got really dark, really dark. Back into the light. And don't do anything except focus. So we're going to finish off this part 5, and then I think that'll be it for today. I forgot to turn the microphone on. So, there's no audio, but I'm going to just essentially sum that up in a lot quicker and better fashion. Of course, Course 5 I've been doing on the Google Cloud Platform Data Engineering certification, a mouthful, has been on streaming data. Now you have your regular data, which may be, say it's data just in a box, doesn't really change, it stays in that box. Streaming data is like stuff going through a river. So imagine YouTube or Spotify or whatever or something, or Google, they always have data flowing through. It's not just a box full of data, it's always getting updated like a river, right? It's like you say, you put your hand in the river, you put it in again, again, it's never the same river. Pretty sure that's a saying somewhere. Anyway, what's important to know this? Well, it's as data grows and grows and grows and sensors become more prevalent, it's a problem that, that has to be taken, taken on, right? You're not always going to have just a, a box of data to work on to build great models off. You're going to have constant input of data. Now all this stuff I've been, I've been learning on here, like all different services like Dataflow and and Pub/Sub and Google BigQuery and Bigtable, they're all new to me, right? So I'm kind of, kind of like, um, still like, what do I do here? So if someone was to ask me, could I build an end-to-end pipeline with it? Not yet, but I'd probably know where to look. I'd know where to find out. And that, that segues into a little question I got. I'll answer, I'll answer this to wrap up the video. I think that's enough studying for today. Someone asked, reached out to me, "When do things, when do you start to understand things?" And he's been coding Python for a couple of months, but struggling a bit with some of the concepts. Well, I'll come straight with that. I'm in the same boat, right? I've been coding for 18 months, 2 years, or let's just say 18 months, whatever. The timeframe doesn't really matter because I've spoken to people who have coded for 10 years and still look up basic things. You'll find blog posts of people who are experts in their field, some of the highest quality developers, and they say, they'll say, the honest ones, the honest ones will say they look stuff up every day. They're on Stack Overflow trying to piece together things. And that's what I do too. That's where I'm at with this Google Cloud Platform Services because it's still all new. I mean, there's so much out there, right? There's always a new framework, a new way of doing things. What I think is important is to get some foundation skills and never lose that curiosity or that eagerness to try and figure something out, right? That's the most important part because when you first start out, it's going to be frustrating. I go through it, I go through it every day. You'll have failures. The other day I failed at committing some GitHub code and— or some code to GitHub and lost 500 lines of Python code. "In the end, it's gone!" So yeah, sorry Michael, that was my colleague we were working on that with. So we We all have failures, we all have struggles, but we learn and we figure out how to, how to look for the right questions, not necessarily the right answer. Look for the right questions of where to go next. So if you're struggling with coding or with data science or any of these concepts, trust me, I'm in the same boat. That's why, that's why I'm here studying every day. I'm studying along with you, trying the best we can to learn. But I'm going to go do a workout in the backyard and have some dinner, and then we'll get back into this tomorrow. Keep learning, team. See you soon. See ya. I tried to say see you next month and see you soon at the same time. The words just crossed over. I need, I need Google, uh, Dataflow. I need a lot of streaming data coming through my head. My brain operates too fast for my mouth. Keep learning, team. You know what I say? Keep learning, keep moving too. See you next week.