Applied Data Science with Python on Coursera Progress | Learning Intelligence 45

Woo!

How good is sunlight? Morning everyone, or evening, afternoon, wherever you're from. It's Saturday and I'm really excited. You know why? 'Cause there's a little package upstairs and I think I know who it's from. So let's open it, shall we? Hey Poppy! Now this got delivered the other day and I read the top. Let's move you around here. You know who it's from? Coursera. So you probably can't see, right in there. Let's check it out, what's in here. Oh yeah! Now, let's substitute this, even though it's summer outside and this is some real summery attire, let's put this on. Hey, don't look! It's supposed to be a real quick transition to getting this shirt off. There we go. You know what happened? A couple weeks ago, someone from Coursera reached out to me and like, "We want to sponsor some of your videos." And I'm like, "For sure!" Because you know why? I love partnering. I love teaming up with epic people. There's nothing, nothing more that gets me excited than working on something with an amazing team, and Coursera are definitely an amazing team. You know why? Because They partnered with industry leaders like Google, IBM, University of Michigan, which is the course we're going to jump into today. There's over 35 million people on the platform. That's more than my whole country. And it's 100% online, which allows people like you and me to study in our bedrooms. I spend a lot of time right here. Alright, so jump on board. I'm excited. The next 3 videos Coursera sponsoring. Today we're jumping into the Applied Data Science course by University of Michigan. Applied Data Science with Python, actually. That's a really important one. And we're up to, we're up to part 4, but I'll show you. That's much better than just talking about it. Now, first things first, before we get in and start, I do this every morning. I get a game plan going. So this is my personal assistant right here. Right now it's not being very helpful because it doesn't have much on it. But we've got a trusty Coursera pen as well. We're going to write that down. So then we have some structure for the day. Today's main focus is of course the, the Applied Data Science program with, with the University of Michigan on Coursera with Python. I keep forgetting, with Python, with, with, with, with Python. We're going to focus on that, do 10 Pomodoros. And if you're not sure what a Pomodoro is, it's 25 minutes of unbroken focus on a single thing. So, 10 times 25 minutes, what's that? 250 minutes ends up in about 4— no, 4 hours, just over 4 hours of solid work without any distractions. So, that's our goal for today. I really, really, really, really, really, really, really want to complete the 4th assignment in part 1, which is what I'm up to now. But I'm not sure if I can do that within that time frame. Of course, we're going to be trying. But I know I can dedicate enough time. So we'll get the game plan sorted and we'll see what, what we got studying for today. All right, we're sorted. Nice and simple. Keep the list nice and short so we can dedicate our focus to the things that matter. And what do we got? We got write because I write every day. We got Coursera Applied Data Science with Python times 10 Pomodoros. We're going to check them off as we go. And then BJJ training to break it all up. So without any further ado, let's jump into this course. All right, so we're gonna get our first timer going. We've got a little Coursera Python course. Now this app is called Be Focused, by the way. It's on Mac. It's a great little app to get Pomodoro timers going, or you can adjust this time, whatever you want it to be. I'm not sure if it's on Windows or anything like that, but it's definitely on Mac. And so I'm gonna hit start. So our goal today is 10. It already knows. And so for the first one, it's going to be, it's going to be exploring the videos again. I know it says I've got them ticked here. This is the notebook that, that is in this first one, which is, which is all code examples that, that go along with the lectures. I really like that this course does that, uh, so I can see it and run it as, as I'm working on it. But this, this section is on statistical analysis in Python and a project Now, as I said, I really want to get that project done, so the first part will be reiterating through the videos, checking back over my notes, and then we're going to tackle that assignment. Yeah! All right, I'll see you in about 25 minutes. All right, first Pomodoro done. That was good. That was good going back through the videos and taking some more notes. I'm probably going to finish off this video once more to go back through it because it's such an important thing, hypothesis testing, A/B testing. There's a little bit of statistics knowledge that I could probably get up to scratch on, but I'm still digging at that. And I found a really— he gave a really good definition of standard deviation. It's a measure of how different each item in our sample is from the mean. And I mean, I still need to investigate this myself to further understand it, but so important I'm finding that statistics knowledge is really fundamental when trying to figure out whether your result, whether you're, I still got my headphones on, whether your result is keepable, right? It's all keepable or significant, right? Or does it just occur due to random chance? I'm going to finish this video The timer is saying to take a break, so we're going to do that. We'll finish the video, 5-minute break. We'll probably go outside and walk around for a little bit and then get back into it. All right, break time. I love getting outside so much. I even do it at work when I'm concentrated and focused on a problem and then I need to take a break or something, I'll just go for a walk, get outside, wander, not even really think about anything. Sometimes think about what I just learned or what I've been working on, but most of the time it's just really to clear up the excess cobwebs in the brain and and get moving.

Stretch it out.

All right, I got training in about 40 minutes, so we've probably got time for, for one more Pomodoro before I go to training. So 25 minutes, we'll check out the assignment. Go through it and get like a framework of what I have to do for each question. So when I come back after training, probably have a little nap because I'll be tired because it's jiu-jitsu, right? And then set ourselves up for the afternoon. 8 more pomodoros to go. All right, 2 pomodoros done. Got 2 minutes to get to training, which I'll be a little bit late for, but that's all right. It seems that the first question, I'm already stumped on this assignment, but that's all right. We'll get back to it. I've already jumped into the forums, as you can see here, checking out what to do. Stack Overflow, we've got that going already as well. So we will figure it out. That's, that's what we do. That's what we engineers do. Here we go. This is what I need to do. Get a list of university towns. So I need to transfer this data frame into being a column of state and town. So Alabama is the state and these are the towns within Alabama, and there's going to be more states further on as we go through the data frame. Uh, like Alaska, there's a state, so I need to put that in the state column and and then the one below it over here in the town column. Not entirely sure how to do that, but we will figure it out. We'll be back in, back in about an hour or so. All right, I made it home. Saturdays are long sessions at BJJ. I got a, a little bit of a blood nose over here. Yeah, that's right. So we're gonna have some food, recover, probably a little sleep, and then we're gonna get back into studying. We have 8 Pomodoros to go. I'm actually really excited to get back in and solve this problem.

We're gonna do it.

All right, nap's done, food's done. Let's get back into study. We're up to Pomodoro 3.

All right, Pomodoro 3, we're done.

But I'm on a roll, so we're gonna skip this break. We're gonna jump straight into Pomodoro 4. We're getting this function solved. It's turning out to be, turning out to be a bit more complicated than I thought, but nonetheless, it's always good to start an assignment on a hard question. They'll probably get harder from here, who knows. By the way, listening to some some white noise as study music to, I don't know, just have something playing in the background, nothing too intense. See, 417 Hz healing music, let go of mental blockages. That's exactly what we want. Alright, we passed the function I was, I was stuck on for most of today. I'm really excited. So I'll run you through, through what happened. So just, I'm going to pass it over to the screen. Alright, so there we go. Function 1 passed correctly. 0.1 points were awarded.

Oh yeah.

Alright, we go into the assignment. So we had— here's the function here, but I'll show you a step-by-step little, uh, little play-by-play. If we go over here, this is how the DataFrame came in. We have State 1 with Edit, and I'll show you over here there's a, there's a little, uh, checkpoint function that I found on the forums to help out. So if we read in the raw DataFrame It comes in like this. So we have a state with edit following it, and then we have some, some regions within that state, but it has some, some yuckiness on the end in the brackets here. And you come down, we have another state here, Alaska, and we have Arizona. And so all the states have this one thing common, that they have edit there. So the data frame we want looks something like this. So it has state and and then region name. State is column 0, region name is column 1. So what I did first was a little— I could probably— if you, if you have a better way of doing what I've just done here, please leave a comment below because this seems really hacky to me. Essentially what I wanted to do was I couldn't find a way to drop this column header into row 0. So what I did was I saved the column header to a list and then treated row index -1 as Alabama edit, and then I added 1 to the index so it turns from, from -1 into 0, and then I sorted by index so it reshuffles the DataFrame to sort 0 through to— let me just show you what this looks like actually. 0 through to, um, 0 through to 517, I believe. So see here, if I just did that, but if I, if I go to the bottom of the DataFrame, I've got a -1 there. So that's when I need to +1 to all of, all of my indexes. So then I've got a 0 down the bottom, and now I need to sort the index, and that brings it up to the top. All right, so that's how we did that in a few steps. Uh, actually, that's screwed it up now, so I need to rerun this. Boom! And that's how we get Alabama edit up the top. And then to create the state data column, I took the Alabama edit column from our DataFrame, which is this column. I did a string extract on it to extract everything This is, this is a group, a regex group. The dot means any character and the star means every single one. So it's going to pick any of these characters before edit, which is in here. So before edit, and then it's not going to expand. So it's not going to, it's not going to fill it out with, with anything else. So you want that to set that to false and then it's going to F fill. So what that means is that forward fills the column until it finds another match. So you can't really see it here, but if we increase that— oh no, please run. There we go. It just hits Alaska and then Arizona. So F fill on the, on the backend here means it's going to continually fill by row until it finds another match and then going to do the same thing over again. So that's how we get, that's how we get Alaska and then that's how we get Arizona. It's going to continually fill that for the whole, whole row, uh, whole DataFrame, sorry. Now to create the region name DataFrame, we pull out the, we say df_region_name equals df again, we're working with this column here to build the region name. Equals, sorry, equals df_alabama_edit, this one here. We pulled the string out of that and then we replace everything in that string after the, that comes within the brackets. So this is what this is here. So we use this to, actually I've got the regex showing you here. So this is an example. So we have Auburn University and we have whitespace. So the first piece of whitespace is the space between Auburn and this bracket. So that's this little whitespace here. And then we're going to replace everything after that whitespace, including the brackets, the things within the brackets, with this empty string here. So this little portion of the regex is just saying everywhere after the first piece of whitespace, which is this s+ first piece of whitespace, replace it with an empty string. Okay, and then that is going to give us our region name column, which is boom, we got there. So now we've turned Auburn into Auburn, Florence, University of North Alabama into Florence. And now, obviously, we don't want region names with, with the same town names after in them, right? So we're going to have to get rid of these, Akronasis, uh, Arkanasis or whatever. So that's when we go dataframe equals dataframe not. So this little symbol here is not dataframe Alabama edit. So this column, we're using this column as our reference column. If that string contains edit, right? So if it contains edit, So that'll be this one and that'll be this one, right? We want it to not equal those, so remove those from there. Then we're going to reset the index because if we remove this, right, we've removed number 9 from our DataFrame. So it's going to go 8, 10. But what we want to do is reset the index and drop=True means it does it, does it in place. So it'll go 8, 9, because we would remove this, right? And then we're going to drop this column, Alabama edit. So dropping that column on axis 1. So axis 1 is, is this one here, this, this column. So this is axis 1, axis 2. And then finally, we drop— means essentially delete— delete this column. Now we've got state and region name. And then we submit our assignment and we pass function 1. All right, now we're on to function 2 for the 5th Pomodoro. All right, I'm going to go for a little walk and then come back and we're going to get straight back into it. That was a little bit of a long-winded explanation of it. What I'll do in the future actually is probably, probably get a DataFrame similar to this and, uh, make a full dedicated video on it on how to go from top to bottom for every function. But that was just a little insight on how I did the first function with a lot of research, a lot of Stack Overflow, and a lot of trawling through the forums for some help.

Number 5! I won't do that again.

5 down, we're going straight into 6, bud. I'm on a roll. You know when you get in a roll, you get in the zone, you just want to keep going?

That's what we're doing.

See you in another 25. These brain waves, man. These brain waves. Whoa, is it dark in here. So, we are up to 8 pomodoros done. We've got 2 more left for the day and I'm, I'm stuck on question 2, right? I've passed question 1. Now question 2 is taking me a lot longer than I thought as well. We've spent, I think it's 4 Pomodoros now, so what's that, almost 2 hours of dedicated work. I mean, just look at the amount of tabs I have open, all these different forum posts, and the forum posts are actually very helpful. But I've got a little bit of a note here of what to do next. Next, fix up question 2, check logic and remake my own if needed. And I reminded to myself, it's okay to go slow. And I think that's a really important thing to take away is that with this, with this stuff, with programming, with machine learning, with data science, with anything, right? It's, it's a good reminder that it's okay if you, if you don't get a concept. And it's because hard things take time, right? And it might not click as fast as, as with you or especially with me. I mean, it's taking me a lot longer than what I thought to do this assignment. And it's only, it's only the first stage of this specialization. So little reminder, don't be afraid to go slow, take your time, learn the concepts. I mean, this stuff isn't going away anytime soon. So plenty of practice. I'm gonna go have some dinner and then we'll come back, finish our 2 Pomodoros. That'll be 10 for the day. Hopefully we get to question 2 out. Coming in? We got a puppy in the house! Alright, dinner's done. Drinking one of my favorite drinks. This is sparkling water with berries in it. I don't have soft drink very often, but, well, at all really, but that is a good replacement. Look, we got support here for our final 2 pomodoros for the night. It's about, I think it's What time is it? 8:22 PM. Boom. So we'll get these two done. Hopefully we'll finish question 2 and—

Hi!

You excited too? Let's get into it. Number 9. Gotta hide that one. There we go.

9.

Alright, I'm on to something. We've got one more pórronora left. I got the answer right, but I want to tidy up my code. I'll show you, look how many submissions I've done. If we go in here, Simon4, I've done about 7 submissions in the last 10-15 minutes. Look, 0, 10, 0, 10, 20, 20. These are all good, but my code is, I'm sort of jackhammering the answer, if you know what I mean, just hacking it. Rather than getting it the code to do it. But we've got one more Pomodoro. We're going to tidy up this code and make sure it's systematically working, make sure it's beautiful and it works pretty. So let's, uh, let's step it up a notch. Let's get some upbeat music going for this final Pomodoro. Come on, you can do better than that. Let's, let's try again. Do you know how hard it is to find music? Yeah, I know. You can do better. Alright, I'll give you another shot. 3, 2, 1.

Woo!

10 pomodoros done. We passed question 2. We did it. I moved on to question 3. But that's, that's going to be for another time. Let's check it out. We go into here. Boom! 20 out of 100. We need to get 80 out of 100 points to pass. But nonetheless, I've left myself some notes for next time. Here we go. Actually, a bunch of notes. When I don't, when I don't really understand something, I write it down. But look what we can do. We can officially check off the list. Boom! What a day! Training, 10 Pomodoros of study. I've got more fingers here than I should have. Now I'm going to do some writing, probably going to watch a movie and then jump into bed. It's 9:20 PM. It's a good day of learning. Probably get back into question 3 tomorrow or sometime next week. But otherwise, big shout out to Coursera. Not sure if that's back to front, maybe not, maybe it is, but anyway, thank you to them for sponsoring this video and the next two that are going to come up. We've got a— probably another one on this, on this course here, and then another one on the Google Cloud Platform. That's, that's another course that I still got to go through. And I mean, I'm just going through Coursera's catalog here and you can study practically anything on here. Arts and Humanities, Business, Communi— Computer Science, Data science, that's us, and computer science, of course. I want to probably get into some life sciences like biology and genomics and stuff in the future. And the best part about it is they're partnered with world-class universities like Johns Hopkins. And I mean, the beautiful— another beautiful thing is there's probably people all around the world like you and me doing what I'm doing right now because there's so many people online. You can do it anywhere. Cheers, Coursera. Otherwise, you know what to do, team. Keep moving, keep learning. We'll see you next week. Peace. I knew it! He didn't mention it, but if you want to sign up, there's a link in the description. He forgets all the time. No, I didn't. I was going to do a little note at the end. Well, where is it? Wait, how are you in the future? Anyway, yeah, as future Dan said, the link's in the description if you want to sign up to the Applied Data Science with Python course. It's one of the best ones I've done, and it's— manipulating data is something that's directly related to what we've been working on at Max Kelsen. I mean, we get data in, it's not always in the best form, so you have to transform it. And I mean, this course on Coursera by the University of Michigan, it's hard, don't get me wrong, it's hard. As you saw, I spent all day trying to get through those two questions, but the The, the skill of manipulating data and cleaning it up and transforming it into a way that you can use it better, very valuable in any, any field you go into relating to data science or machine learning. Give it a try. All right, this time the video is over for real.