Logistic Regression in Python | Databytes | Machine Learning in Python

Hi, welcome to this supervised machine learning tutorial on how to do a logistic regression in Python. Click the link in the video description to open this in Data GAMP workspace so you can follow along. Before we do some code, let's go through with a little bit of a theory. First question might ask is when can I actually use logistic regression? And there are two main criteria. The first thing is that the response variable, the thing you're trying to predict that needs to be binary like true or false or categorical. Secondly, the observations need to be independent. We're going to be using Scikit Learn here, pretty standard. I also quite like status models for doing logistic regression. But the fact is logistic regression is such a common technique that every single machine learning framework in Python allows you to do this. So you can use Pycarat, TensorFlow, Carrots, Pytorch or whatever you fancy. We're going to be doing a case study today on predicting organic product purchases. Bit of a tongue twister. Now, the background to this data set is that a supermarket was trying to incentivize people to buy organic products. So it gave coupons to its loyalty program members and then checked did they actually buy any organic products after they got the coupon? I got this data set from Kaggle. The original problem, this is a bit murky and we've used a subset here just, just to make it a little bit cleaner to analyze. The data set is in CSV format. So we're going to start off using Pandas to import it and then do a little bit of data manipulation and we're going to switch to Scikit Learn for the modeling. So let's write some code. We'll start by importing Pandas using the usual alias PD and then we're going to import some individual functions from Scikit Learn. So I am just going to do a bit of copy and pastex. We're going to import from three different sub modules. So the first thing is from the model selection sub module we're going to import train test split. That's going to allow us to split our data set data set up into training and testing sets. And then from the linear model sub module we're going to import logistic regression and then finally to analyze model performance. So from the metric submodel module we're going to import confusion matrix and classification report. Let's run those. So now we've got our functions available and data in this CSV file called organics CSV so we can do PD readcsv We're going to pass it at the path to the file. So organics CSV. I'm going to copy and paste, so don't do any typos. And in this case I'm going to call it Organics. Just copy and paste the file name and put it at the end of the cell so we can print it out. You see this workspace? Basically it's nice interactive tables and we have six different columns in the data set. So we've got three categorical columns. So gender, geographic region and loyalty status. We've got got two numeric features, we've got affluence, how rich they are and age. On the right hand side of this data frame we've got this purchased organics column that is a zero if they didn't purchase any organics and a one if they did. So this is going to be our response column on the right as a data dictionary if you want more details on what was in the data set. So first thing, scikit Learn really doesn't like categorical columns. So we've got to convert those first three columns, the gender, geographic region, likely status, into dummy columns with ones and zeros. So we're going to use the pandas get dummies, dummies function. I'm going to pass it the whole data frame, Organics and let's call this Organics Dumb, Dumb for dummies and you can see what the result is. Let's run this. So now rather than having a single column for gender, we've got three for the three different cases. Female, male, unknown. And these have got ones and zeros in. So the whole thing is numeric same for geographic region and loyalty status. Next step, we need to split the data set into response and explanatory columns. So the response, response that's just going to be organics dumb. And then we're going to take the purchased organics column. And then the explanatory variables, these are going to be organics. Wrong. It's going to be everything except the purchased organics column. So in this case we'll do drop. We say cons equals. Let's copy and paste this again. Purchased organics. So the response that's going to be a panda series. It's just a single column explanatory. It's going to be a data frame with one less column than the original. So let's run this. Next step. We need to split the data set into training and testing sets. So we're going to fit the model to the training set and we can do predictions on the testing set and we're going to use train test split for this. Train test split. This takes two inputs. It takes the explanatory variables and the response and just use all the other default settings and it's going to return four different things. So it's going to return explanatory train, explanatory test, response train and response test. Let's run this. Now we're about ready to fit the model to the training set. It's going to require two steps. So the first thing is that we need to create a model object. I'm going to call it MDL for model and we call logistic regression. Scikit learns a little bit weird in that by default it uses a technique called regularization. So this is used to minimize the effect of less important parameters in the model. It's really helpful if you've got lots of different parameters in your model. It's a bit of a controversial default. Today we're going to use standard logistic regression. This is a traditional algorithm, no regularization. So we need to set this argument penalty equals. Let's copy and paste that. I'm going to run this. No real exciting output. However, more exciting is we get to fit the model. So we're going to call it the fit method and in this case we're going to pass it the explanatory training set and the response training set. Let's run that again. Not really exciting output, just tells you which model you used. However, now we get to make some predictions on the testing set. So we're going to start again with the model object and we're going to call the predict method. And the predict method takes the explanatory variables from the testing set. So let's run this and see what the output looks like. Now we get a numpy array and the values of ones and zeros. So same format as the actual responses. And let's save this to a variable. So let's call it predicted responses. I'm just going to copy and paste this so you can see the output again. And so we have an answer. But what does it mean? Well, we need to assess the model performance by calculating some metrics. Now it's a classification model. There are four possible outcomes. And so the first outcome is that they didn't buy any organics in real life. And we also predicted they didn't buy organics. So that's a great outcome. We got the the answer correct. The other good case is that they did buy organics and we predicted that they did buy organics. Again we got it correct. And there are two bad cases though, unfortunately. So the first one is that they didn't buy organics but we predicted they did. It's called false positive. And the final case is they did buy organics but predicted they it didn't. That's called a false negative. We can count how many cases we got of each of these four outcomes using the confusion matrix method or function. And so this takes two inputs. The first input is what were the actual responses. So we call this responses test and then we can compare this. The second argument is the predicted responses. So let me run this and you can see, I think I've got response tests. There we go. So here we get a two by two matrix and the good cases are on the the diagonal. The bad cases are on the off diagonal. So in this case there were 2940 customers where they didn't buy organics and would predict it correctly. 482 cases where they did buy organics and we predicted correctly. So you can see the four values in the matrix correspond to these four cases. Open that little table and then we have the bad cases on, on the off date. So 670 false negatives, 189 false positives. Now it's really difficult to tell just from the confusion matrix, like are these numbers good or bad? So there are actually lots of different metrics you can use to assess the performance of your model. And classification report will print a lot of these. So the output is pretty vast. You've got four different columns, lots of different numbers. They're actually typically only 5, 5 numbers that you often care about. And the easiest of them, a good place to start is with model accuracy. So what this means is what are the fractions of values that were correctly predicted? So in this case it's the number of true negatives plus the number of true positives divided by all the possible cases. So let's go back to the matrix. So true negatives, that's 2,940 plus 482 divided by all the different cases. So all these different things added up together, that's basically the number of cases that were in the testing set. And so we do a bit of arithmetic and that's going to give us the model accuracy. There are four of the metrics called precision recall for each of the two cases. I think it's worth me getting into these in detail. I've left these here written down. So if you want to follow through on workspace Then you can look at these at your leisure, but they sort of divide up different ways of assessing, like how many false negatives and false positives you have. All right, so let's print out this classification report. It takes the same arguments as confusion matrix. So the actual responses and the predicted responses. I am going to copy and paste these again so you can see what this looks like. And actually this can print really badly. So classification report is pretty stupid. And this returns a string and just prints terribly. So you have to wrap it in a call to print again and there it looks a bit nicer. As you can see, this accuracy score, we've got 0.8. So the model was right about 0.8 at the time. Is that good or bad? I don't know the retail scenario well enough to say whether or not that's any good. There's probably ways of improving the model to be fair, because we just did something very simple. And then those other four numbers that I said are quite useful, the precision recall values, those are in the top left if you want. But for now we'll just stick with accuracy of 0.8. If you've enjoyed this tutorial, then Datacamp has a lot more courses that cover logistic regression in Python. A good place to start is machine learning with Scikit Learn. So this provides a sort of general introduction to modeling Scikit Learn, and it does cover logistic regression. There are two courses that cover regression, linear and logistic regression regression in a lot more depth. So that's Introduction to Regression with SAS models in Python and Intermediate Regression with SAS models in Python. While they do use a different package, the SAS models package, they do get into a bit more detail about how the algorithms work and how your SAS model performs. Hope you've enjoyed this tutorial. Look out for more tutorials coming from Data Camp. And please do take these courses. Please do have a play around with Workspace. Thank you. It.