Python Machine Learning Tutorial | Centering And Scaling | Databytes

Hi, I'm Richie. In this Python machine learning tutorial, we're going to look at a data pre processing technique called centering and scaling. They come as a pair. I'm using datacamp workspace for this, but you can use any Python environment. Before we start writing any code, we're going to take a look at a bit of theory. So first of all, what is data pre processing? Essentially, data pre processing is just a type of data manipulation. And the idea is that you want to get your data set ready for fitting a machine learning model. Now, we're assuming that your data set is going to be in the form of a Pandas data frame or something similar. And for each of the columns that are going to be input into the model, these are called features. And the idea of data pre processing is that you can do some transformations on these features. So what are centering and scaling? Centering means that you calculate the mean of a feature and then you subtract that mean from each element. So the resulting processed feature then has a mean of zero. Similarly, scaling means dividing each element of the feature by the standard deviation. And that means that the processed feature is going to have a standard deviation of 1. One thing to note is that some people tend to use the term standardizing instead of scaling. It means the same thing. You might wonder when you should use centering and scaling. And it all depends on the type of model that you're fitting. So some models are going to assume that each feature has a standard normal distribution. So, so standard normal means it's got a mean of zero and a standard deviation of one. So if you are using a k nearest neighbors model, if you're using support vector machines with the radial basis function kernel trick, or if you're doing regularized regression, so that means lasso or ridge regression. In this case, centering and scaling the features is absolutely essential. For some other model types, centering and scaling aren't essential steps, but they can help with convergence of the model. So that means sort of deep down in the bowels of the algorithm, it's going to help out. You'll know if you have a problem with convergence because you're going to start getting warnings or errors in your code saying there's a problem with the convergence. So that includes linear regression, logistic regression, and neural networks. So in those cases, centering and scaling are helpful but not necessary. And for some other model types, that includes tree based models, so decision trees, random forest, gradient boost, anything else like that. Also naive Bayes centering and scaling have absolutely no effect, there is no point in bothering with it. Now. In this tutorial we're going to use Scikit Learn. However, pretty much every machine learning framework is going to have tools for centering and scaling features. So you can use Pyrot, you can use Keras, you can use whatever you want. Also, because data pre processing is essentially just data manipulation, you can use data manipulation tools like Pandas and Numpy. Let's try a case study. So we're going to fit a K nearest neighbors model and we're going to use the DIAMONDS data set. This is a pretty famous data set that's been around in the public domain for a while. It was originally made popular in Rs ggplot2 package and it is available in Python via the plot9 package. So the first thing I'm going to do is import this Data set from plot 9. And I'm going to print the data set. Here. You can see we've got just over 50,000 rows. Each row represents a diamond. And when we do the modeling, I'm going to use cut as the response variable and I'm going to use all the numeric features for features. So just for ease of coding, I'm going to ignore color and clarity for now. So we're going to use carat and then depth through to the Z dimension as features. Before we do any modeling, I'm going to calculate some summary statistics using the Describe method. So if we look at the CARET column, you can see the smallest value is 0.2 and the largest value is just over 5. So by contrast, if we look at the price column, I think this is in US dollars, it goes from about 300 to about 19,000. And the sad truth here that you can't buy a one carat diamond for one US dollar. But what this means from a modeling point of view is that these different features have very different ranges. That is they are, they have different scales and this is going to cause problems with our K nearest neighbors model. So we actually only need three functions here. So we're going to use traintestsplit and this is going to split the data set into the training testing sets. We're going to use KnaversClassifier to fit the model. If you're from a Commonwealth country, note that we use US English here and we can use standard scalar to scale the features. So I'm going to import these three functions. Now. Do this three times. So I'm just going to copy and paste because I'm a bad typist. So let's bring in traintestsplit. So this is in the model selection submodule of scikit Learn. Oops, deleted the N. We're going to get knabbours classifier. And this is in the neighbours. Can't spell it correctly. Neighbors submodule of scikit Learn. And we get standard scal. So this is in the pre processing submodule. Let me run this. Make sure I've typed all those correctly, no errors. So that seems to have worked. Now, the next thing, we're going to get the response variable and the features out of that data set. So first I'm just gonna call these X and Y. It's boring names but fairly standard. So the response variable, that's going to be diamonds. And we're going to take the cut column. And then for the features, we want everything except the following columns. So we don't want cut because that is the response. And I said we're going to ignore color and clarity because they're categorical variables and require a bit more effort to deal with. So I'm going to run that. That argument should be columns, not column. And now we're going to do the train test split. We're just going to use the default arguments. And the trick with calling this function is to remember which way round the four results come. So we'll call these xtrained, X test, ytrain and Y test. I think that's the right order. Let's run this. So the first step in running Knarest Neighbors model is to create a kneighborsclassifier object. So I'm just gonna call this knn and I am going to copy and paste this. We're not going to bother with any arguments, just using the defaults. So let's run this. And we have a knighbours classifier object. So to fit the model, we just call KNN fit. I'm going to fit it to the training data. So we're going to pass X train and Y train and run that. The output's not very interesting. However, now we can measure the accuracy of the predictions on the testing data set. So this time we're going to call KNN score. So score gives you the accuracy score of the model and we're going to pass it X test and Y test. Let's run that. So we get 0.55. So it's 55% accurate. That doesn't sound great. It's hard to tell whether that's good or bad, though. Let's try running the model again with standardizing. So we're going to do centering and we're going to do scaling. So first of all, we're going to create a standard scale object. And one thing that's really important is that you have to do this scaling process after you've done the train test split. So if you do the standardization before you've split the data, you're going to have information from the testing data set leaking into the training set. So you have extra information that you shouldn't know. So there's a problem called data leakage. So it's just really important that you do the train test split first and then you do the any sort of manipulating the features afterwards. So we're going to do the same line of code essentially twice. So I'm going to call the resulting variable xtrainscaled and we're going to call the FitTransform method. And so what fitting means, essentially it's calculating the mean and standard deviation of the features. And then the transform part is when you subtract the mean and divide by the standard deviation. So we can do both these things in a single step with this method and we need to pass it the training features. So I'm just going to copy and paste this again. So we move twice, once on the training set and once on the testing set. And let's run this. And we're going to fit the model again. So we're going to call KNN fit and this time we're going to pass it X train scaled and we're going to use the same response as before. So oops, not Y test, Y train. Let's run that. And we're going to calculate the accuracy score again on the testing set. So we're going to call KNN score and this time we're going to pass it X test scaled and Y test. So run this and you see now it's much higher. So before it was 0.55 we had 55 accuracy. Now we have more than 70% accuracy. So it's gone up 15% from just two lines of code. And that's pretty much the best situation you can ask for when you're doing machine learning. It's like, watch out, Kaggle grandmasters. We're getting some high predictions. So that's pretty great. Now, it's worth noting that standard scalar isn't the only scalar available in Scikit Learn. There are a few others available. So if you have a lot of outliers or really skewed data, then robust Scalar might make more sense. So rather than subtracting the mean and divided by the standard deviation, it subtracts the median divides by the interquartile range. So it's kind of a good way of dealing with outliers. You've got MaxABScalar divides by the largest absolute value in each feature, so that means that the sort of maximum range of each feature is going to be from minus one to one. That's occasionally useful. And similarly with Minmax Scalar, this will convert all values to a range. So you quite often see data sets where it goes from like 0 to 1 or something like that. Normalizer is slightly different because it transforms rows rather than columns. And Normalizr will scale each row so the sum of the squares of each values in that row adds up to one. If you are interested in learning more, there are a couple of datacamp courses on this area. So we've got one called Pre Processing for Machine Learning in Python, one called Feature Engineering for Machine Learning in Python. These both help you prepare data for running models. SciKit Learn has its own free Pre Processing data tutorial, and as I was researching this, I came across a very interesting question on Quora. So this goes into a bit more depth in terms of which models need scaling and normalization and why. All right, Happy Data Processing. Thanks.