When your data has imbalanced classes, a random train-test split can create unrepresentative subsets. Use scikit-learn's stratify parameter to preserve the original class distribution: with 100 samples (80 class 1, 20 class 2), stratified sampling gives you 60 and 15 in training, 20 and 5 in test.
Chapters
One of the most necessary steps for pre processing, which you should be familiar with if you've taken other courses on Python and machine learning, is splitting up your data into training and test sets. We do this to avoid the issue of overfitting. If we train a model on our entire set of data, we won't have any way to test and validate our model because the model will essentially know the data set by heart. Holding out a test set allows us to preserve some data the model hasn't seen yet. Just to review this is how you split up your Data set in SciKit learn using the Train Test split function. This should look familiar to you. The function shovels up your data set and then randomly splits it. By default, the function will split 75% of the data into the training set and 25% into the test set. In many scenarios, the default splitting parameters will work well. However, if your labels have an uneven distribution, your test and training sets might not be representative samples of your data set and could bias the model you're trying to train. For example, if you look at the example training and test data sets on this slide, you can see that the training set has only samples labeled N, while there is a Y label in the test set. A good technique for sampling more accurately when you have imbalanced classes is stratified sampling, which is a way of sampling that takes into account the distribution of classes or features in your data set. So, for example, let's say we had a Data set with 100 samples, 80 of which are Class 1 and 20 of which are Class 2. We want the class distribution in both our training set and our test set to reflect this. So in both our training and test sets, we'd want 80% of our samples to be Class 1 and 20% to be Class 2, which means we'd want 60 Class 1 samples and 15 Class 2 samples in our training set of 75 samples. In our test set of 25 samples, we want to have 20 Class 1 samples and five of Class 2. This is on par with the distribution of classes in the original data set. There's a really easy way to do this in scikit Learn using the traintestsplit function. The function comes with a stratify parameter, and to stratify according to class labels, just pass in your Y dataset to that parameter. So here we have our 100 labels. 80 are class 1 and 20 are class 2. Let's run train testsplit and pass the Y labels dataset into the stratify parameter. If we check the distribution of classes for our training and test labels. You can see the distribution of classes is in accordance with the original Y class distribution. Now it's your turn to do some stratified sampling.
