Learn when to drop irrelevant columns versus creating new features to reduce dataset size. Feature selection removes redundant data like correlated weight columns, while feature extraction like PCA can compress four dimensions to two while keeping 96% of the information.
Chapters
Reducing the number of dimensions in your dataset has multiple benefits. Your dataset will become simpler and thus easier to work with. Require less disk space to store, and computations will run faster. In addition, models are less likely to overfit on a dataset with fewer dimensions. The simplest way to reduce dimensionality is to only select the features or columns that are important to you from a larger data set. The hard part here is to decide on which features are important. If you're an expert on what the data is about, you may know this by heart. You would, for example, know that a person's favorite color is irrelevant if you want to predict whether they'll default on a loan. And with the Pandas dataframe drop method, you could remove that feature easily. Just make sure to pass the axis argument one to specify we're dropping a column instead of a row. If you're new to a dataset, you'll have to do some exploring before you can take a decision on which features can be dropped. Seaborn's Spare plot is excellent to visually explore small to medium sized datasets. It provides a one by one comparison of each numeric feature in the dataset in the form of a scatter plot, plus diagonally a view of the distribution of each feature. The example shown here visualizes a sample of the US army body measurement dataset called ansur, and we've set the kind of plot to show on the diagonal to hist for histogram, we can spot that the weight in pounds is perfectly correlated to the weight in kilograms since all points fall on a diagonal line. Since both features hold the same information, it makes perfect sense to drop one of them. If there would have been a numeric feature without any variance in the dataset, such as the constant that we've added here, it too would have been easy to spot visually. Also note that we use the categorical gender feature to color the points. Since all points in the sample are for males, we could decide to drop that feature too. When we apply feature selection, we completely remove a feature and information it holds from the dataset, such as the yellow column. In this example, we try to minimize the information loss by removing features that are irrelevant or hold little unique information, but this is not always possible compared to feature selection. Feature extraction is a completely different approach, but with the same goal of reducing dimensionality. Instead of selecting a subset of features from our initial data set, we'll be calculating or extracting new features from the original ones. These new features have as little redundant information in them as possible and are therefore fewer in number. One downside is that the newly created features are often less intuitive to understand than the original ones. The dimensionality of datasets with a lot of strong correlations between the different features in it, such as the one shown here, can be reduced a lot with feature extraction. Intuitively, this makes sense. If I were to tell you that a person has long feet, long hands, long arms, and so on, I could probably just summarize that this person is tall. In fact, we can reduce the number of dimensions in this answer DataSet sample from 4 to 2 with a technique called PCA or Principal Component analysis, and keep 96% of the variance in the data. We'll be looking into PCA in detail in chapter four. For now, it's your turn to visualize.
