Jose Hernandez shows how to transform raw data in R for machine learning. He uses the AdultIncomes dataset to demonstrate encoding categorical variables like work class into numeric columns, either manually with dplyr or automatically with caret::dummyVars.
Chapters
Hi, my name is Jose Hernandez and I am a data scientist. In this course you will learn about feature engineering. Feature engineering involves adjusting and reworking the raw predictors you will use to represent a model to better uncover predictor outcome relationships. This means you are in search of features that, in combination, will be the best representation of the sample data. To answer a specific question, we will focus on tabular data and in this context, feature engineering involves combining or decomposing information within existing features to create new ones. The engineering part of feature engineering means that you will be manually creating new columns in your data. For example, you are given data on adult incomes and are tasked with building a model that adequately predicts whether an individual is below, above, or above a specific salary range. To take a look at existing features, we can use the table and glimpse functions. The modeling process will be iterative as you determine which features should be selected as inputs to your model. As you continue exploring your data, you may realize that some of your features are categories in the form of strings and can't be understood as inputs by the algorithm. In their current form, the adultincomes data contains raw features that are integers like age and factors or character variables like work class. Let's assume you are interested in using the word class column from the AdultIncomes dataset as a feature in a model that has the same two class income levels as the target variable. The word class variable is a categorical string variable with eight classes which correspond to the sector of a person's job. Most statistical and machine learning models do not understand text inputs. If we want to use these categories as features, we need to encode them in a way that our model could understand. In this course, we will be manipulating data using functions from the DPLYR library. Here we will use the mutate function to create a new column for each word class category in the ifelse function inside the mutate function to specify the conditions for new features. This is useful when you have a small number of categories to encode. For example, the mutate function will create a new column newworkclass of ones if the category is federal government and zero for everything else. We will see a full example of this on the next slide. If you have a large number of categories, functions like dummy vars from the CARET library have automated this process. The function takes in a specific column or a whole data frame and creates a column for each category. Since most machine learning and statistical models cannot interpret these groupings automatically, they have to be numerically encoded. We can use one hot encoding a feature engineering technique in which we assign numerical values of zeros and ones to represent different categories in the string variable. In this case, we will create eight columns representing each category, with a one if the category is present and a zero for other categories. Now it's your turn. Let's.
