In this instructional video, the presenter walks through the essential data‑preparation steps required before training a supervised learning model using a telecom (telco) dataset. The workflow begins with a quick data‑exploration using pandas' head method and a review of column data types, highlighting that most columns are object (string) types that need conversion. The speaker then separates identifier columns (e.g., customer ID) and the target variable (churn) into distinct lists. A rule‑based approach identifies categorical features as those with fewer than ten unique values, demonstrated via df.nunique(). After excluding the target, the remaining columns are classified as numerical. The tutorial covers one‑hot encoding of categorical variables with pandas.getdummies(dropfirst=True), explaining why the first dummy column is dropped to avoid perfect multicollinearity. Next, numerical features are standardized using sklearn's StandardScaler (fit_transform), converting the resulting NumPy array back into a pandas DataFrame with original column names. Finally, the scaled numerical DataFrame is merged back with the original dataset using a left join (pandas.merge) after dropping the unscaled numerical columns, producing a fully pre‑processed dataset ready for model building. Throughout, the presenter emphasizes best practices, rationales for each transformation, and provides code‑level guidance.
Chapters
You've done a great job. Now we will explore the data preparation techniques for supervised learning models. First thing you want to do is to explore the data sample. A head method called On Pandas dataframe will print the first five rows here. We will use the telecom data set next. It's always a good practice to review data types. As you can see, most of the columns are strings while marked as objects. This means they have text data that we'll have to transform so our model can use it. Before we start our data preparation steps, we need to separate the identifiers such as customer ID and the target variable, which in this case is the churn flag. We store them as separate lists that we will use later. Then we separate categorical column names using a rule which defines a column as categorical if it has less than 10 unique values. This number is arbitrary and it is a good practice to explore your data to see if there are variables with more unique values. We can explore them by running Telcoraw Nunique command on a dataframe and exploring the output. We will analyze this in the exercises for this lesson. The next step is to remove the target variable called churn from this list so we don't do any transformations on it. Finally, we store the remaining column names into a list called numerical. We use list comprehensions here, which are like loops, only fit one line of code. It commands to extract all columns that are in the telco raw but excludes the ones that are not in list. We have defined earlier with column names for Customer id, target and categorical variables. Now we will convert these variables into binary columns with ones and zeros. This is called one hot encoding. It transforms a categorical variable with string values like product names by building as many columns as there are unique values in that variable. This way the machine learning gets an integer value instead of a text. Here's an example column with color values for each row we when we transform it with one hot encoding, we will get a different table. Here's the result after transformation. As you can see, we have three columns instead of one and each column has ones or zeros instead of text. There's a simple command in Python that makes this an easy task. One HOT encoding is done using getdummies function from Pandas library. We use the drop first true argument. This ensures we remove the first encoded column as it is redundant and can be inferred from the others. Most machine learning algorithms prefer it that way. For example, if someone asks you if you own a car, there are two yes and no without dropping the first category we would end up with two columns, yes and no with perfect negative correlation. That is whenever one column is one, the other will be zero as there is only one value per observation in the original column. We will explore this later in more depth. Another important thing to do before modeling is to scale the numerical features. A manual way to do this would be to subtract the average value of each column and divide by standard deviation. This gives us the columns with average of 0 and standard deviation of 1. Some machine learning models require this preprocessing step because otherwise the variables with larger mean and standard deviation values would have more influence as predictors. In Python we do this by importing standard scalar library from Scikit Learn, initializing its instance and running a FITT transform method on the numerical columns of the dataset. The result is a numpy array. We can create a scaled PANDAS dataframe from this numpy array by calling the dataframe function and feeding the numpy array and the column names from the original numerical dataframe. Finally, we combine everything together. First we drop the non scaled numerical columns from the telco raw dataset. Then we merge it with the scaled numerical data. This is done with pandas merge function. We tell the function to make a left merge type which is equivalent to a left join in SQL. This means that all values will be kept in the dataset on the left which is telco raw. Then we look for those values in the dataset on the right which is scaled numerical and if there are missing values they will be recorded as null values. This is it. You are now ready to practice these pre processing steps before we move to building a machine learning model.
