In this tutorial, Akansha Yadav walks through three practical pandas techniques for data cleaning and transformation. First, she shows how to replace the default integer index of a DataFrame with a date‑based index by creating pandas Period objects and assigning them to df.index. Next, she demonstrates using df.replace with the regex flag to normalize textual values, correcting misspelled or variably‑cased entries such as different forms of "Australia". Finally, she builds a custom cleaning function that leverages Python's re module to strip bracketed state codes (e.g., "NY", "IL", "CA") from city names, applying the function to a column with .apply. Throughout, she emphasizes how these methods scale from a few rows to hundreds of thousands, and ends with a call‑to‑action for viewers to like and subscribe.
Chapters
Ladies and gentlemen, hello everyone, welcome back to my YouTube channel. In today's video we will be discussing again about pandas. So let's get back on the screen and see what are we going to learn today. So in the last video when we were creating some DataFrame or we were importing the data, so we saw that there was some indexes which were generated automatically. It means 0, 1, 2, 3, 4, which was the by default values. So now let's see how we can replace them with the certain type of data. So what I have done for today is I have created a DataFrame where we a city, event, and cost in the DataFrame. And after that, what I am trying to do is I am trying to add the indexes. Now what I want is I want the date for that particular event. So what I have to do is I have to insert the date as an index. So how to do that? So this is our data, and in the data we have the by default indexes 0, 1, 2, 3, 4. Now to add the date, what I have to use is I have to use the pandas library .period method. So pandas .period method I have to use, and then I have to write down the that I wanted to add. So I want month and year. I'm not writing particularly the date. If you want, you can, but I am writing just the month and the year. So there are 5 values in my data, it means the 5 different rows are there. So I have to write 5 different indexes, it means 5 times I have to repeat pd.period. And then I have to use a variable, df.index, means I'm going to replace it. So I'm using index_variable to replace it. And that's how you can add the dates in your data. So I hope you found it interesting. The next that we are going to learn today is how to apply any function to it. Suppose I wanted to make some modification in my data. So I have the data where I have country, capital, population. So in this country, capital, population, I have one country Australia, and I have written it twice, but I have made a mistake writing it in second time. So first I have written Australia, which is with the correct spelling, but I— what I want is I want Austra underscore Lia. This is what I am trying to make a change in this data. So let's see how we can make the Astra, or Astra it means A capital or lowercase, like lowercase a, with the capital A Astra underscore. I want this as an output in my data. So to do that, what you have to do is first is I know what is my target value. I know Astra I want, I want either it is with the capital or with the small letters, you have to work on it. So I have created a variable, then df.replace. replace method will help me replace it, but with what? To replace, in the to replace I have to write down what I wanted to replace. In the quotation I have written, within the square bracket I am defining lowercase letter also and in the caps also, and then I have written the entire word outside of it within the quotation. Then after comma, what value you wanted to, uh, be there at the— in the output. So what I have done is value is equal to austra_— this is what I want in the output. So I have done regex is equal to true. Regex is one of the function of pandas which helps you to make the change. So here you can see that output is generated, but in the 5th number where I have misspelled the Austria, where I was writing double S, so that there was no change. It was as it is because I have not written anything specific to the double S or S. I have just written for capital and small letters. So that is how it changes the output, and that is how you can make any minor change to your data like this. Like right now we have only 5 rows, but what if we have 5 lakh rows? How you can modify it. So in case of 5 lakh, this method will help you to replace it in your entire data. But if you have 5 rows, you can manually go and edit it. But for the future perspective, I would say use this method. Another is, suppose we have a scenario where we have a restaurant in the different cities, but city's name consists of some additional details enclosed in the brackets. So in this task, what we are going to do is we are going to see how we can customize a function using a regular expression., and using that regular expression, we will be identifying and updating the names to those cities. So first, I have created a normal DataFrame where we have city, restaurant, and rating, and then I have created a DataFrame using the same dictionary data. Now what I have to do is, this is my data where I have New York and bracket, we have NY, Chicago, we have IL, then we have San Francisco, we have CA. So I wanted to remove all these values. So how we can remove it is, there are multiple ways to do it. Now I am going to tell you the first way. First is we need the re package, which is for regular expression. So you have to import it. What you can do is you can directly write import to import it. Then I am creating a function, suppose clean_names, clean_name, and then in the brackets I have, uh, passed a parameter which is city_name. City is a variable, but I'm just passing a parameter similar to that, which is city_name. Then I am writing re.search. I am asking my regular expression to search for this particular character. I have written multiple characters, although I know that there are brackets only, but I am telling you Through this, that you can write any character which you wanted to search. Right now we have brackets, so I have written 5 characters, slash, comma, and asterisk mark as well. And then I am writing comma city name. From city name, you have to search for this particular thing using the regular expression. This is what it says. Now what I have to do is I have to extract the position of the beginning of the pattern. It means wherever it searched for this particular character, I want that particular thing to start and to extract it. It means, uh, rest of the things I don't want from that particular column. And I am returning the city name from starting to that particular character. It means from starting to that particular character, rest of the things I am going to delete it. So I have already deleted it in the first command, pos is equal to research. So that's how return statement works. And else, what I am requesting it to do, if cleanup need, return the same name, and there I am returning the city name. So let's apply this. To apply this, I'm using city, df.city, and then I'm applying a function onto that particular column. df bracket city dot apply and the function name. Let's first run this program so that we will get the output in the— we are getting the name error just because we have not run that above command. So that's how you can remove any character or any particular detail that you wanted to make in the tita. So I hope you like this video. Don't forget to hit the like button, subscribe to my channel for more informational videos like this on pandas.
