David Aralaiya shows how to load multiple CSV files into pandas DataFrames using pd.read_csv, then combine them with loops or list comprehensions. He demonstrates using glob to automatically find and read files matching a pattern.
Chapters
Welcome to Merging Dataframes with Pandas My name is David Aralaiya. I'm an applied mathematician and data scientist. This course is all about merging and combining data frames for your data science needs. Your data rarely exists as data frames. From the outset you generally have to deal with text files, spreadsheets and databases. Let's first check out how to read multiple files into a collection of data frames. The primary tool we've used for data import is readcsv. This function accepts the file path of a comma separated values file as input and returns a Pandas dataframe directly. Readcsv has about 50 optional calling parameters permitting very fine tuned data import. Pandas has other convenient tools with similar default calling syntax that import various data formats like Excel, HTML, or JSON into data frames. To read multiple files using Pandas, we generally need separate data frames. For example, here we call PD readcsv twice to read two CSV files sales January 2015csv and sales February 2015csv into two distinct data frames. It's generally more efficient to iterate over a collection of file names. With that goal, we can create a list file names with the two file paths from before. We then initialize an empty list called dataframes and iterate through the list file names within each iteration. We invoke readcsv to read a dataframe from a file, and we append the resulting dataframe to the list dataframes. We can also do the preceding computation with a list comprehension. Comprehensions are a convenient python construction for exactly this kind of loop where an empty list is appended to within each iteration. You can check out datacamp's python programming courses for more details on comprehensions. When many file names have a similar pattern, the glob module from the Python standard library is very useful here. We start by importing the function glob from the built in glob module. We use the pattern sales CSV to match any strings that start with the prefix sales and end with the suffix CSV. The asterisk is a wildcard that matches zero or more standard characters. The function glob uses the wildcard pattern to create an iterable object filenames containing all matching file names in the current directory. Finally, the iterable filenames is consumed in a list comprehension that makes a list called dataframes containing the relevant data structures. Now it's your turn to practice reading multiple files into dataframes in the exercises.
