Learn how to import CSV files into pandas DataFrames, which store labeled two-dimensional data with mixed column types. The tutorial covers read_csv, the .head() method to preview data, and how pandas handles missing values and comments in flat files.
Chapters
Congrats. You're now able to import a bunch of different types of flat files into Python as numpy arrays. Although arrays are incredibly powerful and serve a number of essential purposes, they cannot fulfill one of the most basic needs of a data scientist to have two dimensional labeled data structures with columns of potentially different types that you can easily perform a plethora of data sciencey type things on manipulate slice, reshape group by join merge perform statistics in a missing value friendly manner deal with time series the need for such a data structure, among other issues, prompted Wes McKinney to develop the Pandas library for Python. Nothing speaks of the project of Pandas more than the documentation itself. Python has long been great for data munging and preparation, but less so for data analysis and modeling. Pandas helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R. The data structure most relevant to the data manipulation and analysis workflow that Pandas offers is the dataframe, and it is the Pythonic analog of R's dataframe. As Hadley Wickham tweeted, a matrix has rows and columns. A data frame has observations and variables. Manipulating data frames in Pandas can be useful in all steps of the data scientific method, from exploratory data analysis to data wrangling, pre processing, building models and visualization. Here we will see its great utility in importing flat files, even merely in the way that it deals with missing data Comments along with many other issues that plague working data scientists today. For all of these reasons, it is now standard and best practice in data science to use Pandas to import flat files as data frames. Later in this course we'll see how many other types of data, whether they're stored in relational databases, HDF5, MATLAB or Excel files can easily be imported as dataframe. To use Pandas, you first need to import it. Then, if we wish to import a CSV in the most basic case, all we need to do is to call the function readcsv and supply it with a single argument, the name of the file. Having assigned the dataframe to the variable data, we can check the first five rows of the dataframe, including the header with the command data head. We can also easily convert the data frame to a numpy array by calling the dataframe attribute values. Now it's your turn to play around importing flat files Using Python, you'll get experience importing a flat file that is straightforward, and you'll also get experience importing a flat file that has a few issues, such as one that contains comments and strings that should be interpreted as missing values. Have fun importing.
