Learn to load flat files into NumPy arrays using loadtxt and genfromtxt. The tutorial covers key arguments like delimiter, skiprows, usecols, and dtype with examples, then explains why pandas DataFrames work better for mixed data types like the Titanic dataset.
Chapters
Okay, so you now know how to use Python's built in open function to open text files. What if you now want to import a flat file and assign it to a variable? If all the data are numerical, you can use the package numpy to import the data as a numpy array. Now why would we want to do this? First off, numpy arrays are the Python standard for storing numerical data. They are efficient, fast and clean. Secondly, numpy arrays are often essential for other packages such as Scikit Learn, a popular machine learning package for Python. Numpy itself has a number of built in functions that make it far easier and more efficient for us to import data as arrays. Enter the numpy functions loadtext and genfromtext. To use either of these, we first need to import numpy. We then call loadtext and pass it the FileName as the first argument along with the delimiter as the second argument. Note that the default delimiter is any white space, so we'll usually need to specify it explicitly. There are a number of additional arguments you may wish to specify. If, for example, your data consists of numerics and your header has strings in it, such as in the mnist digits data, you will want to skip the first row by calling loadtext with the argument skiprows1. If you want only the first and third columns of the data you'll want to set, use cols equal to the list containing ints 0 and 2. You can also import different data types into NumPy arrays. For example, setting the argument dtype equal to string will ensure that all entries are imported as strings. Load text is great for basic cases, but tends to break down when we have mixed data types, for example columns consisting of floats and columns consisting of strings, such as we saw in the Titanic data set. Now it's your turn to have fun with load text. You'll also gain hands on experience with other functions that can handle mixed data types. In the next video we'll see that although numpy arrays can handle data of mixed types, the natural place for such data really is the data frame.
