R Tutorial: Tidy your models with broom

Now that you know how to work with list columns in a tidy manner, you can begin to work with the tools you need to explore and evaluate machine learning models. As you can probably imagine, the bulk of the work of machine learning resides in step two of this workflow. Since you can store complex model objects in your data frame, you can also work with these objects using the tools available in various R packages. In this video, we will focus on the Broom package, a package designed to convert useful model outputs into tidy data frames. The core of Broom is encapsulated by three functions which aim to extract conceptually different information from any model. Tidy is used to extract the statistical findings of a model, Glance provides a one row summary of a model, and Augment appends the predicted values of a model to the data being modeled. Let's explore each of these in greater detail by reviewing the results of the linear model that you created for Algeria. If you look at the summary of the Algeria model, you can see that there is a lot of useful information here. However, this information is not particularly easy to extract directly from the object as it is to simply print it. But using Tidy and Glance, you can easily extract this information into data frames. The Tidy function collects the statistical findings of a model into a data frame. When used with a linear model, Tidy returns the coefficients and and their corresponding statistics for that model. To extract these statistics, you simply apply the Tidy function to the model object as shown here. The next broom function, Glance, is used to return a one row summary of a model for a linear model. This summary contains various statistics about the fit of the model, such as the R squared. Extracting this information into a dataframe is as simple as calling the function on the model object. Finally, the Augment function builds an observation level dataframe containing the original data used to build the model as well as the predicted value for each observation as the column dot fitted. Furthermore, Augment appends model specific statistics A fit for each observation by constructing a data frame containing both the original values and those predicted by our model, you can explore the fit of the model. For instance, you can visualize how well your model fits the data by plotting the predicted and actual values of life expectancy with respect to year. In this plot, the actual values are the black points and the fit of the model or predicted values is shown as the red line. By examining this plot, you can learn that a simple linear model may not be the best approach for this example, and you would consider either including more features or using a nonlinear approach to better capture this relationship. Using these three tools makes it easy to extract model coefficients, fit statistics, and observation level performance for many different machine learning models. In Chapter two, we will use BROOM as a part of the list column workflow to do this for all 77 of our country level models with just a few lines of code. But first, let's review what you have learned with a few exercises.