Learn ensemble averaging, or soft voting, which combines multiple machine learning models by averaging their predictions. Using scikit-learn's VotingClassifier and VotingRegressor, you can weight stronger models more heavily. The tutorial builds an example with K-Nearest Neighbors, Decision Tree, and Logistic Regression, then has you apply it to predict whether Game of Thrones characters are alive.
Chapters
In this lesson, you'll learn about another popular ensemble averaging. Say you are participating in a game of guessing how many jelly beans there are in a jar. The idea is to guess the number of jelly beans and the participant who is closer to the actual receives the full jar or another price. As you are not allowed to open the jar and count them individually, your best bet is to estimate the amount. But how can we provide a good estimate? The easiest option would be to choose a random reasonable number. A more intelligent approach would be to approximate the volume of the jar, and you could take even more advanced approaches. Did you know that the average of the individual guesses tends to be close to the actual value and can be as good as or better than any of them? It sounds counterintuitive, right? But it's the same principle we saw in the previous the Wisdom of the Crowd. The average higgin symbol takes advantage of this principle. This ensembling technique is known as averaging or softboarding, and it can be applied to both classification and regression. In this technique, the combined prediction is the mean of the individual predictions. For regression we use the predicted values, and for classification we use the predicted probabilities. As the min doesn't have ambiguous cases like the mode, we can use any number of estimators as long as we have at least two of them. To build an averaging classifier, we'll use the same class as votingclassifier. The main difference is that we specify an additional parameter voting with the value of soft. The default value is hard. We can also pass the optional parameter weights which specifies a weight for each of the estimators. If specified, the combined prediction is a weighted average of the individual ones. Otherwise the weights are considered uniform. In a similar way, we can build an averaging regressor. For this purpose, we use the booleanregressor class from the SciKilearn Ensemble module. The first parameter is also the list of string estimatorstupples, but instead of classifiers we use regressors. Let's see an average in ensemble in action. Here we have a Find nearest neighbors classifier, a decision tree, and a logistic regression. We create a narration classifier passing the list of estimators and a parameter voting as soft. In addition, assuming that we know that decision tree has better individual performance, we give it a higher weight. Ideally, the weights should be tuned while training the model, for example using grid search cross validation. For the exercises that follow, you'll be using a dataset consisting of characters from the popular series Game of Thrones. Your target is to predict whether a character is alive or not. For that purpose, you'll use features like age, gender, books in which the character appears, its popularity, and whether or not the character's relatives are alive. The TV show may be over, but there are still two books to go. It's your turn now to put averaging into practice and predict the deaths of characters from.
