The tutorial introduces the ensemble technique of majority voting, drawing on the "wisdom of crowds" analogy from TV game shows where the audience’s most‑voted answer often predicts the correct outcome. It explains why voting works best with an odd number of diverse, independent classifiers to avoid ties and ensure each model contributes its own prediction. The speaker outlines key requirements for an effective voting ensemble: diversity (different algorithms or data), independence (uncorrelated errors), and autonomous predictions. The lesson then demonstrates how Scikit‑Learn’s VotingClassifier simplifies building such ensembles. Using the estimators keyword, the instructor creates a list of (label, estimator) tuples for a K‑Nearest Neighbors classifier, a Decision Tree, and a Logistic Regression model, assigns them labels (knn, dt, lr), and constructs the voting classifier. The combined model is fitted on training data (Xtrain, ytrain), used to predict on test data (Xtest), and its accuracy is measured with accuracyscore(ytest, ypred). The tutorial concludes with a hands‑on example of building and evaluating the first voting ensemble.
Chapters
In this lesson, you are going to learn how to use an ensemble technique known as voting. In TV shows like who Wants to Be a Millionaire? When a contestant doesn't know the right answer, they have the option of asking the audience. The contestant usually ends up choosing the answer most voted for by the audience, hoping it is the correct one. In fact, according to stats from TV studios, the audience predicts the correct answer more than 90% of the time. This is thanks to the concept known as wisdom of the crowds. It refers to the collective intelligence based on a group of individuals instead of a single expert. The aggregated opinion of the crowd can be as good as and is usually superior to the answer of any individual, even that of an expert. This is a useful technique commonly applied to problem solving, decision making, innovation and prediction. We are particularly interested in prediction. As the name implies, this priority voting technique combines the output of many classifiers using a majority voting approach. In other words, the combined prediction is the mode of the individual predictions. It is recommended to use an odd number of classifiers. For example, if we use four classifiers, the predictions for positive and negative classes could be tied. Therefore, we need at least three classifiers and when problem constraints allow it, use five or more. There are some characteristics you need in your crowd for a voting ensemble to be effective. First, the ensemble needs to be diverse. You can do this by using different algorithms or different datasets. Second, each prediction needs to be independent and non correlated from the rest. Third, each model should be able to make its own prediction without relying on the other predictions. Finally, the ensemble model should aggregate individual predictions into a collective one. Keep in mind that majority voting is a technique which can only be applied to classification problems. Building a voting classifier, receiving a list of classifiers, and returning the combined model will be a cumbersome script for us to build. Luckily, Scikit Learn already provides this functionality with the votingclassifier class. The main input with keyword estimators is a list of string estimator tuples. Each string is a label and each estimator is a Scikit Learn classifier. You do not have to feed the classifiers individually as the voting classifier will take care of that for us. In this example, we instantiate a five nearest neighbors classifier called clfknn and decision tree called CLF dt and a logistic regression called clflr. After that we create a voting classifier passing the estimators under labels in as a list. Here we use the labels knn for the finders neighbors, classifier dt for the decision tree, and lr for the logistic regression we use find neighbors to avoid multimodal predictions. The combined model can be fitted to the training data and then be used to make predictions. Remember that fit is called with S train and Y train and predict only with edge test. Then we can evaluate the performance on the Test set, passing yTest and yPrev to the AccuracyScore function. Let's now build our first ensemble models.
