Decision trees predict outcomes by splitting data into branches. Boosting combines many weak trees into a strong predictor by learning from mistakes. XGBoost, a popular boosting library, wins machine-learning competitions and stays interpretable unlike neural networks.
Dear fellow scholars, this is two minute papers with a decision tree is a great tool to help making good decisions from a huge bunch of data. The classical example is when we have a bunch of information about people and would like to find out whether they like computer games or not. Note that this is a toy example for educational purposes. We can build the following tree. If the person's age in question is over 15, the person is less likely to like computer games. If the subject is under 15 and is a male, he is quite likely to like video games. If she's female, then less likely. Note that the output of the tree can be a decision like yes or no. But in our case we will assign positive and negative scores instead. You'll see in a minute why that's beneficial. But this tree was just one possible way of approaching the problem and admittedly not a spectacular one. A different decision tree could be simply asking whether this person uses a computer daily or not. Individually, these trees are quite shallow and we call them weak learners. This term means that individually they are quite inaccurate but slightly better than random guessing. And now comes the cool part. The concept of tree boosting means that we take many weak learners and combine them into a strong learner. Using the mentioned scoring system instead of decisions also makes this process easy and straightforward to implement. Boosting is similar to what we do with illnesses. If a doctor says that I have a rare condition, I will make sure and ask at least a few more doctors to make a more educated decision about my health. The cool thing is that the individual trees don't have to be great. If they give you decisions that are just a bit better than random guessing, using a lot of them will produce strong learning results. If we go back to the analogy with doctors, then if the individual doctors know just enough not to kill the patient, a well chosen committee will be able to put together an accurate diagnosis for the patient. An even cooler adaptive version of this technique brings in new doctors to the committee according to the deficiencies of the existing members. One other huge advantage of boosted trees over neural networks is that we actually see why and how the computer arrives to a decision. This is a remarkably simple method that leads to results of very respectable accuracy. A well known software library called XGBOOST has been responsible for winning a staggering amount of machine learning competitions in Kaggle. I'd like to take a second and thank you fellow scholars for your amazing support on patreon and making 2 minute papers possible. Creating these episodes is a lot of hard work and your support has been invaluable so far. Thank you so much. We used to have three categories for supporters. Undergrad students get access to a Patreon Only activity feed and get to know well in advance the topics of the new episodes. PhD students who are addicted to two minute papers get a chance to see every episode up to 24 hours in advance. Talking about committees in this episode, full professors form a committee to decide the order of the next few episodes. And now we introduce a new category, the Nobel Laureate Supporters in this category can literally become part of two Minute Papers and will be listed in the video description box in the upcoming episodes. Plus all of the above. Thanks for watching and for your generous support and I'll see you next time.
