Learn what hyperparameters are and how they differ from model parameters. The video shows how to inspect and override defaults in scikit-learn models like Random Forest and Logistic Regression, explains which ones affect performance versus computation, and covers why tuning matters for different problems.
Chapters
In the previous lesson you learned what parameters are. You will now learn what exactly hyperparameters are, how to find and set them, as well as some tips and tricks for prioritizing your efforts. Let's get started. Hyperparameters are something that you set before the modeling process begins. You could think of them like the knobs and dials on an old radio. You tune the different dials and buttons and hope that a nice tune comes out. The algorithm does not learn the value of these during the modeling process. This is the crucial differentiator between hyperparameters and parameters. Whether you set it or the algorithm learns it and informs you. We can easily see the hyperparameters by creating an instance of the estimator and printing it out. Here we create the estimator with default settings and call the print function on our estimator. Those are all our different knobs and dials we can set for our model. There are a lot, but what do they all mean? For this we need to Turn to the SciKit learn documentation. Let us take the example of the Nestimators hyperparameter. We can see in the documentation that it tells us the data type and the default value and it also provides a definition of what it means. We can set the hyperparameters when we create the estimator object. The default number of trees seems to be a little low, so let us set that to be 100. Whilst we are at it, let us also set the criterion to be entropy. If we print out the model, we can see the other default values remain the same, but those we set explicitly overrode the default values. What about our logistic regression model? What are the hyperparameters for that we follow the same steps. Firstly we create a logistic regression estimator. Then we print it out. We can see there are less hyperparameters for this model than for the random forest. Some are more important than others. But before we outline important ones, there are some hyperparameters that definitely will not help model performance. These are related to computational decisions or what information to retain for analysis with the random forest classifier. These hyperparameters will not assist model performance. How many cores to use will only speed up modeling time. A random seed and whether to print out information as the modeling occurs also won't assist. Hence some hyperparameters. You don't really need to train during your work. There are some generally accepted important hyperparameters to tune for a random forest model. The N estimators. How many trees are in the forest should be set to a high value 500 or 1000 or even more is not uncommon, noting that there are computational costs to higher values. The max features controls how many features to consider when splitting, which is vital to ensure tree diversity. The next two control overfitting of individual trees. The criterion hyperparameter may have a small impact, but is not generally a primary hyperparameter to consider. Remember, this is just a guide and your particular problem may require attention to other hyperparameters. There are hundreds of machine learning algorithms out there and learning which hyperparameters matter is knowledge you will build over time from a variety of sources. For example, there are some great academic papers where people have tried many combinations of hyperparameters for a specific algorithm on many data sets. These can be a very informative read. You can also find great blogs and tutorials online and consult the Scikit Learn documentation. Of course, one of the best ways to learn is just more practical experience. It is important you research this yourself to build your knowledge base for efficient modeling. Let's explore some hyper parameters.
