In this instructional video, the presenter applies hierarchical clustering to a real dataset consisting of mean monthly temperatures for 23 European capitals, using centered and standardized variables, Euclidean distance, and Ward's linkage. The resulting dendrogram reveals natural groupings such as Sofia‑Sarajevo and a Mediterranean cluster (Athens, Lisbon, Madrid, Rome). A companion bar plot displays the loss of between‑class inertia as clusters are merged, illustrating how the total inertia (equal to the number of variables, 12) is partitioned across merges. The speaker explains how to interpret the size of each bar, the ratio of between‑class inertia to total inertia, and compares these ratios to the variance explained by the first principal components of a PCA. By examining inertia loss, tree branch lengths, and the interpretability of resulting groups, the video outlines practical strategies for deciding how many clusters to retain, emphasizing the trade‑off between overly coarse groupings that hide heterogeneity and overly fine groupings that lack meaningful distinction. The discussion concludes with guidance on when to stop merging clusters based on data size, variability, and the ability to meaningfully characterize each class, setting the stage for the next video on fixed‑class partitioning methods.
Chapters
In the previous video, we saw how to build a hierarchical tree. Now, we're going to apply this to a real example. Our illustration involves a dataset of meteorological data. The rows, or statistical individuals, here are 23 European capitals. The columns are quantitative variables, giving the mean monthly temperatures. These have been calculated and averaged over 30 years. So, for example, in Amsterdam, in January, on average, it's 2.9 degrees Celsius. This value of 2.9 is the mean over all the days of January and over 30 years. We have therefore 12 variables, one for each month of the year, and added to this, one qualitative variable corresponding to the region of Europe the capital is in. North, south, east, or west. We are going to construct our hierarchical tree using only the temperature data. Here, constructing the hierarchical tree means looking to group together capitals with similar temperature data and then trying to find characterizing features of each group. A little note before we continue: we are going to work with the centered and standardized versions of the variables rather than the raw data to help give the same importance to each variable when building the hierarchical tree. Here is the hierarchical tree we get using the Euclidean distance and Ward's criterion. We see, for example, that Sofia and Sarajevo have similar weather profiles. The temperatures in these 2 towns are quite similar throughout the year. Another fairly homogeneous group is visible: Athens, Lisbon, Madrid, and Rome. Going further in, we see that Madrid and Rome are the most similar to each other in this group. We can also get an idea of the distance between cities and groups of cities. In the bar plot at the top right, we show the evolution of the inertia for different partitions. Let's have a closer look at this plot. It shows the loss of between-group inertia when we group together 2 classes. More precisely, it shows the loss of inertia when moving from 23 to 22 classes, from 22 to 21, and so on, right down to moving from 2 classes to 1. If we calculate the sum of the losses in inertia, we get the value 12. 12 corresponds to the sum of the variances of the variables in the dataset, Because here we have 12 variables, and they have been centered and standardized. Thus, by taking the sum of the losses of between-class inertia, we indeed find the total inertia, which is 12. Let's now try to understand what each bar in the bar plot is telling us. The big bar, on the left, shows the loss of between-class inertia when moving from 2 classes to 1. The loss of inertia is 6.76, which is rather large. This means that this attempt at grouping together 2 classes brings together quite different individuals. We really don't want to do this. The red bar shows the loss of inertia when moving from 3 to 2 classes. This loss of between-class inertia is 2.36. This is still quite large, and again, we'd rather not group together the 3 classes into Looking at the plot from the other end of the scale, we see that moving from 23 to 22 classes, from 22 to 21, etc., involves very small losses of between-class inertia, so these groupings are quite acceptable. So, the question hanging over us is: how far should we go when grouping together classes And when should we stop? Another way of saying this is, how many classes should we have? In our example, how many groups should we retain? 2, 3, 4? This is a very good question. If we choose the cut here in orange, we end up with 2 groups. The between-class inertia, as we've seen, was 6.76. As compared with the total inertia 12. This means a ratio of between-class inertia over total inertia of 56%. Thus, by separating the cities into 2 groups, those in green, Athens, Lisbon, Madrid, and Rome, from those in orange, Reykjavik, Moscow, etc., we recover 56% of the information contained in the dataset. We are tempted to call this a raw but relatively useful 2-class summary of the data. But what should we compare this 56% with? Well, we could compare it with the percentage of variability found in the first PCA dimension in this plot. This represents around 83% of the information in the dataset. From the clustering, separating just the green cities from the orange ones, we get 56% of the information, so rather less than the PCA's first dimension. Indeed, the first dimension gives more detailed information. It separates Athens from Lisbon and Madrid, with Athens pushed further out from the center. Similarly, Helsinki is pushed out much further than Paris, whereas in the classifier, these 2 cities are in the same class. Clearly, the classifier gives a less refined summary of the data than the first PCA dimension. Now, if we separate the orange cities, the ones that are cold, into 2 groups, we will have a clustering with 3 groups in total. Moving to 3 classes helps us not to lose the inertia lost when moving from 2 to 3 classes. We recover a between-class inertia of 2.36, which is about 20% of the information. Separating the cold cities into 2 groups gives us back 20% of the total information. So, with a 3-class classifier, we recover 56% plus 20%, that is 76% of variation in the data. 3 classes give us 76% of the total variation. Note that with PCA and 2 dimensions, we recover about 98% of the information. The choice of the number of classes is important because partitioning with too few classes risks leaving classes which are not at all homogeneous. On the other hand, partitioning with too many classes risks creating classes that are not very different from each other. Clearly, the big question is, how many classes should we stop at? One possible strategy is to choose the number of classes by looking at the tree. We might consider cutting the tree somewhere where the branches are quite long. We can also look at the plot with the bars and choose to stop the grouping together at the point where the jump from one bar to the next becomes small, meaning that little information is gained and it's no longer useful to group together any more classes. Or, looked at from the other end of the tree, there is no point continuing to cut classes in 2 when the jump from one bar to the next gets small. Choosing a good number of classes also depends on the data itself, and the number of individuals. If we have a survey with thousands of individuals, it may be useful to have 5, 6, or even 10 or more classes. And if we have much less individuals, it's probably better to have fewer classes. As for the data itself, if we have a huge variety of very different individuals, it's better to have more classes. When all is said and done, The most important criteria is perhaps: after building the classes, can we interpret what they mean or not? Can we understand and characterize each class? It's useless to divide a class in 2 if we don't really understand what it is that makes each class different from the other and similar within its own members. In this example, we have seen how hierarchical clustering helps us to suggest the number of classes to keep. In the next video, we'll look at how to build a partitioning method when the number of classes is fixed.
