Multiple Correspondence Analysis (part 2/4: Visualizing the point cloud of individuals)

We have now seen what the data looks like in MCA, and the questions we want to answer. In MCA, like in all principal component methods, we are going to build point clouds: point clouds for the rows and point clouds for the columns. We are going to begin now with the point cloud for the rows, that is, the cloud of individuals. In a complete disjunctive table, individuals are represented as rows. This means a set of k numbers, so a point in a k-dimensional space. Each dimension corresponds to a category. In MCA, each category is associated with a weight proportional to the number of individuals in it. As the sum of weights has to equal 1, the weight for the kth category is Pk over J. The point Mi has the coordinate x_i,k, and we have seen that each individual is given a weight 1/i. When looking at the whole set of points, we call the cloud N_i. This cloud N_i has a center of gravity G_i, which is in fact the origin, as the variables have been centered. Now, we bring in the point i' so that we can look at the distance between the individuals i and i prime. The square of this distance is written: the sum of the squares of the differences of the coordinates, where each square of the differences of the coordinates is weighted corresponding to the weight of that category, that is p_k over J. If we express this distance as a function of the complete disjunctive table, we get the following formula. Which, after simplification, looks like this. We write this distance in this way when we are performing a multiple correspondence analysis as a correspondence analysis on the complete disjunctive table. Now, a few remarks on the choice of this distance measure. If 2 individuals are in the same set of categories and thus have exactly the same profile, the distance between them is 0. If 2 individuals share many categories, the distance between them will be small. If 2 individuals share many categories except one, a rare one, which one of them is in, then the distance between them will be relatively large due to the PK which will be small for one of them. If 2 individuals share a rare category, the distance between them will be relatively small due due to this rare shared similarity. Now let's calculate the distance of a point from the origin. The square of the distance between an individual and the origin is equal to the sum of the squares of the weighted coordinates. If we work from the complete disjunctive table, we have the following formula which, after simplification, looks like this. We can clearly see the sum of the y_ak over PK in it. This means that the sum gets bigger when the categories possessed by an individual are rarer, that is, when that individual is associated with small PKs. Basically, this means that the more an individual possesses rare categories, the further they are from the origin. To finish, let's calculate the total inertia of the point cloud NI. First, we start by calculating just one point's inertia, that is, its weight multiplied by its distance to the origin. Then, for the total inertia of the cloud, we sum over all the individuals. When all the calculations are done, we find that the total inertia is K over J minus 1. This value doesn't actually depend on the content of the table itself, but only on some details of its format, that is, the number of categories and the number of variables. This is different from the result in correspondence analysis where the total inertia is phi squared and therefore measures the deviation from independence. On the other hand, this result is analogous to the one from principal component analysis with normalized data where the total inertia is equal to the number of variables and doesn't depend on the table's content, only on its format. So we have defined what the point cloud of individuals is. Like in any principal component methods, we are now going to perform a projection of this point cloud into a smaller dimensional space. Often we will even simply project into a plane, that is 2 dimensions. In order to do this, we have to apply factor analysis to the cloud, that is project the cloud onto a sequence of orthogonal axes with maximal inertia. In terms of the first 2 dimensions obtained, we will end up with a factor plane for the individuals which is the best 2D representation of them. Looking now at the dataset on French leisure activities, the first 18 variables, each corresponding to one activity, are going to be considered the active ones. So it's these 18 variables that will be used to calculate the distance between individuals. The supplementary variables are treated as such and will be used later to help interpret the results. Like in any principal component method, let's start by looking at the decreasing values of the inertia associated with each subsequent dimension. Here, the bar plot shows that the first dimension has a rather larger inertia than those that follow. This makes us want to try and interpret the first dimension on its own and initially put the others aside. The percentage of inertia associated with this first dimension is 24%. Can we consider this to be a large number or not? Well, recall that we are analyzing fairly complex activity profiles as there are 18 leisure activities and high variability. Among our sample population, there are young people, old people, men and women, and we expect a great diversity of activity profiles. And it would be foolish to expect that all this diversity could be captured with 2 dimensions. If we now look at the 3rd and 4th dimensions, we see that they have comparable and relatively large inertia. In this video, We are not going to go on and analyze the plane defined by the 3rd and 4th dimensions, but this doesn't mean that it can't be usefully interpreted. If we do go up to 4 dimensions, we arrive at 35% of the total inertia. As for the rest of the inertia, it's just a collection of individual variability. In Multiple Correspondence Analysis, as in Principal Component Analysis, the first thing we look at is the general form of the point cloud of individuals. Here, there is nothing particularly unusual about it. It's quite normal. There is not much more to say, really. But you may ask, what would an unusual cloud look like? Here is an example which shows 3 distinct classes of individuals. If something like this appears in the data visualization, Clearly, these 3 classes will be important in the interpretation step. This is why it's always important to first look at the general shape of the point cloud of individuals, even if we end up simply concluding that it's an entirely unremarkable point cloud showing nothing out of the ordinary. Here is another example of a point cloud of individuals with a quite particular shape which comes up quite often in MCA, a horseshoe shape. This is known as the Goodman effect or the horseshoe effect. When the horseshoe effect is in action, what tends to happen is that on one dimension, here the horizontal one, individuals are placed according to categories with increasing values, for example, lowest values categories to the left, medium in the center, high to the right, and on the 2nd dimension, here the vertical one, separate outlier individuals, which take very high or low values, from more average individuals. To get a better understanding of the main dimensions of variability, we can use qualitative variables from the dataset. One idea is to color the points representing individuals in terms of the categories of a chosen variable. For example, here, for the gardening variable, those that garden are shown in red, those that don't in black. We see that the red points are mostly at the top of the plot, while the black ones are mostly at the bottom. It therefore seems that the 2nd dimension separates individuals that garden from those that don't. Of course, we can do this kind of coloring exercise for each variable. But this quickly becomes tiring. However, this leads us to the more general idea of using the variables' categories, or the variables themselves, to try and characterize and interpret the point cloud of individuals. First, let's think about how we can use the categories of a given variable. A category can be seen as a group of individuals, so it makes sense to show this directly on the graph of the individuals. To do so, we can put a point representing a category at the barycenter of the individuals with that category. For the gardening variable, we have therefore 2 points: one for gardening yes, in red, and one for gardening no, in black. Just remember that the origin of the cloud is at the barycenter of a given variable, Since each of its categories has a weight proportional to the number of individuals in it. Here, the gardening no point is closer to the origin than gardening yes. This means that there are fewer people that do gardening than don't. 3,356 has said they do, and 5,047 says they don't. And these numbers are proportional to the distance between the tech categories and the origin. Hence, categories with fewer adherents will be further from the origin. In this plot, we show all of the categories of all the variables, along with the points for the individuals. As the categories' points represent means, because they are barycenters, they are all quite close to the center of gravity. The yes categories are in red and the no categories in black. If we forget about the points for the individuals and zoom in, this is what the categories look like on the first 2 dimensions. The first thing we notice is how the yes activities are all clustered together, as are the no categories. We should say straight away that this is no accident. In the indicator matrix, nothing sends a signal that the yes categories are any different to the no ones. This is therefore evidence of a real phenomenon in the data. This separation is not really with respect to the first dimension nor the second, but more like a diagonal across the 2. Let's take some time and try to interpret what this means. To help us, let's look at a few of the individuals that seem well separated by this diagonal. We can take individuals 5938 and 2432 on one side, and 8325 and 203 on the other. The plot showing the categories shows us that individuals 5938 and 2432 are on the side of the yeses, which suggests they participate in a lot of leisure activities. In contrast, individuals 8325 and 203 are far on the no side, suggesting that they don't participate in many leisure activities. If we look back at the data table to check, this information jumps out at us. 5938 and 2432 participate in nearly all the activities, whereas 8325 and 203 don't do any, unless you count watching a lot of TV as an activity. This contrast, defined by the diagonal, thus clearly represents how active people are and separates the active ones from the less active. Now that we have interpreted the first diagonal, let's look at the other diagonal and try to understand what it means. The group of categories to the bottom right includes cinema yes, computer yes, shows yes, sport yes, playing a musical instrument yes, gardening no, cooking no, knitting no. Over on the top left, it's the opposite: cinema no, computer no, shows no, sport no, Playing a musical instrument, no. Gardening, yes. Cooking, yes. Knitting, yes. Let's again take a few individuals to try and understand what is going on with this diagonal. Individuals 255 and 6766 do the activities that tend to be associated with younger active people and not the more home-based activities like gardening and cooking. In contrast, individuals 1143 and 5676 participate in the home-based activities associated with older people and none of the active outside activities. We should note that each of these 4 individuals do about the same number of activities in total, which explains why they have about the same coordinates with respect to the diagonal. We are in fact in the process of commenting on a diagonal which is orthogonal to the first and thus answering the question: for those doing the same number of activities, what separates them into groups? Overall, what we can say is that variability among the individuals can be understood using these 2 diagonals. One diagonal separates individuals doing many activities from those doing few. And the other diagonal separates the types of activities, seeming to separate younger person activities from older person activities. Later on, we're going to see how to project the categories of the supplementary variables onto the graph in order to answer the following question: are the dimensions of variability that we have just seen related to some of the supplementary variables, like age for example? For now, let's go back to the graph of the individuals, colored according to the gardening variable. Let's see what happens when we project them all onto one of the dimensions. Here, they have all been projected onto the 2nd dimension. What we can try to do is calculate an indicator function of the link between the coordinates of the individuals on the 2nd dimension and the qualitative gardening variable. An indicator of the link between a quantitative and qualitative variable is the correlation ratio between them. This is the indicator used in one-way analysis of variance. Often, we work with the square of the correlation ratio, which is equal to the percentage of variability of the quantitative variable explained by the qualitative variable. This indicator varies between 0 and 1. It's equal to 0 if there is no link between the 2 variables, that is, if the means of the quantitative variable for each category of the qualitative one are equal. And it's equal to 1 if the means are different and the within-category variability is 0, that is, if all the individuals from the same category have exactly the same value of the quantitative variable. Here, the square of the correlation ratio between the gardening variable and the 2nd dimension is 0.453. This plot doesn't allow us to entirely see this, but it does give us an idea of the separation of the points from the different categories of the gardening variable. And it must be said that a squared correlation ratio of 0.453 really does represent a good separation of the points from the 2 categories. The square of the correlation ratio between gardening and the 1st dimension is 0.047. This is quite small and, as we can see on the plot, the projections of the red and black points seem to be quite mixed. There is no real separation between them. So what we do is calculate the squares of the correlation ratios of each variable with the 1st and 2nd dimensions, which then helps us to plot a graph showing the variables. Here, all the points are located in the 1 by 1 square with non-negative values. Note that on this graph, a variable with 3 or more categories could well have a coordinate of 1 in both the first and second dimensions. For the gardening variable, we place its point at 0.047 on the first dimension and 0.453 on the second. In order to begin the interpretation, we can look at the variables with the largest correlation ratios for the first and second dimensions. But we still take into account the actual value of the squared correlation ratios because even if a variable is the one the most connected to a certain dimension, it won't be seen in the same way if its value is 0.3 rather than 0.9, for instance. In our example, the connections are not very strong, not very close to 1. If we zoom in to the graph, we see that the exhibition, show, computer, and cinema variables are linked to the first dimension. Their squared correlation ratios vary between 0.3 and 0.4. These are not very big, but given the large number of individuals, these squared correlation ratios are very very significantly different to 0. It's therefore OK to take into account these variables when interpreting the first dimension and the plane defined by the first 2 dimensions. Be careful nevertheless not to interpret the p-values exactly like in a classical test. Indeed, the principal components have been constructed using the qualitative variables. So we shouldn't be surprised that connections can be found and that some of the squared correlation ratios are large. We made this happen. In other words, these p-values can be interpreted in the same ways as for classical tests, except for supplementary variables. Still, the information we get from them can be quite useful. We also make note of a very interesting property. MCA looks for principal components that are as related as possible to the variables in the dataset, in terms of squared correlation ratio. Concretely, to find the S dimension, we look among the orthogonal directions to the previously found dimensions for the one that's most related to the variables of our dataset, where most related means the one that maximizes the sum of the squared correlation ratios. We'll see in the next video how to build a point cloud of categories and how to get an optimal representation of it for a given number of dimensions. We'll also underline the link between this representation and the way we have plotted and represented the individuals. It's the link between these 2 representations that makes MCA so useful.