Multiple Factor Analysis -MFA (course 4): compare qualitative variables groups or contingency tables

We have seen in the previous videos how to take into account group structure using a multiple factor analysis when all of the groups contain quantitative variables. We are now going to have a look at some additional stuff, starting with how to take into account groups of qualitative variables. Then, we'll see what to do when one or more groups of variables correspond to one or more contingency tables. And lastly, we'll see which interpretation aids are useful for interpreting the results of an MFA. We will focus on ones which are specifically made for MFA. Consider a data table where all variables are qualitative and all are structured by group. This situation is then similar to that in which the variables are all quantitative. Indeed, we will try to balance the influence each group of variables has in the global analysis And then we will construct specific MFA plots to study the similarities and differences between groups. The individuals plot will be identical to the one obtained in multiple correspondence analysis, and thus it will also show the categories. Remember that a category will be found at the barycenter of the individuals that are part of it. Then we'll be able to construct the plot showing the superimposed representations the groups plot, and the partial axes plot, which will show connections between the global analysis and the separate ones. The methodological approach is therefore the same as before except that, on each group of variables, we use MCA rather than PCA. Thus, for each group, we will construct a complete disjunctive table, then we will weight it as in MCA as a function of the column margins. Then, all that remains is to balance the groups by weighting by the first eigenvalue of their MCA. In our example, we can add the group of qualitative variables, restricted just to the grape variety variable. This will help us to illustrate the outputs obtained when certain groups are qualitative, depending on whether they are active or supplementary. Here, we have the groups plot, where we can see that the grape variety group has coordinates close to 0.4 on both dimensions. This means that this group is related to the first 2 dimensions of the MFA, and therefore that the wines of the 2 grape varieties are distinguished from each other on both dimensions. We can check this on the individuals plot by coloring the wines according to the qualitative variable grape variety. The partial axes plot shows a single dimension for the qualitative group. This is normal because, since there is only one variable with 2 categories, the space generated has only one dimension, that is the total number of categories, here 2, minus the total number of variables, here 1. This dimension is well projected in terms of MFA, so the variability induced by the group of qualitative variables is recovered well by the MFA plane. Next, we can construct the graph of partial points of individuals and categories. Here we show only the partial points for the categories to avoid overloading the plots. We can see that the expert jury tends to separate to a greater extent the Vouvray from the Sauvignon because the partial points are much further apart compared to those for the consumers which are close to the center of the plot. It is also possible to mix the variable types from one group to another and to study, using MFA, data tables in which certain sub-tables are made up of quantitative variables and others of qualitative variables. Within each group, the relationships between variables are studied like in PCA when the variables are quantitative and like in MCA when the variables are qualitative. The influence of each group is then balanced using the MFR weighting, that is the first eigenvalue of the principal component method constructed on the data of each group. So first eigenvalue of the PCA or first eigenvalue of the MCA according to the group's variable type. The associated plots will be the same as those we have already seen, that is plots with individuals and categories, variables plots, and then specific representations including the groups plot, the partial points plot, and the partial axes plot. Let's also take note of the following specific situation: if each group of variables consists of only one variable, quantitative or qualitative, then we arrive at a method called factor analysis of mixed data. This method allows us to analyze simple data tables consisting of quantitative and qualitative variables. The visual outputs are a mix between PCA ones for the quantitative variables and MCA ones with an individuals and categories plot along with a variables plot with the squared of the relationships. That is, the square of the correlation ratios for the qualitative variables and the square of the correlation for the quantitative variables. This multiple factor analysis of mixed data thus balances the influence of each quantitative and qualitative variable. It is quite a useful way to proceed because many data tables contain both quantitative and qualitative variables. MFA can also be extended to contingency tables. Thus, some groups of columns can be contingency tables. All these tables therefore need to have one dimension which is the same. The main difficulty is then to find a weighting for the rows of the table, since the weights of the rows are different from one contingency table to the next. To choose the shared weight for a given row, we can take the sum of the entries of this row across all contingency tables, divided by the total sum of the entries of all the contingency tables. When working with contingency tables, the analysis of the results and the interpretation of the MFA are exactly the same as for the usual MFA. Let's take a close look at a few applications of this. Firstly, there are many examples of this in text analysis, which corresponds to an extension of studying text data with correspondence analysis. The following example involves the same survey but carried out in several countries. Each column corresponds to a question and the rows correspond to the categories of a qualitative variable, age groups for example, and the table entry corresponds to the number of respondents who answered yes to that question. Each country corresponds to a table and we want to look for information common to all countries or specific to certain countries using MFA. Another example is in ecology, with a table matching sites with species gathered over several years. We'd be interested in looking at the evolution of associations between sites and species over time. The sites, corresponding to rows, could be the same from year to year, while some species, the columns, may disappear and no longer be present some years. We also note that MFA can simultaneously deal with frequency tables, tables with quantitative variables, and tables with qualitative variables. The row weights are then calculated using the frequency tables. Now let's go back and take a look at interpretation aids which are useful for all types of principal component methods including PCA, MCA, and MFA. First, it's possible to add supplementary information, supplementary individuals or supplementary variables, and of course, in MFA, supplementary groups of variables. These additional elements do not participate in the construction of the dimensions but may be of help in the interpretation. In our wine example, we add 3 groups of variables corresponding to the tastings of the 3 juries, But we also add preferences data. A new group of consumers tasted the wines and gave them appreciation scores: 0 for I do not like at all this wine, up to 10 for I very much appreciate this wine. A data table can then be constructed on this, with the 10 wines as rows and the 60 tasters as columns. Each table entry corresponds to the score given to a certain wine by a certain taster. This table can then be juxtaposed with the data table used in the MFA since the rows are the same. We can then ask ourselves the following question: is the appreciation of wines linked to the sensory descriptions? To answer this question, we are going to redo an MFA with taking into account the preferences group made up of 60 additional variables. We can also use the grape variety variable as supplementary information, but we have already seen how to analyze the results of this qualitative group, so we will content ourselves with analyzing the results of the supplementary quantitative group. The preferences group having been added as supplementary information, the plot of the mean individuals and the variables plot obtained by MFA are the same as before. They give the sensory description of the wines constructed from the tasting of the 3 juries. The preferences group has relatively strong coordinate values on the first 2 dimensions of the MFA. This means that, overall, this group is linked to these 2 dimensions and therefore to the sensory description of the wines. That is, wine appreciation is not independent of the sensory descriptions. Now, going into more detail, all of the preferences variables can be represented on the variables plot, but for the sake of readability, we have put them on a second plot. Each arrow corresponds to a tester, and we can see that most of the arrows are pointing towards the bottom left and are rather well represented. This means that the wine to the bottom left is highly appreciated by many testers. This is the Aubusier Silex wine, which is a wine described as sweet by the sweet juries, as can be seen on the sensory variables plot. With MFA, we again find the classic indices for contributions and representation quality. For individuals and variables, these indices are calculated exactly as in PCA. For groups, we can also calculate these indices. First, the contribution of a group is equal to its coordinate value, and not its coordinate value squared, divided by the sum of the coordinate values if we want to express relative contributions. We can therefore directly read off a group's contribution to the construction of an axis using the groups plot. Lastly, the representation quality of a group is measured as usual by the squared cosine of the angle between the vector starting from the center of the cloud and going to the point representing the group in the space, and the vector starting from the center of the cloud and going to the projection of the group onto the dimension. We can then sum the squared cosines over several dimensions to calculate the representation quality in a plane or a subspace. As with other principal component methods, the dimensions can be characterized automatically by calculating the links between each variable and each dimension. This is particularly useful in MFA because the number of variables is often quite large. To describe each dimension using the quantitative variables, we can calculate the correlation coefficient between each variable and the dimension, and then sort them from largest to smallest, keeping only those that are significantly different to 0. In our example here, we only use the active variables, that is the sensory descriptors to characterize the axes. We could also characterize the axes using the supplementary variables, but here the preferences are not very interesting in terms of being able to understand the dimensions. The variable most linked to the 1st dimension, in fact negatively linked to it, is the typicity of the Chenin taste as evaluated by the students. Next up is vanilla odor as evaluated by the experts, which is positively correlated with the 1st dimension. In this way, we have characterized the first dimension. We have therefore found a way to see which of the 57 variables best characterize the dimension. To describe the dimensions using the qualitative variables, we construct a one-way analysis of variance model in which we characterize the coordinates of the individuals, that is the wines, on a dimension according to the qualitative variables. The qualitative variables are then sorted according to the correlation ratios and we keep the ones whose correlation ratio is significantly different to 0. This makes it possible to see overall which variables are most related to the dimensions. In addition, Student's t-tests are performed in order to sort the categories. We can then test whether the coordinate value of a given category is significantly different to 0 or not. Again, here, the categories are then sorted according to the p-values and as a function of the sign of the coefficient. In our example, the grape variety variable is related to the first 2 dimensions. The p-value is very slightly less than 5%, and therefore we reject the hypothesis that the correlation ratio is equal to 0. Recall that here, the grape variety variable is a supplementary one and therefore was not used to construct the dimensions. Thus, the test we have built is valid. On the other hand, for the sensory variables, the tests must be used with more caution because the variables themselves were used to construct the dimensions and are therefore automatically related to them. If we look at the categories, we can see that the Vouvray wines have significantly positive coordinates on the first dimension and significantly negative ones on the 2nd dimension. They are therefore found bottom right while the Sauvignons are found top left. To conclude, here is a slide summarizing the important steps required to implement MFA. To begin, the first question to ask is, is there a group structure in the dataset? Or should we do a simple analysis of the table using PCA if the data is quantitative, or MCA if the data is qualitative. If there is a group structure, which groups should we build? A group corresponds to a subset of the table, and therefore most often to a certain type of information. This choice of grouping is essential because it is used to define the weightings and balance between groups. Then, we have to choose the groups that will be active and the groups that will be supplementary. For the groups of quantitative variables, we then have to decide whether the variables in each group should be normalized or not, depending on the importance that we want to give to each variable within each group. This really means looking at the importance of the variables within each group and not between groups, because the weighting of the MFA by the first eigenvalue of each group means granting the same importance to each group of variables. Then we run the MFA. Next, it's necessary to choose the number of dimensions to be interpreted by looking, for example, at a plot of the eigenvalues of the MFA. Then we begin by interpreting simultaneously the individuals plot and the variables plot before using the specific outputs of the MFA to study the groups overall to see, first, which groups resemble each other with the help of the groups plot, and second, which are the individuals not seen in the same way by each group with the help of the partial points plot. The separate analyses can then be examined in detail to see which are the dimensions of each group that are related to the dimensions of the MFA. Lastly, the indices for quality of representation, contribution, LG, or RV coefficients can be used to enrich our interpretations. Some software packages have a function for running MFA. The most advanced one is the MFA function in the FactoMineR package since it allows us to do MFAs with any type of groups of quantitative variables, groups of qualitative variables, and contingency tables. Each group can be either active or supplementary. Via this function, you can also access all of the interpretation aids described in this course. So let's conclude. MFA is a multi-table method useful for analyzing tables with the same rows. The groups, or tables, must have quantitative or qualitative variables or represent frequency tables. What MFA does well is to balance the influence of each group, then represent the information provided by each group in a shared frame of reference. An MFA provides 2 main plots, an individuals plot and a variables plot, which are classical outputs in principal component methods. These plots are analyzed the same way as in PCA and MCA. But MFA also provides its own specific outputs, which contribute to its importance and relevance. These specific outputs make it possible to compare information provided by each table. This information can be compared in a very general way thanks to the groups plot. We can then see if overall the groups have the same dimensions and if the dimensions provided by the MFA are present in each group. The information provided by each group can then be compared more precisely by the representation of dimensions in the separate analyses. Essentially, we can see whether the dimensions of variability of a given table obtained by PCA, MCA, or CA are related or not to the shared dimensions, that is to the MFA dimensions. Lastly, the information provided by each group can also be compared in terms of each individual by means of the partial points plot. Here is a book written by Jérôme Pagès dedicated to multiple factor analysis. In this book, you can find more details on the mathematical justifications of the methods we have presented here and also plenty of examples which use MFA. We have now reached the end of the multiple factor analysis course. So now you can go on and see how to implement multiple factor analysis using FactoMineR. Don't forget to have a go at the quizzes and exercises that come with the course. Good luck!