The video presents a thorough tutorial on Principal Component Analysis (PCA) aimed at exploring, visualising, and interpreting large multivariate datasets. It begins by defining the data structures suitable for PCA (individuals in rows, quantitative variables in columns) and enumerates diverse application domains (sensory analysis, ecology, economics, genetics, sociology). Using a concrete wine‑sensory dataset (10 wines × 27 attributes plus preference scores and a qualitative label), the speaker illustrates every step of a PCA workflow. The lecture covers: (1) geometric view of individuals (points in a K‑dimensional space) and variables (points in an I‑dimensional space); (2) similarity via Euclidean distance, the need for centering and optional standardisation; (3) the concept of inertia (multivariate variance) and how PCA seeks orthogonal axes that maximise projected inertia, derived mathematically by SVD or eigen‑decomposition; (4) construction of low‑dimensional (2‑D) views that preserve distances as much as possible; (5) interpretation tools – correlation circle, loadings, scores, squared cosines (cos²), contributions of individuals and variables, and quality of representation; (6) handling supplementary quantitative and qualitative elements, including how to project them without influencing component construction; (7) practical criteria for selecting the number of components (scree plot, inertia percentages, statistical tests, cross‑validation); (8) diagnostic checks (contributions, cos², variable representation quality) and the importance of returning to raw data for validation; (9) a concise R implementation using the FactoMineR package, with code snippets for reading data, scaling, running PCA, visualising individuals/variables, and obtaining automatic dimension descriptions. The lecture concludes with recommendations for further reading and resources. Throughout, the speaker adopts an instructional, measured tone, emphasizing practical guidance while remaining neutral about theoretical nuances.
Chapters
Hello everybody. In this video, I will present you a method that allows to study and visualize large datasets: the principal component analysis. The outline of this talk is as follows. I will first define what types of data on which we can achieve a principal component analysis. Then I will give examples of fields in which the principal component analysis can be performed. And then I'll define some useful notations. Then I will focus on the study on the individuals, then on the study of the variables, and finally I will spend time on how to interpret the results of such analyses and therefore provide helps for the interpretation. So what kind of data? Principal component analysis applies to data tables where rows are considered as individuals and columns as quantitative variables. Let xik be the value taken by individual i for variable k. i varies from 1 to I. I is the number of individuals. And k from 1 to K, the number of variables. Overline xk denotes the mean of variable k, calculated over all the individuals. And Sk is the uncorrected standard deviation of the sample of variable k. Here we use 1/i, which means that the standard deviation is calculated on the data and we do not try to get an estimate of the standard deviation of a population. Data tables with individuals in rows and variables in columns are numerous and we can perform PCA on very diverse datasets. I have listed a few application areas with examples. Here is a first example with sensory analysis, where products are described by different variables, often called sensory attributes, such as acidity, bitterness, sweetness, and so on. Therefore, there is a table with various products, such as wines, and each wine is going to have its score for acidity, for bitterness, for sweetness. So the aim of PCA is to study this data table. Here is an example in ecology where individuals are different rivers and variables are different pollutants. So we have a data table that crosses rivers in rows and pollutants in columns. In the economic fields, we can have years in rows and different economic indicators as variables. We can then track the the evolution of economic indicators through the years. The rows can be years, but it can also be countries, and we will compare the economic situation of several countries. Very often, in genetics, patients are described by genes. The dataset is very large since there are lots of genes. And in sociology, we can have different social classes and different activities. And the value is the average time spent by individuals of a social class for a given activity. Such tables with individuals in rows and variables in columns are very common in many fields. Then I will take an example to illustrate this course. So in rows we have 10 wines and in columns 27 quantitative variables that are sensory attributes such as sweetness, bitterness, odor, fruity, and so on. The values in the data table correspond to the average of the score given by several judges for a same wine and a same descriptor. So for example, the wine S. Michaud has a mean score of 4.3 for the odor fruity. It is the average on all the judges. And so it is the same for all the wines and all the attributes. We also have 2 quantitative variables that correspond to preference scores preference for odor, and overall preference. And we also have a qualitative variable that corresponds to the label of the wines. There are 2 labels: the first one is Sauvignon and the second one is Vouvray. We will see how we can consider this information in the analysis. So the aim of our PCA is to characterize the wines according to their sensory descriptions. So we will first focus on the 27 sensory attributes to describe the wines. This dataset can be studied in different ways. We can see this data table as a set of rows, so we can try to see differences from one row to another, or similarities from one row to another. We can also see this data table as a set of columns and see the similarities or the links between columns. For the study of individuals, that is to say for the study of the rows, I need to know when 2 individuals are close and when they are different from the point of view of all the variables. If there are many individuals, I'd like to propose a typology of the individuals. Which are the most similar individuals and the most dissimilar? Are there any groups of individuals which are homogeneous in terms of their similarities? In addition, we should look for common dimensions of variability which oppose extreme and intermediate individuals. In the example, 2 wines are considered similar if they were evaluated in the same way according to all the sensory descriptors. In such cases, the 2 wines have the same main dimensions of variability and are thus said to have the same sensory profile. More generally, we want to know whether or not there are groups of wines with similar profiles, that is sensory dimensions which might oppose extreme wines with more intermediate wines. Following the approach taken to study the individuals, might it also be possible to interpret the data from the variables? What are the variables that provide similar information? So between variables, rather than similarity, we talk about relationship. And the best-known relationship between variables is a linear relationship. So PCA focuses on the linear relationships between variables. More complex links also exist, such as quadratic relationships, logarithmic, exponential functions, and so forth, but they are not studied in PCA. This may seem restrictive, but in practice, many relationships can be considered linear, at least for an initial approximation. In exactly the same way as for constructing groups of individuals, creating groups of variables may be useful with a view to synthesize. When confronted with a very small number of variables, it is possible to draw conclusions from the correlation matrix which groups together all of the linear correlation coefficients between pairs of variables. However, when working with a great number of variables, the correlation matrix groups together a large quantity of correlation coefficients, and it is therefore essential to have a tool capable of summarizing the main relationships between the variables in a visual manner. So the aim of PCA is to draw conclusions from the linear relationships between variables by detecting the principal dimensions of variability. As you will see, this conclusion will be supplemented by the definition of the synthetic variables offered by PCA. It will therefore be easier to describe the data using a few synthetic variables rather than all the original variables. In the example, the correlation matrix brings together the 351 correlation coefficients, so grouping the variables from the correlation matrix is very tedious. The average is a synthetic variable, but this indicator is defined a priori. Here, we want to define an indicator from the data, so a posteriori. So obviously, since we work on the same data table that can be seen by rows or by columns, it exists a link between these 2 studies. So studying jointly the individuals and the variables can only reinforce their respective interpretations. When studying individuals, I built groups of individuals, and then I want to characterize the different groups of individuals, the different classes. And to characterize them, I would like to use variables. For instance, I want to say that these products are similar because they are acidic and bitter, whereas others are sweet. So for that, I need an automatic procedure, especially if I have a lot of variables. Similarly, when there are groups of variables, it may not be easy to interpret the relationships between many variables, and we can make use of specific individuals, that is, individuals who are extreme from the point of view of these relationships. For instance, the link between size and weight can be illustrated by the opposition between 2 extreme individuals: a dwarf and a giant. So the issue of PCA is to explore and to describe the data. We can visualize the data with simple graphics and the visualization of the data table gives a synthesis, a summary of the data table. So we summarize, we synthesize the information contained in a data table by viewing it. Then in more details, I said that we can see our data table as a set of rows or a set of columns. If I see it as a set of rows, I'm studying individuals. If I see it as a set of columns, I'm studying variables. Studying individuals means that I am considering a cloud of points, a point corresponding to an individual. This cloud of points evolves in a space with many dimensions. If there are k variables, the cloud of points evolves in a k-dimensional space. Studying variables, a variable is a point in a space with i dimensions. There are i coordinates for each variable. Each variable is a point in a space with i dimensions, so I will see a cloud of variables that evolves in a space R^i. So let us now study the individuals. I said that an individual is a row of the data table and therefore a point in a k-dimensional space. If k is equal to 1, there is only 1 dimension and it is easy to represent the individuals. I put them on an axis according to their value. If there are 2 variables, I can build a classical scatter plot. As before, a linear regression. If there are 3 variables, it becomes a little bit more difficult, but there are some software that can view a dataset in 3 dimensions. The software moves the cloud of points, and we will have an idea of the point cloud by moving it in 3 dimensions. But if there are 4 variables, or more than 4 variables, it is impossible to represent a cloud of points in K dimensions. It is impossible to represent the dataset in 4 dimensions, but the mathematical concept behind is simple. There are K coordinates for the K dimensions. Now, how are we going to be able to see the cloud of individuals if the cloud of individuals evolves in a very large space? For that, we will start by defining the notion of resemblance between 2 individuals. When can we say that 2 individuals are similar? 2 individuals are similar if they are close. So we can define the squared distance between 2 individuals, thanks to Pythagoras, by the sum of the squared deviations on each variable. So this is a straightforward definition of the concept of similarity. And therefore, when I finally want to study the cloud of individuals, I just have to study the shape of the cloud. So analyzing the distances between individuals is tantamount to studying the shape of the cloud of points. Here we have a 2-dimensional representation of a cloud of points, a cloud of birds, that evolves in 3 dimensions. So individuals evolve in a space that is large, and we want to visualize it with a 2-dimensional graphical representation. With PCA, we are going to study the shape of the cloud. To study the shape of the cloud, we have to talk about centering and data reduction or data standardization. Centering data is tantamount to translating the cloud of points in such a way that the center of gravity of the cloud coincides with the origin of the graph. The shape of the cloud of points points remains the same even when translated. Centering presents technical advantages and is always conducted in PCA. So now we can see whether or not it is necessary to reduce the data. In the graph on the right, we have a representation of individuals with their size and their weight. The size is in centimeters and and the weight is in quintal. Note that data are centered. We can see that the cloud of points is very elongated and vertical. In the diagram in the middle, the size is in meters and the weight in kilograms. The shape of the cloud is very different from the previous one because we have a very elongated and horizontal cloud of points. And we say that we wanted to study the shape of the cloud of points. We see that according to the chosen units, quintal and centimeters or meters and kilograms, the shape of the cloud is very different. How can we handle this? An idea is to reduce the data, to standardize the data. If variables are standardized, variables have no units, they are therefore comparable. So the data reduction is essential if the units of measurement are different from one variable to another. Now, if the variables are in the same units, variables can be reduced or not. We will see that the reduction allows to give the same importance to each variable. Without reduction, variables are all the more influenced as they have a large variance. So it can be discussed if it is necessary to standardize or not in this case. In the example of this course, we will reduce the dataset. The standardized dataset already gives some information. For instance, we can see that the wine has a very small score for bitterness and aroma intensity. The values less than -2 mean that they are below the average of the variables, and they are quite extreme. We know that when the values are following a normal distribution, they are between -1.96 Here it is unclear that the data follow a normal distribution, but a standardized value less than -2 is very extreme. So the wine is particularly not bitter and with a very small aroma intensity. The wine Aubusière Silex is particularly sweet and not acid. So PCA will analyze these data. This standardized data table. Then analyzing this standardized data table means that we want to view it. It is difficult to see this table since there are 27 variables. I cannot visualize too many dimensions, so I will seek to have an approximated viewpoint of the cloud of points. For a representation to be successful, it must select an appropriate viewpoint More generally, PCA means searching for the best representation space of reduced dimension, thus enabling optimal visualization of the shape of a cloud with k dimensions. We often use a planar representation alone, which can prove inadequate when dealing with particularly complex data. An appropriate viewpoint allows to have an idea of the distances between individuals, and these distances between individuals should be distorted as little as possible. Thus, the viewpoint will reproduce the shape of the cloud of points as closely as possible. So you can see that on a small 3D animation. Then there is a cloud of points in 3 dimensions. I will move this cloud of points to see it. So this cloud of points moves. You have an idea of the cloud of points. It is possible to have only one 2-dimensional representation of this cloud of points. So this is a first proposition for the viewpoint of the 3D cloud. Therefore, here is a second proposition. And here is a third proposition. And so the question is, among these 3 propositions, which one do you choose to have the best understanding of the cloud of points? I can just have a single photograph of my 3D cloud. Which one will I choose? I would choose the 3rd proposition. So why? Because it separates points. It seems to see better the distances between individuals. This is the intuition that told us to separate the points. And this intuition is good. So the viewpoint quality is the one that best reproduces the shape of the cloud. A viewpoint is good if one visualizes the diversity, the variability in the data, and a viewpoint is especially good if it does not disturb too much the distances between individuals. Distances are calculated between individuals in a space with many dimensions, and I put them in a 2-dimensional space. So I modified the distances between individuals, and I want these distances to be the less distorted as possible. So how can we quantify the quality of a viewpoint? With the concept of dispersion. A cloud of points can be better seen if its projection on the 2-dimensional representation is dispersed, if there is a high variability. This variance is on several dimensions and is thus called inertia. So we will talk about inertia now. The inertia is a generalization of the variance to several dimensions. Here is another small illustration, again a visualization of a 3D cloud. Here I have a picture of an animal that evolves in a 3-dimensional space. And I have a photograph that is only in 2 dimensions. And which is the photography that best reproduces the shape of my cloud in the global space? It is rather the photography on the right. I recognize the camel because I can see more information and in particular I can see 2 bumps and 4 legs. If we go deeper in detail, how can we adjust the cloud of individuals? How can we find the best approximate viewpoint of the cloud? We begin with finding the first dimension, also known as the first axis, that distorts the cloud as least as possible. By denoting h_i the projection of individual i on the axis, O the center of gravity of the cloud, i h_i squared is the squared distance between the individual in the cloud and its projection on the axes. And I want the squared distance between an individual and its projection to be as small as possible. Since the distance O_i between an individual and the origin is always the same, I_H_i is small if O_H_i is large. That is thanks to Pythagoras. Since O_i squared is constant, I h i square is small if O h i square is large. So we want O h i square to be as large as possible. Since O h i square should be large for all the individuals, I want the sum of all the O h i square the largest as possible. And the sum of all the O h i square is the dispersion of the projected point. And I want these projected points to be as dispersed as possible. This is the way to find the first dimension. Then I will try to find the best viewpoint of the cloud in 2 dimensions. To obtain this representation, the cloud is projected on a map chosen in such a manner as to minimize the distortion of the cloud of points. The map is selected so that the distances between the projected points on the plane Hi, Hi is now the projection on the plan. Might be as close as possible to the distances between the initial points. Since in projection distances can only decrease, we try to make the projected distances as high as possible. So once again, I want the sum of all the Ohi squared to be as large as possible. It It can be shown that the best plan contains the best components. So to find the best map, you can find the best dimension, and then find an orthogonal dimension to the first dimension, and find the one that maximizes the sum of the OHi squared. Then one can find a third dimension, and therefore seek dimensions sequentially, one after the other. Each dimension is orthogonal to all the previous ones, and maximizes the variance, the inertia. So from a technical point of view, PCA dimensions are obtained through a singular value decomposition or through diagonalization of the correlation matrix, which extracts the associated eigenvectors and the eigenvalues. One can visualize that with a small illustration. So here I have a teapot in 3 dimensions, and I will find the best dimensions to visualize this teapot in the best way with just one viewpoint. So I find the first dimension. Here is the first dimension that allows to see the teapot in the best way. So if I had to project the points of the teapot on a dimension, I would have taken the red dimension. And now I want a second dimension orthogonal to the first one. So I define the first dimension and I find a dimension orthogonal to the first. So here is the 2nd dimension in green. Let us now consider the wines dataset with 10 wines and 27 sensory attributes, with 2 preference variables for the odor preference and overall preference, and a qualitative variable corresponding to the label of the wines. I want to describe the wines from a sensorial point of view, so first I will consider the 27 quantitative variables corresponding to the sensory attributes. So if I represent the individuals, I obtain this graph. This graph shows, for instance, that S. Michaud and S. Trotignon are very close. What does it mean when I say S. Michaud and S. Trotignon are very close? It means that scores for S. Michaud and S. Trotignon are approximately the same, and that is the case whatever the attributes. The scores are almost the same, so points are close. Similarly, Aubussières-Mareigny and Fontaine Coteaux are 2 wines that have sensory scores that are close for the 27 attributes. On the other hand, Fontaine-Brûlée and Est Trottignon have very different sensory profiles because the first principal component, the first principal axis of variability, between the wines separates these 2 wines. Since these 2 wines are far on the first dimension, it means that they are very different because the first dimension is the one that separates the points as much as possible. So now, why are these 2 wines separated on the first dimension? To answer this question, we can be a French oenologist and have a good knowledge of these wines. But this is not my case, so I will use variables to interpret these dimensions. What are the differences between Fontaine-Brûlée and Estroguignon? To answer this question, we'll consider the coordinates of the individuals on the dimensions. For instance, Aubusier Select has the coordinate 1.1 on the first dimension. I will note its coordinate f_i,1, 1 for the first dimension, and then its coordinate on the second dimension f_i,2 is equal to -6. So for each individual i, I consider its coordinate on the horizontal axis and on the vertical axis. So I can create 2 vectors, a first vector which includes the coordinates for all the individuals on the first dimension, and a second vector that includes the coordinates for all the individuals on the second dimension. Therefore, these vectors have i coordinates, the same number as the variables of the data table. Thus, in order to interpret the graph of the individuals, I will calculate the correlation between the variable, that is to say odor of vanilla, and dimension 1. So I calculated the correlation between odor of vanilla and dimension 1, and I calculated the correlation between odor of vanilla and dimension 2. And if odor of vanilla is very linked to dimension 1, it means that the values for odor of vanilla are related to the coordinates on the first dimension. Thus, if the correlation is close to 1, It means that individuals who have low values for odor of vanilla have low values on dimension 1, and thus wines will be on the left of my map. And individuals who have high coordinates for odor of vanilla have high values on dimension 1 and will be on the right of the map. If the correlation is negative, it means that individuals who have low values on axis 1 have high values for odor of vanilla. And individuals who take high values on axis 1 have low values for odor of vanilla. This is the same for dimension 2. So we will be able to build a graph with a representation of all the variables. All the variables will gather in a circle named the correlation circle. In the example, the graph of the correlation circle shows that the variables odor passion, odor citrus, freshness that are on the left have a correlation close to -1 with the first dimension. Thus, wines that, that are on the left have a small coordinate on the first dimension, and since the correlation is close to -1, it means that these wines have high values for these odors. On the other hand, Wines that are on the right have a high coordinate on dimension 1, and thus a small value for these odors. For astringency, visual intensity, mushroom odor, and candied fruit odors, for instance, it is the contrary. The correlation with dimension 1 is close to 1, and thus the values move in the same way. Wines with a small coordinate have small values for these variables, and wines with high coordinates have high values for these variables. So the first dimension opposes wines that are considered as fruity and flowery, on the left, and wines that are woody or with vegetal odors. And this is the main source of variability. So now, how can we interpret dimension 2, the vertical axis? On the top, wines have a high coordinate on the vertical axis, and since the correlation coefficients between dimension 2 and variables such acidity or bitterness are close to 1, it means that wines on the top take high values for these variables. And wines on the bottom have low coordinates on dimension 2, and thus low values for these variables. For sweetness, the correlation coefficient is close to -1. So wines that have a low coordinate on dimension 2 are sweet, and wines that have high coordinates on dimension 2 are not sweet. The 2nd dimension opposes wines on the top that are acid and bitter to wines that are sweet on the bottom. So the main dimensions of variability are first the odors, And the second dimension is an opposition between sweet wines and acid and bitter wines. So let us now study the variables. So a variable is a point in a space with I dimensions since each variable has I values, one per individual. So we will look at the cloud of variables that evolves in a space with I dimensions. The variables are represented by arrows in principal component analysis. So I represent the variable k by an arrow from the origin. The cosine of the angle theta k l between the arrow that represents the variable l and the arrow representing the variable k is equal to the scalar product of the variable k and the variable l. Divided by the norm of variable k times the norm of variable l. This is equal to the sum of x_ik times x_il, the sum over all i, divided by the square root of the sum of the square x_ik times the sum of the square x_il. Since the data matrix is centered, we can read sum of xik minus the mean of xk times xil minus the mean of xl divided by the standard deviation of the variable k times the standard deviation of the variable l. So, when the data are centered, I finally recognize the correlation coefficient between the variable k and the variable l. So we have here a geometric representation of the correlation coefficient between 2 variables k and l. The geometrical representation of this correlation is the cosine of the angle between the variables l and k. If variables are standardized, the length of the arrows in the global space equals 1. And all arrows will be at a distance 1 on an i-sphere of radius 1. So for a variable, we have an arrow from the center of the i-sphere, and that goes on the i-sphere of radius 1. The question now is, how can we view the cloud of variables? This cloud evolves in a space with i dimensions, and I cannot see it on the i dimensions. So we will find dimensions that allow us to see the cloud of variables in the best way. So it is the same trick as for individuals. We will seek orthogonal axes that can best represent the variables. The first dimension, the dimension that allows to see the variables in the best way, is the dimension that maximizes the sum of the correlations between the dimension and each variable. So the best axis, V1, is the factor which is the most linked to all the variables. The most linked according to the squared correlation coefficient. So the variable V1 is a synthetic variable that best summarizes all variables as closely related to the set of variables. Once we have the first dimension, we will find a second dimension orthogonal to the first one, which is the most linked to the variables. This second dimension synthesizes the rest of the information which has not yet been synthesized by the first axis. Once I have found the second axis, I can find sequentially a third dimension and so on. And thus fitting the cloud of points on our dataset, we find the same representation as before. Earlier, we had built a representation of variables to help us interpreting the cloud of individuals. And then, when I built the best representation to see the cloud of variables, I find the same representation again. So that's a little magic. It's amazing. Thus, this representation serves to assist in the interpretation of the cloud of individuals, to characterize the individuals. So in my example, fruity and flowery wines, sweet wines, acid and bitter wines. It is also, as we have seen here, an optimal representation of the cloud of variables. And then I have a visualization of all the correlations between pairs of variables by the cosines of the angles with 2 variables. So this representation is also a graphical visualization of the correlation matrix. It is quite magical because it is the same representation. The relationships between the cloud of individuals and the cloud of variables are brought together under the general term of relations of duality. This term refers to the dual approach of one single data table by considering either the lines, the rows, or the columns. This approach is also defined by transition relations that allow to calculate the coordinates in one space from those in the other. Let us note f i s the coordinate of individual i and g k s the coordinate of variable k of the component of rank s. The vector f point s are called the scores and the vector g point s divided by the square root of the eigenvalue are called the loadings. The transition relations are the two following equations. The coordinate of an individual explained by the data and the coordinates of the variables, and the coordinate of a variable explained by the data and the coordinates of the individuals. This result is essential for interpreting the data and makes PCA a rich and reliable experimental tool. This may be expressed as follows: Individuals are on the same side as their corresponding variables with high values and opposite their corresponding variables with low values. It must be noted that x_ik are centered and carry both positive and negative values. This is one of the reasons why individuals can be so far from the variables for which they carry low values. For instance, the coordinate of the variable sweetness is low on dimension 2, and the value of Aubusier Silex for sweetness is high. Thus, the multiplication is negative, and it contributes to the fact that the coordinate of the wine Aubusier Silex is low on dimension 2. The coordinate of the variable acidity is high on dimension 2, And the acidity of Aubusière silex is small. It is less than the mean of the variable acidity. So the positive coordinate of the variable acidity multiplied by the negative value gives a negative result, and it contributes to the fact that the coordinate of Aubusière silex is low on dimension 2. This is exactly the same reasoning for the variables. So let us come back to the graphical representation of the variables. We have seen that the correlation coefficient between a variable B and a variable A is equal to the cosine of the angle between variable A and variable B. But this is equal to the angle in the global space. However, I do not visualize the angle in the global space. I visualize it in the projection map. So here is an illustration that allows you to see the scatter plot of the variables and their projections. For instance, the variables D and E are very close to the projection map. So the projection of variable D, Hd, is very close to D because the variable D is close to the projection map. This is the same for He, which is close to E. So, the angle between D and E is very close to the angle between HD and HE. And therefore, the angle in the projection map can be used to visualize the correlation between D and E. More exactly, the cosine of the angle HD-HE is very close to the cosine of the angle between E and D, because the variables are well projected. So, the cosine of the angle between H_D and H_E is very close to the coefficient of the correlation between D and E. On the contrary, when the variables are unwell projected, which is the case for instance for the variables A and B, the angle in the projection map is small while the angle in the global space is very large. The error has a high coordinate on the 3rd dimension when the arrow B has a low coordinate. And so these 2 arrows are projected approximately at the same place, but they are poorly projected, and therefore the cosine of the angle here is not the cosine of the angle in the space. And therefore I cannot read the correlation coefficient between variables A and B. The cosine of the angle that I see here, a low angle here, is not close to the correlation coefficient. So I can read the correlation coefficients only for variables that are well represented. And variables are well represented if they are close to the correlation circle. Indeed, we have an isosphere of radius 1 cut by a plane and therefore the ice sphere is projected into a circle of radius 1. If an arrow is close to the edge of the circle, the variable is well represented and I can see the angle between the variables in the global space. On the contrary, it is impossible just with the projection to have an idea on the correlation coefficient between 2 variables unwell represented such as variables A and B. So we can only interpret the variables well represented. We will now see some helps to interpret the results of the principal component analysis. So an important help for the interpretation is the quality of the projection. This quality of the projection can be measured by the percentage of inertia which is explained by a dimension or by a plan. Thus, the percentage of inertia is also the percentage of information explained by a dimension. So we can construct a bar chart with the percentage of variance, or percentage of inertia, explained by each dimension. So here we see that the percentage of variance explained by the first dimension is approximately 43%, and the percentage of variance explained by the second dimension is approximately 25%. So with the first and the second dimension, I will explain 68% of the information that is contained in the 27 variables of the dataset. These percentages of inertia are given on the graphs of the individuals and on the graph of the variables. This is what is written here for 43% of the information on the first axis and 25% on the second axis. So initially there were 27 variables and I summarize with 2 dimensions and I get 68% of the information. So I have a very good summary of the 27 variables. And then on the last dimensions there is less information. So this bar chart is useful to choose the number of dimensions that should be used for the interpretation. We are therefore looking for a visible decrease or gap in the bar chart. It also exists a test to determine the number of dimensions or cross-validation methods that can also be used as it is the case with the function estim_ncp. We can also see PCA as a denoising method that will separate the signal on the first dimension and noise on the last dimensions. This is a little bit out of scope of this course, but PCA is interesting as a preprocessing step before clustering, for instance. There is a table which gives us the 95th percentile of the percentage of inertia of the first 2 dimensions of the PCA when we have independent variables. So for instance, with 15 individuals and 12 variables, this value of 47.8 tells me that having performed 10,000 PCAs with 15 individuals and 12 independent variables, I calculate the percentage of variance of the first 2 dimensions And 95% of values are less than 47.8. Here is another table when the number of variables is greater. With 10 individuals and 27 variables, the 95th percentile of the percentage of variance equals approximately 26%. In our example, we had a value of 68%. Which is higher than 26%. So my map explains more information than a PCA plan I would have with independent variables. So that means that my summary is very concise and there are not only independent variables in my dataset, there are strong relationships between my variables. Here are other helps to interpret the results. If there are supplementary variables or supplementary individuals. By definition, active elements contribute to the construction of the principal components, contrary to supplementary elements. Thus, the variance of the cloud of individuals is calculated on the basis of active individuals, and similarly, the inertia of the cloud of variables is calculated on the basis of active variables. The supplementary elements make it possible to illustrate the principal components, which is why they are referred to as illustrative elements. Contrary to the active elements, which must be homogeneous, we can make use of as many illustrative elements as possible. There can be quantitative variables, qualitative variables, or individuals as supplementary elements. By definition, a supplementary quantitative variable plays no role in calculating the distances between individuals. They are represented in the same way as active variables to assist in interpreting the cloud of individuals. So the coordinate of a supplementary variable on a dimension corresponds to the correlation coefficient between the variable and the principal component. Supplementary variables can therefore be represented on the same graph as the active variables. In our example, we can see that the preference variables are not strongly linked to the sensory attributes since the 2 preference variables are not well projected on the 2 first dimensions. For supplementary qualitative variables, we will project the center of gravity of individuals which take a category. For the variable Label that takes the 2 categories Sauvignon and Vouvray, I will put Sauvignon at the center of gravity of the 5 wines Sauvignon, and I will put Vouvray at the center of gravity of the 5 wines Vouvray. Note that the qualitative variable is represented through its categories on the graph of the individuals, and not on the graph of the variables. If I would have several qualitative variables, I would calculate the centers of gravity of each category of each qualitative variable. Another indicator that is useful when interpreting the results: the quality of representation. We will consider the quality of representation of a variable or the quality of representation of an individual. The quality of representation of a variable is measured by the squared cosine of the angle between a variable and its projection on the map or its projection on an axis. And the quality of projection of an individual is the squared cosine of the angle between the vector from the center of gravity of the cloud and the point i corresponding to the individual and the vector from the center of gravity of the cloud and the projected point h_i. The cosine is calculated axis by axis. So we can see, for instance, that for Renaudy, the squared cosine with the first dimension is 0.73, and the squared cosine with the second dimension is 0.82. Thus, for the quality of projection on the map, I sum 0.73+ 0.15 0.88. So that means that the angle is close to 0, and therefore the individuals are well projected. I can then interpret the distance between this individual and other individuals that are well projected. If 2 individuals are close on the map but unwell projected, I cannot interpret their proximity because they may be far on last dimensions. For the variables, passion odor is well projected whereas fruity odor is unwell projected on the 2 first dimensions. The contribution to the construction of one dimension is another criterion. It can be calculated for a variable or an individual. The contribution of a variable is just the square of the correlation coefficient between the variable and the dimension divided by the sum of squared correlations between all the variables and the dimension. It will tell me if there is a variable that contributes significantly to the construction of the map. The contribution of an individual is the square of the coordinate of the individual on the dimensions divided by the sum of the squares of the coordinates of all individuals. These contributions are expressed as a percentage. Again, if an individual highly contributes to, to the construction of a dimension, it means that the dimension is probably mainly due to this individual, and without this individual, the dimensions would be different. So in this case, we will say that the individual is very particular. We will explain why it is very special, and then it can be interesting to perform the analysis again without this individual to see if the dimensions change, if the first dimension of variance is always the same, or if it is a little bit different without this particular individual. Here are now some tools that can help to the interpretation, especially when there are lots of variables. The components provided by principal component methods can be described automatically by all the variables, whether quantitative or categorical, supplementary or active. For a quantitative variable, the principal is the same whether the variable is active or supplementary. First, the correlation coefficient between the coordinates of the individuals on the component s and each variable is calculated. We then sort the variables in descending order from the highest coefficient to the weakest and retain the variables with correlation coefficients significantly different from 0. For the description of the first dimension, we see that the most correlated variables is the odor of candied fruit and the grad that have correlations close to 1 and thus that best describe the first dimension. There are other variables which are highly correlated and are positively correlated with the first dimension. And there are also some variables that are strongly linked to the first dimension but with a correlation coefficient close to -1. There are lots of variables that characterize the first dimension, and then for the second dimension, there are fewer variables that describe this second dimension. This is awaited since the second dimension is less related to the variables than the first by construction, since the first is the main dimension of variance. Odor intensities are strongly positively linked to the second dimension, whereas sweetness is negatively correlated. For a categorical variable, we conduct a one-way analysis of variance where we seek to explain the coordinates of the individuals on the component S by the categorical variable. We use the contrast the sum of the coefficients equals 0. Then, for each categorical variable, an F-test is first conducted to know if the categorical variable is linked to the dimension. Secondly, a Student t-test is conducted to compare the average of the individuals who possess that category with the general average. We test the coefficient equals 0 considering that the variances of the coordinates are equal for each category. The one-way analysis of variance coefficients are sorted according to the p-values in descending order for the positive coefficients and in ascending order for the negative coefficients. For the label variable, the squared correlation ratio equals 0.87. So 87% of the variance of the coordinates of the individuals on the first dimension is explained by the variable label. And it is significant. And then, in more detail, we see that Vouvray have significantly positive coordinates on the first dimension, whereas the Sauvignon have significantly negative coordinates. These tips for interpreting such data are particularly useful when— for understanding those dimensions with a high number of variables. So to summarize, what is the practice for PCA? First, it is necessary to choose which variables will be active, that is to say which variables will be used to calculate the distances between individuals, and which variables will be used as supplementary. The other variables do not contribute to build the dimensions but may be used for interpreting the dimensions. Then, it is necessary to decide whether we must standardize variables or not. Standardization is essential as the variables are in different units. When the variables are in the same units, both solutions are possible. Standardization means that every variable has the same importance. Not standardizing the data, therefore, means that each variable has a weight corresponding to its standard deviation. This choice is also more important as the variables have different variances. Then perform PCA. And then we will choose the number of dimensions that will be interpreted. Do we interpret the first 2 dimensions or also the 3rd and the 4th? In this latter case, we will build the representation with the dimensions 3 and 4. Then we will interpret the PCA results studying simultaneously the graph of the individuals and the graph of variables. We are going to go back and forth between these 2 graphs to express and understand the differences and similarities between individuals and to understand the relationships between variables. We can use indicators to enrich the interpretation, calculating contributions using the square cosine to be sure that The distances on the map are representative of the distances in the global space. And finally, when we say that 2 individuals are similar, it is interesting to go back to the raw data to be sure that we do not misinterpret. Whenever the analysis suggests an interpretation, we must return to the raw data or the standardized data to validate what is being interpreted. Here are the lines of code to perform the PCA on the wine dataset. First, we read the data. Then, we perform PCA on standardized data using the argument scale=True. The first variable is qualitative, so a supplementary variable, and the variables 29 and 30 are supplementary quantitative variables. The graph of the individuals and the graph of the variables are given. Then we use the summary PCA function to see the outputs. We then draw the bar chart to decide the number of components we will interpret. And the next line of code is useful to draw the individuals according to their labels. And the last line of code gives the automatic description of the dimensions. You can see the following movie to have more information on the use of the PCA function. So to conclude, here are some supplements. There are many books on exploratory multivariate data analysis, and I suggest this one that follows the presentation of this course, Exploratory Multivariate Analysis by Example Using R. There is an R package called FactoMineR that allows us to perform PCA with supplementary quantitative variables, supplementary qualitative variables, with many indicators such as the quality of interpretation, the contributions. And finally, you can find movies on YouTube as well as a YouTube channel playlist and more videos in French and some English videos. Some movies show how to perform PCA with missing values or how to handle missing values in PCA. So thank you for your attention. You can now study and visualize your large datasets.
More from Yamossoun
Algorithme de calcule de la racine carrée par dichotomie #dichotomy #squareroot #algorithm