The video explains how to perform principal component analysis (PCA) on datasets that contain missing values, using the R packages missMDA and FactoMineR. It begins by illustrating why simple strategies—dropping rows/columns with missing entries or replacing missing values with variable means—are inadequate, especially when variables are correlated. The presenter then introduces an iterative PCA‑based imputation algorithm: start with a reasonable guess (e.g., mean or zero), run PCA on the completed matrix, project individuals to obtain updated estimates for missing entries, update means and standard deviations, and repeat until convergence. Because the basic algorithm can overfit, a regularized version (implemented in missMDA) is recommended, which shrinks imputed values toward variable means. The optimal number of principal components to retain for imputation is estimated with the `estim_ncpPCA` function; using too few or too many components can bias the explained inertia of the subsequent PCA. After imputation, a standard PCA is run with FactoMineR, but the analyst is warned that inertia percentages for the first dimensions may be inflated and that imputed values carry uncertainty. To quantify this uncertainty, the video demonstrates multiple imputation via the `mipca` function, visualizing variability with `plot.mipca` both for individual positions (ellipses) and for axis stability. The overall message is that PCA‑based imputation, especially when regularized and combined with multiple imputation, provides a principled way to handle missing data, but results must be interpreted cautiously, acknowledging the inherent uncertainty of imputed values.
Chapters
In this video, we're going to look at how to do PCA on an incomplete dataset, that is, one with missing data. This is a very common situation in practice, and there are, unfortunately, many reasons why data can be missing, such as people refusing to answer certain survey questions, machines that break down, misplaced data, etc., etc. To take into account and deal with missing data in PCA, We're going to use the missMDA package, which can be used in harmony with the FactoMineR package. In this video, we're going to use a dataset called orange, which can be found in the missMDA package. This dataset contains 12 individuals, here 12 orange juices, characterized by 8 quantitative variables. Some of the data is missing and represented by the letters N/A, which means not available. Not available. One simple solution involves removing individuals or variables which have at least one missing entry. In our example, this would leave us with only 3 individuals and 1 variable, which is clearly a bad way to progress. Indeed, simply removing whole rows or columns like this is rarely a good idea, and speaking as someone that has to spend a lot of time obtaining the data, No one likes throwing away a whole individual or variable for the sake of a missing entry. A different way to go about things is to replace each missing value by the mean over the relevant variables. This is the default thing that happens in many software packages, including FactoMineR. So if we run a standard PCA as previously, PCA Orange, We get the PCA plots from the dataset in which missing elements have first been replaced by the mean of the relevant variables. This is a rather unsophisticated way to approach the question. To understand why, let's take an example with 2 variables, X and Y, strongly correlated, both with missing values. If we input missing data by replacing it by the mean, This is what we get. The black points are individuals without missing values, and the strong link between X and Y is very obvious. The green points are individuals who had a missing value in X, and the red points are individuals who had a missing value in Y. Clearly, imputing with the mean completely deforms the distributions of X and Y, and the relationship between the 2 variables. What we can conclude is that replacing missing data by the mean is not a very satisfactory way to proceed either. So, what can we do? Well, other more sophisticated methods exist, which take into account structure in the data and tend to work better in practice. Here, for example, as the variables X and Y are strongly correlated, If a value of Y is missing for a certain individual, it would seem natural to estimate it using the value of X for the same individual, via, perhaps, a simple linear regression. What about when there are lots of variables? Well, if individuals i and j have very similar values across all the variables, and individual i has a missing value for the kth variable, it would make sense to estimate the missing value using the one for individual j for the case variable, because the 2 individuals seem to be very similar. We can apply these 2 ideas at the same time for a set of individuals and a set of variables by taking into account both the global similarities between individuals across all the variables, and the links between variables across all individuals. Let's pause for a second and ask what this means exactly. By taking into account both the global similarities between individuals across all variables and the links between variables across all individuals. Does it ring a bell? Basically, we are talking about PCA. So yes, it's true, we can impute missing data using PCA. This is the basic idea behind an iterative PCA algorithm which works as follows. To simplify, let's work with a little example with 5 individuals and 2 variables, XY and X2. 4 individuals have no missing data, while one, the 4th, has a missing entry for X2. Let's put the 4 individuals with no missing data on the plot, and put a little mark, here, for the individual with an X2 value missing. The first step of the iterative PCA algorithm is to impute the missing data using a random yet reasonable value. For example, the mean for the variable in question or, in our case, simply the value 0. Then, we alternate between 2 steps. The first consists of doing PCA on the now complete data. We do the PCA and draw the PCA axes on the plot. Each individual is projected onto this line, giving the little red dots. This gives us coordinate values for these points in terms of x1 and x2. The 4th individual, which had a missing value for x2, can now be projected and ends up with a value of 0.57. We can now take this value of 0.57 as the new starting point for the next iteration. We start again with all the observed data values and with 0.57 in the place of the missing entry. We keep on iterating these 2 steps: doing the PCA, then getting a new coordinate value for the 4th individual for x2 and updating the data table. We keep on iterating until it converges. In our example, the x2 coordinate for the 4th individual converges to 1.48 and the final PCA line is the one shown here. Here is a summary of the algorithm's steps. First, we provide an initial guess for the missing values such as, for example, the mean of each variable. Then, we iterate the A, B, and C steps. Step A consists of doing PCA on the imputed table. We choose the number of dimensions we want to use to do the next update. In our example, we chose 1 dimension because we were constructing 1 PCA line. The choice of number of dimensions for helping to impute the table is an important one. Step B involves updating the missing value estimations using PCA. Using both the individuals' and variables' coordinates. And then, step C, we update the means and standard deviations of each variable after updating. Then, we go back to step A and do a PCA. B, we update the missing value estimates. And then, we iterate these 3 steps until convergence. Usually, convergence is quick, So, the algorithm can sometimes overfit. In practice, overfitting is common. It comes from believing too much in the links between variables, which might not be as strong as we think, notably due to missing data. For this reason, we usually prefer to use a regularized version of the iterative PCA algorithm. We are not going to go into the details here, but This version is what's implemented in imissMDA, and what we use in practice. The basic idea is to not believe quite so much in the links between variables, and to impute missing data with a value that is a bit closer to the variables' means than it would have been otherwise. This is a way of taking less risk when imputing the missing data. Ultimately, the algorithm estimates the missing data values with values that have no influence on the PCA results, that is, no influence on the coordinates of the individuals or variables. And because the algorithm is based on PCA, it takes simultaneously into account similarities between individuals and links between variables. We have to say as well that this imputation method gives good quality results compared with other statistical imputation methods, and also remember that this method can be used to do PCA but also other statistical analyses. This regularized iterative PCA algorithm is implemented in the missMDA library. This library contains several functions. As we've already seen, it's necessary to estimate the number of PCA dimensions to use when imputing the missing data. We can do this using the estim_ncppca function. This function finds the optimal number of components to use when imputing missing data. To be clear, we'll use a standardized PCA for imputing, which is the default option, and it's what we have to use if we want to do a standardized PCA on the dataset after. What we mean is, in our example, we get that the estimated optimal number of components is 2. This is the number to use when imputing the missing data. We can then run the imputation algorithm using the iterative method we talked about earlier, where the 2 means that we want to use 2 PCA components, and scale=True because we want to use the standardized PCA to impute the missing data in the table. The resulting imputed data table is found in the complete_obs object. If we compare it to the original data, we see that the initial data hasn't changed, and where there was missing data, such as for individual 1 and the sweet variable, we now have the value 5.54. This estimate should be better than simply using the mean because we have taken into account the links between variables and similarities between individuals in its calculation. We can now run a normal PCA on this table using the PCA function in FactoMineR. It outputs the usual PCA results, with the plots for the individuals and variables. OK, so a few remarks about imputation using PCA for this dataset and its consequences on the PCA that follows for the imputed dataset. When we are doing the imputation, the missing data are, if you like, predicted in the chosen subspace. This subspace depends on the number of dimensions we use to impute the data. If we use 2 dimensions, the missing data are imputed using values from where the individuals are placed in the first 2 dimensions and the variables 2. We therefore amplify the projection quality on the first 2 dimensions and thus overestimate the percentages of inertia of the PCA associated with the first 2 dimensions. Taking again the toy example with the 2 variables and imputation using the PCA line, so by 1 dimension only, we see clearly that the points corresponding to the imputed data are on the PCA line. And so, in what follows, if we do a PCA on this imputed dataset, the first PCA component will have an overly large percentage of inertia. Therefore, the percentages of inertia for the PCA associated with the first dimensions of the PCA will be overestimated if we impute missing data. Therefore, when we move to interpreting the results, we should not go overboard in what we say about the percentages of inertia associated with the first dimensions. Especially if there are a great many missing data. Another problem: the imputed data are, when the PCA is performed, considered like real observations. But no, they are estimations, and so there is uncertainty in each of them. One way to evaluate this uncertainty is to perform what we call multiple imputation. Instead of imputing the data table one time only, We're going to do it several times, and with different values each time in order to see the plausible values that can take a missing value, and thus the variability induced by the uncertainty of the imputed values. The mipca function from the missMDA package lets us generate this set of tables, each with different imputed values for each missing data. The plot.mipca function, which can simply be called using plot then allows us to visualize this variability, that is uncertainty, on the plane defined by the 2 PCA axes. There are 2 ways to visualize this uncertainty. The first consists of projecting, as supplementary variables, the multiple tables onto the PCA plane obtained using one-time imputation. We therefore consider that the factor dimensions are fixed and only the uncertainty in the positions of the individuals and variables which have missing data is observed. Thus, in our example, individual 12 has no missing data, so there is no uncertainty in its position, while individual 5 has several missing data values, so uncertainty in its position is represented by an ellipse. Similarly, the TPCT variable has no missing data values, whereas the SWED variable has several, so there is uncertainty in its position. Each blue point corresponds to a neurose tip for this variable. The cloud of blue points is quite spread out, which means we should interpret its position with care. A second way to take uncertainty into account is to suppose that the imputed values have a certain level of variability. And that this variability also has an influence on the factor dimensions. So we're going to represent this uncertainty in terms of the individuals' positions, all of them, including those that didn't have any missing data values, because the individuals which do have missing data values make the axes move, and thus the positions of all individuals. This strategy leads to this plot for the individuals, this time with an ellipse around all of them, including individual 12 who didn't have any missing data values. The plot on the right shows the projections of the PCA dimensions of each imputed table on the PCA plane obtained using the original imputed data table. As all of the rows are close to either the first or second axes, this means that the axes are stable with respect to the set of imputed tables. If there had been evidence of instability here, we might not have wanted to interpret the PCA results too closely because they seem to depend too much on the imputed values of the missing data. This strategy helps us to be sure that there are not too many missing data points and that it remains possible and meaningful to do PCA. And to finish up, try to keep in mind that missing data can be imputed by a variety of methods, but none of them will never be as good as having the actual observed data point, and we should always keep this in mind when interpreting the post-imputation results. If possible, try to measure or get a feel for the impact of the imputation, as we have just explained, before coming to too strong conclusions, especially when there is a lot of missing data.
