Principal Component Analysis (PCA)

Read First

PCA is a way to create an index from a group of variables that are similar in the information that they provide. This allows us to maximize the information we keep, without using variables that will cause multicollinearity, and without having to choose one variable among many.

Guidelines

The spatial principle of a PCA

In a 3 dimensional space, say income in x, savings in y, and consumption in z, we have 12 vectors that represent our 12 similar variables that were measured in the field. Those vectors combined together create a cloud in 3D. That cloud has 3 principal directions: the first 2 like the sticks of a kite, and a 3rd stick at 90 degrees from the first 2. The longest of the "sticks" that represent the cloud is the main Principal Component.

In fact, our variables explain more than 3 dimensions, so the space that contains our vectors can be in 8, 12, or 15 dimensions, etc. and so is the cloud. You observe this in your results, as there are several principal components that are listed. The same applies than for our example in 3D, though, so that the PCA provides the size of the dimension that represents the cloud the best (so the longest stick within the several-dimensions cloud).

PCA provides us information on the one main component, which corresponds to the information that similar variables have the most in common. Thus, the other components are not taken into account. All complementary information (orthogonal to the main component) is then lost. Therefore, we will want to use PCAs only on variables that have a lot in common, so the loss of complementary information is minimized.

How to do it

First, check the multiple correlation between your similar variables. Keep only the ones with the highest correlations. Most of the art happens here: The more similar the variables are, the more charged with information the PCA is.

Beware of variables with a lot of missing values, because the new PCA variable will be set as missing for those observations.

Then the manipulation is simple: you can use the functions pca or pcamat, and predict in Stata. You will want to take a close look at the proportion of the variance that is explained by your first component. You can also use estat kmo (Kaiser-Meyer-Olkin), that tests if your variables were appropriate for factor analysis.

Use of multiple imputation prior to calculation of a PCA

When having to deal with several missing values for a few variables among the group of similar variables, one may be tempted to use multiple imputation (MI) prior to doing the PCA.

However, the problem is that the MI is done by regression, so will be sensitive to multicollinearity, whereas the PCA is the most charged with information when the variables are the most similar. Therefore it is feasible, but it will cost you in the information that your final PCA will contain, as you will have had to drop some of your most similar variables prior to doing your MI.

Back to Parent

This article is part of the topic Data Analysis

Additional Resources

Please add here related articles, including a brief description and link.