Principal Component Analysis (PCA)

Read First

PCA, is a way to create an index from a group of variables that are similar in the information that they provide. This allows maximizing the information we keep, without using variables that will cause multicolinearity, and without having to choose one variables among many.

Guidelines

The spatial principle of a PCA

In a space of 3 dimensions, that are for instance income in x, savings in y and consumption in z, we have let’s say 12 vectors that represent our 12 similar variables that were measured in the field. Those vectors combined together create a cloud in 3D. That cloud has 3 principal directions; the first 2 like the sticks of a kite, and a 3rd stick at 90 degrees from the first 2. Well, the longest of the sticks that represent the cloud, is the main Principal Component.

In fact, our variables explain more than 3 dimensions, so then the space that contain our vectors can be in 8, 12, 15 dimensions, etc, and so is the cloud. You observe this in your results, as there are several principal components that are listed. The same applies than for our example in 3D, though, so that the PCA provides the size of the dimension that represents the cloud the best (so the longest stick within the several-dimensions cloud).

PCA provides us information on the one main component, which corresponds to the information that similar variables have the most in common. Thus, the other components are not taken into account. All complementary information (orthogonal to the main component) in then lost. Therefore, we will want to use PCAs only on variables that have a lot in common, so that the loss of complementary information is minimized".

How to do it

First, check the multiple correlation between your similar variables. Keep only the ones with the highest correlations. Most of the art happens here: The most similar the variables are, the most charged with information the pca is.

Beware of variables with a lot of missings, because the new pca variable will be set as missing for those observations.

Then, the manipulation its simple: you can use the functions pca or pcamat, and predict in Stata. You will want to take a close look at the proportion of the variance that is explained by your first component. You can also use estat kmo (Kaiser-Meyer-Olkin), that tests if your variables were appropriate for factor analysis.

Use of multiple imputation prior to calculation of a pca

When having to deal with several missing for a few variables among the group of similar variables, one may be tempted to use multiple imputation prior to doing the PCA.

However, the problem is that the MI is done by regression, so will be sensitive to multicollinearity, whereas the PCA is the most charged with information when the variables are the most similar. Therefore it is feasible, yes, but then it will cost you in the information that your final PCA will contain, as you will have had to drop some of your most similar variables prior to doing your MI.

Back to Parent

This article is part of the topic Data Analysis

Additional Resources

• list here other articles related to this topic, with a brief description and link