Factor analysis

Factor Analysis a multivariate statistical technique is primarily used to examine the structure
of data by explaining the correlations among variables. Factor analysis summarizes data into
a few dimensions by condensing a large number of variables into a smaller set of latent
variables or factors. Factor analysis is commonly used in the social sciences, market research,
and other industries that use large data sets.

Consider a credit card company that creates a survey to evaluate customer satisfaction. The
survey is designed to answer questions in three categories: timeliness of service, accuracy of
the service, and courteousness of phone operators. The company can use factor analysis to
ensure that the survey items address these three areas before sending the survey to a large
number of customers. If the survey does not adequately measure the three factors, then the
company should re-evaluate the questions and retest the survey before sending it to
customers.

Example

An investigator record the following characteristics of 14 census tracts: total population

(Pop), median years of schooling (School), total employment (Employ), employment in
health services (Health), and median home value (Home). The investigator would like to
investigate what "factors" might explain most of the variability. As the first step in factor
analysis, the principal components extraction method is employed to examine an eigenvalues
in order to help you to decide upon the number of factors.

Unrotated Factor Loadings and Communalities

Variable Factor1 Factor2 Factor3 Factor4 Factor5 Communality

Pop 0.972 0.149 -0.006 -0.170 0.067 1.000

School 0.545 0.715 0.415 0.140 -0.001 1.000

Employ 0.989 0.005 -0.089 -0.083 -0.085 1.000

Health 0.847 -0.352 -0.344 0.200 0.022 1.000

Home -0.303 0.797 -0.523 -0.005 -0.002 1.000

Variance 3.0289 1.2911 0.5725 0.0954 0.0121 5.0000

% Var 0.606 0.258 0.114 0.019 0.002 1.000

Factor Score Coefficients

Variable Factor1 Factor2 Factor3 Factor4 Factor5

Pop 0.321 0.116 -0.011 -1.782 5.511

School 0.180 0.553 0.726 1.466 -0.060

Employ 0.327 0.004 -0.155 -0.868 -6.988

Health 0.280 -0.272 -0.601 2.098 1.829

Home -0.100 0.617 -0.914 -0.049 -0.129

Interpreting the results

Five factors describe these data perfectly, but the goal is to reduce the number of factors
needed to explain the variability in the data. The proportion of variability explained by the
last two factors is minimal (0.019 and 0.002, respectively) and they can be eliminated as
being important. The first two factors together represent 86% of the variability while three
factors explain 98% of the variability. The question is whether to use two or three factors.
The next step might be to perform separate factor analyses with two and three factors and
examine the communalities to see how individual variables are represented. If there were one
or more variables not well represented by the more parsimonious two factor model, you
might select a model with three or more factors.
Factor analysis model

The factor analysis model is:

X = m + L F + e,

where X is the p x 1 vector of measurements, m is the p x 1 vector of means, L is a p x m

matrix of loadings, F is a m x 1 vector of common factors, and e is a p x 1 vector of residuals.
Here, p represents the number of measurements on a subject or item and m represents the
number of common factors. F and e are assumed to be independent and the individual F's are
independent of each other. The mean of F and e are 0, Cov(F) = I, the identity matrix, and
Cov(e) = Y, a diagonal matrix. The assumptions about independence of the F's make this an
orthogonal factor model.

Under the factor analysis model, the p x p covariance matrix of the data, X, is:

Cov(X) = L L' + Y,

where L is the p x m matrix of loadings, and Y is a p x p matrix of variances of residuals. The

ith diagonal element of L L', the sum of the squared loadings, is called the ith communality.
The communality values can be judged as the percent of variability explained by the common
factors. The ith diagonal element of Y is called the ith specific variance, or uniqueness. The
specific variance is that portion of variability not explained by the common factors. The sizes
of the communalities and/or the specific variances can be used to judge the goodness of fit.

Key Points

The goal of factor analysis is to find a small number of factors, or unobservable variables,
that explains most of the data variability and yet makes contextual sense. You need to decide
how many factors to use, and find loadings that make the most sense for your data.

Number of factors

The choice of the number of factors is often based upon the proportion of variance explained
by the factors, subject matter knowledge, and reasonableness of the solution. Initially, try
using the principal components extraction method without specifying the number of
components. Examine the proportion of variability explained by different factors and narrow
down your choice of how many factors to use. A Scree plot may be useful here in visually
assessing the importance of factors. Once you have narrowed this choice, examine the fits of
the different factor analyses. Communality values, the proportion of variability of each
variable explained by the factors, may be especially useful in comparing fits. You may decide
to add a factor if it contributes to the fit of certain variables.

Rotation

Once you have selected the number of factors, you will probably want to try different
rotations. A similar result from different methods can lend credence to the solution you have
selected. At this point you may wish to interpret the factors using your knowledge of the data.