UNIT 5: FACTOR ANALYSIS

1] Meaning of Factor Analysis

2] Types of Factor Analysis
1. Confirmatory factor analysis:
Objective: To determine the ability of a predefined factor model to fit an observed set of data.
Key components of Confirmatory Factor Analysis include:
Latent Factors: Factors that represent the underlying constructs or dimensions that explain the
observed correlations among variables.
Observed Variables: These are the measurable variables that are directly observed or measured in
the study.
Uses:
A] Establish the validity of a single factor model
B] Compare the ability of two different models to account for the same set of data
C] Test whether a set of factors are correlated or uncorrelated
2. Exploratory FA
Usually done near the onset of research and is Summarizing data by grouping correlated variables
Terms:
Factor Loadings: Factor loadings represent the strength and direction of the relationship between
each observed variable and each latent factor. High factor loadings indicate a strong association
between the variable and the factor.
Eigenvalues: Eigen values are used to determine the number of factors to retain. An eigenvalue
represents the amount of variance explained by a factor. Factors with eigenvalues greater than 1 are
typically considered meaningful.
Communality: Communality refers to the proportion of the variance in an observed variable that is
accounted for by the factors. Higher communality values indicate that a larger proportion of the
variable's variance is explained by the factors.

3] BARTLETT’S TEST OF SPHERICITY

Bartlett’s test checks whether the correlation is present in the given data. It tests the null
hypothesis (H0) that the correlation matrix is an Identical matrix. The identical matrix
consists of all the diagonal elements as 1. So, the null hypothesis assumes that no correlation
is present among the variables.
We want to reject this null hypothesis because factor analysis aims at explaining the common
variance i.e. the variation due to correlation among the variables. If the test statistic value is
less than 0.05, we can decide that the correlation is not an Identical matrix i.e. correlation is
present among the variables with a 95% confidence level.
4] KAISER-MEYER-OLKIN (KMO) TEST
The KMO Test is a tool used in the preliminary stages of factor analysis. The Kaiser-Meyer-Olkin
(KMO) Test is a statistical measure used to assess the suitability of data for factor analysis. KMO Test
measures the proportion of variance that might be a common variance among the variables. In other
words, it assesses how much the variables are correlated or share common information. Larger
proportions are expected as it represents more correlation is present among the variables thereby
giving way for the application of dimensionality reduction techniques such as Factor Analysis. KMO
score is always between 0 to 1 and values more than 0.6 are much appreciated. A higher KMO score
indicates a larger proportion of common variance, suggesting that the variables are more
interrelated or correlated. We can also say it as a measure of how suited our data is for factor
analysis.

5] SCREE PLOT
A scree plot is a graphical representation that helps in determining the appropriate number
of factors or principal components to retain in a factor analysis or principal component
analysis. It is a diagnostic tool that displays the eigenvalues of factors or principal
components in descending order. The plot typically starts with high eigenvalues for the first
few factors and then gradually levels off, forming a "scree," or a steep slope that eventually
becomes more horizontal. As you move down the scree plot, the eigenvalues become
smaller, indicating that additional factors or components contribute less to explaining the
variance.

The scree plot is a line graph that shows the eigenvalues on the y-axis and the factor or
component number on the x-axis.

Factors or components before the inflection point(elbow point) (the steep slope) are typically
retained, as they contribute more substantially to explaining the variance.

Use of Scree Plot:

1] Eigenvalue Visualization:
 The scree plot displays the eigenvalues of factors or principal components in
descending order. Eigenvalues represent the amount of variance explained by
each factor or component.
2] Identifying Significant Factors:
 Significant factors contribute substantially to explaining the variance in the
data. In the scree plot, factors with higher eigenvalues are considered more
important. The scree plot helps identify the point where eigenvalues start to
decrease, indicating the diminishing contribution of additional factors.
3] Determining Model Complexity

4] Preventing Overfitting
6] METHOD OF FACTOR ANALYSIS

1] Formulate the Problem: The objectives of factor analysis should be identified. The variables to be
included in the factor analysis should be specified. The variables should be measured on an interval
or ratio scale.
2] Construct the Correlation Matrix
3] Method: In Principal components analysis, the total variance in the data is considered.
In Common factor analysis, the factors are estimated based only on the common variance.
4] Determine the Number of Factors:
Determination Based on Eigenvalues. Only factors with Eigenvalues greater than 1.0 are retained.
• Determination Based on Scree Plot. A scree plot is a plot of the Eigenvalues against the number of
factors in order of extraction. The point at which the scree begins denotes the true number of
factors.
• Determination Based on Percentage of Variance
5] Rotation of Factors: Through rotation the factor matrix is transformed into a simpler one that is
easier to interpret. The rotation is called orthogonal rotation if the axes are maintained at right
angles. 2 types: Varimax, Oblique
6] Interpret Factors
7] Calculate Factor scores
8] Determine Model Fit