Exploratory factor analysis

Date: December 17, 2013

Mosiur Rahman
Department of International Health and Medicine
Division of Public Health
Tokyo Medical and Dental University

Exploratory factor analysis. In: Field A. Discovering statistics using SPSS. 3rd ed.
Thousand Oaks: Sage; 2009;627-685.
 Outline

 Factor Analysis Overview

 Principal components Analysis: Basic Concept
 Principal Components Analysis: Strategy
 Principal Component Analysis: in SPSS
 Steps in Principal Component Analysis
Factor Analysis Overview
 What is Factor Analysis?

Factor analysis is an interdependence technique whose

primary purpose is to define the underlying structure among
the variables in the analysis
●Examines the interrelationships among a large number of
variables and then attempts to explain them in terms of their
common underlying dimensions
-These common underlying dimensions are referred to as factors
●Is a summarization and data reduction technique, not have
independent and dependent variables, but an interdependence
technique in which all variables are considered simultaneously
 Types of Factor Analysis

 Two Types of Factor Analysis:

■Exploratory Factor Analysis (EFA): used to discover

the factor structure of a construct and examine its
reliability. It is data driven

■Confirmatory Factor Analysis (CFA): used to confirm

the fit of the hypothesized factor structure to the
observed (sample) data. It is theory driven
 What Can We Do With
Exploratory Factor Analysis (EFA)?

■ Reduce the number of variables

■ Examine structure or relationship between variables
■ Evaluates the construct validity of a scale, test, or
instrument
■ Addresses multi-collinearity (two or more variables
that are correlated)
■ Used to develop theoretical constructs
 Statistics Associated with Factor Analysis

Communality: Amount of variance a variable shares with all the other

variables. This is the proportion of variance explained by the common
factors
Eigen value: Represents total variance explained by each factor
Factor loadings: Correlations between variables and the factors
Factor matrix: A factor matrix contains the factor loadings of all the
variables on all the factors Factor scores. Factor scores are composite
scores estimated for each respondent on the derived factors
Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy:
Used to examine the appropriateness of factor analysis. High values
(between 0.5 and 1.0) indicate appropriateness. Values below 0.5 imply not
Percentage of variance: The percentage of the total variance
attributed to each factor.
Screen plot: A screen plot is a plot of the Eigen values against the
number of factors in order of extraction
 Determine the Method of Factor Analysis

 Extraction decisions:
 Principal components analysis (PCA)
 Principal axis factoring (PAF)
 Maximum likelihood
 Unweighted least squares
 Generalized least squares
 Alpha factoring
 Image factoring
Principal components analysis: Basic Concept
 Principal components analysis

In Principal components analysis, the total variance

in the data is considered
- Used to determine the min number of factors that will
account for max variance in the data

- Most commonly used form of factor analysis

- Seeks linear combination of variables that extracts the
maximum variance

- This variance is removed and the process is repeated

Principal Components Analysis: Strategy in Problem
Solving
 Computing Principal Component Analysis: Strategy
for Solving Problems - 1

A principal component factor analysis requires:

Variables included must be metric level or dichotomous nominal level
Sample size must be greater than 50 (preferably 100)
Ratio of cases to variables must be 5 to 1 or larger
Correlation matrix for variables must contain 2 or more correlations of
0.30 or greater
Variables with sampling adequacy less than 0.50 must be removed
The overall measure of sampling adequacy is 0.50 or higher
The Bartlett test of sphericity is statistically significant.

The first phase of a principal component analysis is devoted to

verifying that we meet these requirements. If we do not meet
these requirements, factor analysis is not appropriate
 Computing a Principal Component Analysis:
Strategy for Solving Problems - 2

The second phase of a principal component factor analysis

focuses on deriving a factor model, or pattern of relationships
between variables and components, that satisfies the following
requirements:
The derived components explain 50% or more of the variance in
each of the variables, i.e. have a communality greater than 0.50
None of the variables have loadings, or correlations, of 0.40 or
higher for more than one component, i.e. do not have complex
structure
None of the components has only one variable in it

To meet these requirements, we remove problematic

variables from the analysis and repeat the principal component
analysis
 Computing a Principal Component Analysis:
Strategy for Solving Problems - 3

If, at the conclusion of this process, we have components that

have more than one variable loading on them, have components
that explain at least 50% of the variance in the included variables,
and have components that collectively explain more than 60% of
the variance in the set of variables, we can substitute the
components for the variables in further analyses.

Variables that were removed in the analysis should be included

individually in further analyses.

Substitution of components for individual variables is

accomplished by using only the highest loading variable, or by
combining the variables loading on each component to create a
new variable
 Computing a Principal Component Analysis:
Notes - 1

When evaluating measures of sampling adequacy,

communalities, or factor loadings, we ignore the sign of the
numeric value and base our decision on the size or magnitude
of the value

 Sign of the number indicates the direction of the relationship

A loading of -0.732 is just as strong as a loading of 0.732.

The minus sign indicates an inverse or negative relationship;
the absence of a sign is meant to imply a plus sign indicating a
direct or positive relationship
 Computing a Principal Component Analysis:
Notes - 2

If there are two or more components in the component

matrix, the pattern of loadings is based on the SPSS Rotated
Component Matrix. If there is only one component in the
solution, the Rotated Component Matrix is not computed, and
the pattern of loadings is based on the Component Matrix .

It is possible that the analysis will break down and we will
have too few variables in the analysis to support the use of
principal component analysis
Principal Component Analysis: Computation
Producers in SPSS
 Computing a Principal Component Analysis:
Problem 1

 In the dataset, is the following statement true, false, or an incorrect

application of a statistic? Answer the question based on the results of a
principal component analysis. Assume that there is no problematic
pattern of missing data. Use a level of significance of 0.05.
 8 variables:
i. claims about environmental threats are exaggerated: grnexagg
iii. danger to the environment from modifying genes in crops:genegen
iii. America doing enough to protect environment: amprogrn
iv. international agreements for environment problems: grnintl
v. poorer countries should be expected to do less for the environment:
ldcgrn
vI. economic progress in America will slow down without environment:
econgrn
vii. likelihood of nuclear power station damaging environment :nukeacc
viii. respondent's socioeconomic index: seii

 Component 1: genegen and "nukeacc

 Component 2 : grnexagg ldcgrn
 Individual variables : sei, econgrn, grnintl, amprogrn
Computing a Principal Component
Analysis
Add the Variables to the Analysis
Compete the Descriptive Dialog Box
Select the Extraction Method
Compete the Extraction Dialog Box
Select the Rotation Method
Compete the Rotation Dialog Box
Complete the Request for the Analysis
Level of Measurement Requirement
Sample Size Requirement: Minimum
Number of Cases
Sample Size Requirement: Ratio of Cases
to Variables
Appropriateness of Factor Analysis:
Presence of Substantial Correlations
Appropriateness of Factor Analysis:
Sampling Adequacy of Individual Variables
Appropriateness of Factor Analysis:
Sampling Adequacy of Individual Variables
Excluding a Variable from the Factor
Analysis
Repeating the Factor Analysis
Removing the Variable from the List of
Variables
Replicating the Factor Analysis
Appropriateness Factor Analysis: Sample
Adequacy for Revised Factor Analysis
Appropriateness of Factor Analysis:
Sample Adequacy for Set of Variables
Appropriateness of Factor Analysis:
Bartlett Test of Sphericity
Number of Factors to Extract: Latent Root
Criterion
Number of Factors to Extract: Percentage
of Variance Criterion
Evaluating Communalities
Communality Requiring Variable Removal
Repeating the Factor Analysis
Removing the Variable from the List of
Variables
Replicating the Factor Analysis
Communality Requiring Variable Removal
Repeating the Factor Analysis
Removing the Variable from the List of
Variables
Replicating the Factor Analysis
Communality Satisfactory for all Variables
Identifying Complex Structure
Repeating the Factor Analysis
Removing the Variable from the List of
Variables
Replicating the Factor Analysis
Checking for Single-variable Components
Variable Loadings on Components
Interpreting the Principal Components
Variance Explained in Individual Variables
Total Variance Explained
Answering the Problem Question
Steps in Principal Component Analysis
Steps in Principal Component Analysis - 1
Steps in Principal Component Analysis - 2
Steps in Principal Component Analysis - 3
Steps in Principal Component Analysis - 4
Steps in Principal Component Analysis - 5
Thank you for your patience
hearing