Topic Five: Multivariate Factor Models

• Upon completion of this lecture, participant should

be able to demonstrate a clear understanding and
application of:
– Principal component analysis
– Factor Analysis

1
Reading:
• Greene (2018) Chapter 4
• Alhija F.A.N.(2010) Factor Analysis: An Overview and Some Contemporary Advances, Elsevier Ltd
• Bartholomew, D.J. (2010) Principal Components Analysis, Elsevier Ltd
• Articles for class discussion
 Aziakpono, MJ, Kleimeier, S and Sander, H (2012), “Banking market integration in the SADC countries:
Evidence from interest rate analysis”. Applied Economics 44(29), 3857-3876.
 Meric, I., Ratner, M. and Meric, G., (2008) “Co-movements of sector index returns in the world’s major
stock markets in bull and bear markets: Portfolio diversification implications”. International Review of
Financial Analysis, 17: 156–177.
 Becker B and Hall, S. G (2009) “How far from the Euro Area? Measuring convergence of inflation rates
in Eastern Europe”, Economic Modelling, 26: 788–798
 Jalil, A., Feridun, M., and Ma, Y. (2010) “Finance-growth nexus in China revisited: New evidence from
principal components and ARDL bounds tests” International Review of Economics and Finance, 19:189–
195.
 Perignon, C., Smith, D.R., and Villa, C., 2007. Why common factors in international bond returns are not
so common. Journal of International Money and Finance 26: 284-304.
 Goyal A., Perignon, C. Villa, C. (2008) “How common are common return factors across the NYSE and
Nasdaq?” Journal of Financial Economics, 90: 252–271.

2
 Principal Component Analysis (PCA)
 Definition:
 PCA is a technique that is used to reduce the dimensionality of a
data set consisting of a large number of interrelated variables, while
retaining as much as possible of the variation present in the data set.
 The PCA helps to transform the original set of variables to a new set
of variables, the principal components (PCs), which are uncorrelated,
and which are ordered so that the first few PCs retain most of the
variation present in all of the original variables
 The PCA derives the PCs in such a way that they explain the
variations in the set of original variables in a descending order.
Thus, the first PC accounts for the main part of the variation of the
original variables; the second PC will explain the main part of the
remaining of the variations in the variables after the effects of the
first PC has been removed from the data.

3
 Principal component problem
• The principal component problem can be expressed thus:
P  AX (1)
• where P is a vector of orthogonal factors or principal
components and they are a linear combination of the original
series,

• X is the m observed sets of variables and

• A is a matrix of coefficients or factor loadings with each

coefficient representing the weight of the corresponding
original variable in the relevant principal component (PC)

4
Key terms-1
• Standard deviation
• Variance
• Covariance
• Correlation
• Eigenvalue
– This represents the amount of variance that is accounted for by a given
component. The sum of Eigenvalues is equal to the total number of variables
and the average is equal to 1.
• Proportion
– Eigenvalue for the component of interest as a ratio of the total eigenvalues of
the correlation matrix
• Cumulative value
• Cumulative proportional
• Eigenvector
• Factor loading
– This is a general term for a coefficient that appears in a factor pattern matrix
or a factor structure matrix. In orthogonal PCs the factor loadings are
equivalent to a bivariate correlations between the observed variables and the
components
5
Key terms-2
• Orthogonal factor: Uncorrelated factors
• Oblique factors: Correlated factors
• Communality:
– The proportion of variance in the observed variable that is explained
by a factor structure
• Uniqueness:
– The proportion of a variable’s variance that is not shared with a factor
structure
• Scree diagram
– This is the plot of the eigenvalues associated with each component.
Look for a break between the components with relatively large
eigenvalues and those with small eigenvalues
• Rotation
– This is a linear transformation that is performed on the factor solution
for the purpose of making the solution easier to interpret
• Latent variable:
– A theoretical variable hypothesized to influence a number of observed
variables
6
Steps in Conducting PCA-1
• Step1: Initial Extraction of the Components
– The PC extracted is equal to the number of variables being analysed
– The 1st PC normally would account for a fairly large amount of the total
variance
– The subsequent PCs will account progressively for smaller amounts of
variance
• Methods:
• Correlation and Covariance
– The transformation of variables can be accomplished by using either the
correlation matrix or the covariance matrix. The use of correlation matrix is
particularly advantageous if the scales of the variables are different, since it is
mean-centered (Jolliffe, 2002) and it is the commonly used
• Explanatory power of each PCs
– In order to determine the explanatory power of each of the PCs two measures are
conventionally used, namely, the eigenvalue and the cumulative R2 of the PCs.
– The cumulative R2 has the same interpretation as the ordinary regression
analysis (i.e. R2), with a value close to one indicating high explanatory power,
while a value close to zero indicate low explanatory power
7
Steps in Conducting PCA-2

• Step 2: Determine how many PCs to retain

i. Kaiser Rule: Eigenvalue greater than one

ii. The scree test

iii. Proportion of variance accounted for (e.g. 5%, 7%, 10%)

(or Cumulative percent of variance accounted for, e.g.
70%-80% )
• Note: the critical values are arbitrary

8
 Steps in Conducting PCA-3
• Step 2: Determine how many PCs to retain
iv. Interpretability criteria (Very important criterion)
– At least three variables with significant loadings on each retained
component
– Do the variables that load on a given PC share the same conceptual
meaning?
– Do the variables that load on different components seem to be
measuring different constructs
– Does the rotated factor pattern demonstrate simple structure? (i.e.,
most of the variables have relatively high factor loadings on only one
component, and near zero loadings on the other PCs, and most
components have relatively high factor loadings for some variables,
and near-zero loadings for the remaining variables)
• Recommendation: Combine all four in a structured sequence
9
Steps in Conducting PCA-3

• Step 3: Rotation to a final solution

– Most popular method: VARIMAX Rotation.

• A varimax rotation is an orthogonal rotation, meaning that it

results in uncorrelated components.

10
 Steps in Conducting PCA-3
• Step 4: Interpreting the Rotated Solution
– This mean determining just what is measured by each of
the retained components.
– Decisions:
– Decide how large a factor loading must be to be considered large
(May be: if its absolute value exceeds 0.40)
– Read across the row for the first variable (In many situations,
researchers want to drop a variables that load on more than one
component, because the variables are not pure measures of any one
construct
– Review all of the surviving variables with high loading on component
to determine the nature of this component and assign a name to the
component
– Determine whether the final solution satisfies the interpretability
criteria mentioned above
11
Steps in Conducting PCA-4

• Step 5: Creating Factor Scores

(component scores)
– Factor score is a linear composite of the optimally
weighted observed variables
• Step 6: Summarising the Results in a
Table
– Follow good published articles

12
Principal Components in Regression Analysis-1

• Biased estimators
• Strategies for selecting components in principal component
regressions
– Objective: to eliminate large variances due to multicollinearities
(delete all those components whose variances are very small),
but at the same time, it is undesirable to delete components that
have large correlations with the dependent variable y.
• Delete all those components whose variances are less than l*,
where l* is some cut-off level.
– The choice of l* is rather arbitrary
– Attractive and simple strategy, but low variance for a component
does not necessarily imply that the corresponding component is
unimportant in the regression model.

13
 Principal Components in Regression Analysis-2

– Note: Such a procedure can be dangerous if low-variance

components have predictive value
– Based on values of t-statistics measuring the independent
contribution of each PC to the regression equation
– To delete PCs sequentially starting with the smallest
variance, then the next smallest variance and so on, the
deletion stops when the first significant t-value is reached.
This strategy is likely to retain more PCs than are really
necessary.
– There are other more sophisticated approaches
• Note: It is difficult to give any general advice regarding the
choice of a decision rule for determining n.
14
 Factor Analysis(FA)

• Definition:
– Factor analysis is used to uncover the latent structure (dimensions)
of a set of variables. This is a collection of methods used to examine
how underlying constructs influence the responses on a number of
measured variables
– In PCA, the intrinsic interest is in the observed variables
– In FA, the intrinsic interest is in the underlying factors, the observed
variables are of little interest.
– Factor Analysis Methods: Two basic types of FA
• Exploratory factor analysis (EFA): This attempts to discover the nature of
the constructs influencing a set of responses
• Confirmatory factor analysis (CFA): This tests whether a specified set of
constructs is influencing responses in a predicted way.
• We focus on EFA.
15
 Exploratory factor analysis (EFA)

• Objectives:
– To determine the number of common factors influencing
a set of measures
– To determine the strength of the relationship between
each factor and each observed measure
– Common uses:
• To identify the nature of the constructs underlying responses in
a specific content area
• To determine what features are most important when classifying
a group of items
• To generate factor scores representing values of the underlying
constructs for use in other analyses.

16
 Steps in Conducting EFA

1. Collect measurements: variables must be measured on the same

experimental units
2. Compute covariances, correlation, or other measures of association
3. Select the number of factors
– Same methods as in PCA
4. Obtain initial uniqueness estimates
5. Extract your initial set of factors
– Popular methods: Maximum likelihood, Generalised least squares,
Unweighted least squares, principal factors, Iterated principal factors, and
Partitioned
6. Factor rotation
– Orthogonal rotations: produce uncorrelated factors (Varimax, Factor
Parsimony, Target, Simplimax).
– Oblique rotations: produce correlated factors (Promax, Oblimax, Oblimin)
7. Interpret factor structure
8. Estimate factor scores
17
 Topic Five: Multivariate Factor Models (Presentation 11/04/2024)

 Meric, I., Ratner, M. and Meric, G., (2008) “Co-movements of sector index returns in the
world’s major stock markets in bull and bear markets: Portfolio diversification
implications”. International Review of Financial Analysis, 17: 156–177.
Team A Thula BAKADA
Munesu CHATIKOBO
Thembelani CHILI
Seitebatso Kabi
Manyanani KAMANGA
Vusimani Kubhayi

 Becker B and Hall, S. G (2009) “How far from the Euro Area? Measuring convergence of
inflation rates in Eastern Europe”, Economic Modelling, 26: 788–798

Team B Mbali Masanabo

Success Masondo
Reekeditswe MPHELA
Nkosinathi NONKONYANA
Xolela Ndzamela

18
 Practical Example-with EViews

• Compute covariances, correlations, or other measures of

association

• Specify the number of factors

• Obtain initial uniqueness estimates

• Extract (estimate) factor loadings and uniquenesses

• Examine diagnostics

• Perform factor rotation

• Estimate factor scores

19
Thank you

20