Professional Documents
Culture Documents
(1996) Mastrangelo, Runger and Montgomery, Statistical Process Monitoring With PCA, Qual Real Eng Int
(1996) Mastrangelo, Runger and Montgomery, Statistical Process Monitoring With PCA, Qual Real Eng Int
GEORGE C . RUNGER
University of Maryland, College Park, MD 20742, U.S.A.
AND
DOUGLAS c. MONTGOMERY
Arizona State University, Tempe, AZ 85287-5906, U.S.A.
SUMMARY
Most industrial processes are characterized by a system of several variables, all of which are subject
to drifts, disturbances, and assignable causes of variation. In the chemical and process industries, there
are often inertial forces arising from raw material streams, reactors and tanks that introduce serial
correlation over time into these variables. This autocorrelation can have a profound impact on the
effectiveness of the statistical monitoring methods used for such processes. This paper reviews some
of the available methodology for multivariate process monitoring and shows the effectiveness of principal
components in this context. An application of the principal components approach with correlated
observation vectors is presented. The effectiveness of this procedure to indicate process upsets is dis-
cussed.
I
1------ c- - - - - - - 1-
I .
I
" ~ X l
variate data.*-1°
Traditional multivariate quality control methods
assume that the observation vectors are independent
over time. However, when processes exhibit
0 1.. . . dynamic behaviour, such as typically observed in
the chemical and process industries, autocorrelation
I
I
0
0
.. . -
I .
0
0
1
I
XI
between successive observation vectors may result.
Even though multivariate procedures assume that
the observation vectors are independent over time,
I 0 I
I we can show that the usual Hotelling P control
I
I------- c------ -1-
I
I
=x,
chart attenuates the autocorrelation in the process
data.
I
I
uZ-
Let pi denote the lag one autocorrelation of the
XZ ith component of x. The lag one autocorrelation of
Figure I . Two univariate control charts the P statistic is
STATISTICAL PROCESS MONITORING 205
covariance matrix or the correlation matrix of the Furthermore, if the latent variables are used in con-
original variables, and (b) the observations are not trol charts, we are essentially monitoring these
assumed to follow a multivariate normal distribution. physical characteristics of the process, and we can
The new variables generated by PCA are often obtain information useful for diagnosing the root
referred to as latent variables. Suppose there are m cause of a control chart signal. Because the latent
original variables. The first latent variable is that variables are uncorrelated, ideally each one measures
linear combination of the original variables with a fundamentally different characteristic of the pro-
greatest variance (for all coefficient vectors of unit cess and avoids the redundancies that often occur
length). The second principal component variable is in the raw variables.
that linear combination with greatest variance chosen Another advantage of PCA is that the reduction
from among coefficient vectors of unit length that to fewer latent variables can increase the sensitivity
are orthogonal to the first coefficient vector. Con- of a control scheme to assignable causes. It is well
tinuing in this manner, up to m latent variables can known that for a F control scheme, the performance
be constructed. to detect a shift of the process from, say, the zero
Although using all m latent variables does not vector, to the vector p is a function of ( I ) the
provide a dimensional reduction of the original data, number of variables, and (2) p and 8 only through
the benefit from PCA is the following: if the data the non-centrality parameter A = pT.-’p. Let
falls in a lower dimensional region of the full m- vlrv2,...,v, denote the eigenvectors of Z correspond-
dimensional space, then a few (say, three or fewer) ing to ordered (decreasing) eigenvalues dl,d2,.. .,d,,,
latent variables can be used to describe most of the of 2.Then,
variability in the process. Thus, the m-dimensional
data set can be approximated with three or fewer
variables. In Figure 3, the data in three dimensions
can be described by a single variable which is the
distance of each point from the origin along the
vector c,. It is convenient to normalize each original For a control chart, larger values of h are easier to
variable to a zero mean. The distance of each point detect. If only the first k latent variables are used
is the value of the first latent variable and c, is the in a multivariate control chart, such as the chi-square
vector of coefficients of the first latent variable. If chart, then the non-centrality parameter equals the
a second latent variable is used, each point in the sum of the first k terms in (6) and this is denoted
Figure can be described by its distance from the as A& If the off-target mean is substantially in the
origin along each of the vectors cI and c2. This pair direction of the first k latent variables, then the
of latent variables approximate each point by a point decrease from h to Ak is more than compensated by
in the plane defined by cI and c2. using fewer variables in the principal component
The third latent variabIe in this example would chart and detection sensitivity improves.
equal the distance of each point along a vector The disadvantage of PCA as a process monitoring
perpendicular to the plane defined by cI and c2. In method is that shifts of the process mean to a vector
this case, each point can be well approximated by that is orthogonal to the first k eigenvectors of 2:
one or two latent variables and the third latent are invisible to the control scheme. This may or
variable is approximately zero at each point. may not be important, depending on the type of
One advantage of PCA is that the latent variables assignable cause that is anticipated. Consequently,
can sometimes be interpreted as measures of funda- it has been suggested that principal component
mental physical characteristics of the process such residuals should be charted in some casesL2to sup-
as heating rates, material balances, or surface plement or replace a control scheme based on the
smoothness. In these cases, process insights are first k latent variables. A principal component
gained by interpreting the coefficients that incorpor- residual vector is the difference between a vector x
ate the measured variables in the latent variables. and its approximation by the first k latent variables.
Let the matrix V have columns equal to the rn
eigenvectors of I; and let V, be the submatrix that
consists of the first k columns of V. The residual
associated with x is
independent and normally distributed. This assump- resents the variance of the ith principal component.
tion is implicit when PCA is used. JolliffeI3 and Note that in Table I, the principal components are
Jackson12 both point out that if PCA is to be used ordered from the largest to smallest proportion of
in a descriptive or exploratory manner, rather than total variance explained by the ith component.
an inferential tool, then the traditional assumptions Selecting the number of principal components
of normality and independence are not essential in to retain is subjective. However, there are several
order for the technique to be effective. Even though guidelines that can be used. One rule of thumb
O u r use of PCA tends toward the descriptive or suggests retaining components until at least 70-80
exploratory nature of PCA, the use of principal per cent of the variation is explained. Another sug-
Components with correlated observation vectors indi- gests retaining only those components whose vari-
cate process upsets remarkably well. ance li is greater than unity, or retaining only those
The application of PCA to statistical process components whose individual variances explain at
monitoring is increasing. Scranton et ~ 1 .applied
'~ a least a proportion l l m of the total variance is
multivariate exponentially weighted moving average another alternative. For this example, l/m = 117 =
control chart to principal component variables to 0.1428. Using these guidelines, the first three princi-
improve the detection of small process shifts. pal components or latent variables are retained and
Nomikos and Ma~Gregor'~ employ multiway PCA have the following form:
to monitor batch processes over time. Runger and Pc 1: 21= O.27hl + 0.270X6 + 0.267~7+ 0.260~4
discuss the selection of appropriate principal
components for multivariate process monitoring.
+
+ 0*006x,- 0.039~3 0.038~5
Pc2: z2 = 0.061~1- 0.05 1x6 - o*@ktr7+ 0*104x4
These analyses assume that the measurements
obtained at each sample time are uncorrelated.
+ 0.508~2+ O . 5 O h 3 - 0'045x5
Pc3: z3 = 0-047X1- 0.125X6 - 0.138x7 O.O78X4
Rungerl7 showed that the residuals from a PCA are + 0.041x2+ 0 . 0 0 0 2 ~+~0 . 9 7 6 ~ ~
not autocorrelated. The residuals can be monitored
in an application in which an assignable cause shifts Recall that the new variables, z,, are linear combi-
the process mean out of the subspace defined by nations of the original variables multiplied by the
the first few principal components. In this work, we eigenvectors of S, the sample covariance matrix.
focus on graphical approaches for the more difficult The factor pattern measures the correlation of
Problem of autocorrelated data. Exponential smoo- each principal component with the original variable
thing might be useful even for autocorrelated data, and is used to interpret the principal component by
and it might be combined with simple graphics, showing how much an original variable 'contributes'
but more research is needed to make good use of to ti. In Table 11, significant correlations between
the method. the original variables, xi, and the latent variables zi
The same philosophy used for univariate monitor- are highlighted. The first principal component can
ing can be applied in developing multivariate SPC be interpreted as essentially a weighted average of
Procedures; that is, start with an appropriate refer- variables 1, 6, 7, 4, whereas the second component
ence set that defines normal operating conditions for is an average of variables 2 and 3 and the third
the process, chart observations, and then look for component is primarily directed by variable 5. Note
assignable causes when a point signals out Of con- that the interpretation of the latent variables obtained
trol. by examining the factor pattern could be obtained
by examining the coefficients of xi in the principal
components. The factor pattern may be a more
EXAMPLE reliable guide in attempting to interpret the principal
Seven variables from a continuous chemical process components because the correlations correct for dif-
comprise the dataset. The observations are collected ferences in the variances of the original variables.
relatively frequently compared to the process time By substituting the values from the dataset into
constant, thus resulting in correlated data. Using the the above principal components, individual transfor-
Box and Jenkins'" approach, the time series were med observations, or z-scores, are obtained. Figure 4
estimated and fitted by ARIMA models. The depicts the in-control status of the process; this data
ANMA modelling was not required as part of the will serve as the reference distribution. The plotted
Principal components approach, but it gives some points are the z-scores of the first two latent vari-
idea of the stochastic structure of the data. The first ables, z , and z2, To determine the effectiveness of
three variables exhibit AR( 1) behaviour, variable 4 using principal components with autocorrelated data
is an ARMA( 1, I ) model, variables 5 and 6 are for process monitoring purposes, shifts of each vari-
4R(2), and variable 7 is modelled as an able were simulated. The factor scores for a lo
A N A ( 2 , l ) . Strong autocorrelation exists in this shift are shown in Figure 5. The numbers 51 to 60
dafa set which makes it typical of the data found denote the subsequent time periods after the simu-
In many process plants. lated shift began. As Figure 5 shows, two 'pools'
Table1 summarizes the variance contributions of of factor scores result with a transitional data stream
the principal components. Recall that the eigenvalue in-between. As the shift increases, the transitional
Of a principal component or latent variable rep- data stream becomes easier to recognize; thus
208 C. M. MASTRANGELO, G. C. RUNGER AND D. C. MONTGOMERY
Latent variables
Original variables, xi 22
v)
H2'
:I
LO]
-0.5
- 1 .o-j
' .
..
0
.
.. .
0
.
0 0
0
0
0
.
0
.
0
I
a
0
0.
. .
.. ..
0.
0
.
0
0
-.
0
0
-1.5 1 I I I I I I I 1
-6 -4 -2 0 2 4 6 8 10
Y2 Scores
Figure 4. Factor scores of the reference dataset
quickly indicating a process change. For the small operation, data far from this plane should be ident-
lo process shift incurred in this example, the shift ified. The supplemental multivariate control chart
is discernible after 6 or 7 periods. accomplishes this objective quite simply, and may
Owing to the geometry of PCA, the monitoring be interpreted as a control chart on the residuals of
of PCI and PC2 is effective for detecting assignable an observed data point (vector) from the projection
causes that shift the process mean in the plane of on the PCI-PC2 plane.
the first two latent variables. Assignable causes that This example demonstrates the appeal of using
change the process mean in a direction perpendicular principal component analysis to monitor a multi-
to this plane are not detected by monitoring these dimensional process with autocorrelated data. Instead
two variables, because the projection onto the PCI- of seven individual control charts, only one process
PC2 plane of data perpendicular to this plane score chart is needed to detect a process change.
appears identical to the original data. Consequently,
we strongly recommend monitoring the remaining CONCLUSIONS
m - 2 components with any multivariate control
chart such as the TL control chart, the MEWMA or In this paper, we have examined multivariate SPC
the MCUSUM. Although, our primary interest might methods for monitoring processes whose variables
be to observe the process in its plane of normal exhibit non-random behavior. We have shown that
STATISTICAL PROCESS MONITORING 209
3-
. ..
’
9 .
.‘a
.
*
’. . ’ .
.I
2-
s
.
. *.
’e
. . ..
m
a .*
60
1- I
.59
al
s 0-
, ’ . ... m56
.55
.. .. ...
W
*
8 . ‘$34
1.. a m
. . ’.. .
m m %
a . a *52
-1 - m m
. -51
. .
s
.*
-2 I I I I I 1 I
1
-5 0 5 10 15
Y2 Score
Figure5. Factor scores of a 10 shift in all variables
10. C. A. Lowry, W. H. Woodall. C. W. Champ and S. E.
the conventional P control chart may be less sensi- Rigdon, ‘A multivariate exponentially weighted moving aver-
the to autocorrelation because the lagged correlation age control chan‘, Technomerrics, 34, ( 1 ), 46-53 (1992).
of the Tt statistic is smaller than the lagged corre- 1I. J. V. Kresta, J. F. MacGregor and T. E. Marlin, ‘Multivariate
lation of the individual variables. Monitoring auto- statistical monitoring of process operating performance’, Can-
adian Journal of Chemical Engineering, 69. 3 5 4 7 (1991).
correlated process variables with latent variables was 12. J. E. Jackson, A User’s Guide to Principal Components.
also explored. Preliminary investigation shows that Wiley, New York. NY, 1991.
in applications where dimensionality may be reduced 13. I. T. Jolliffe. Principal Component Analysis, Springer-Verlag,
New York. NY, 1986.
to two or three dimensions, principal component 14. R. Scranton, G. C. Runger. J. B. Keats and D. C.Montgom-
analysis may be used as a viable process monitor- ery, ‘Efficient shift detection using multivariate exponentially-
ing tool. weighted moving average control charts and principal cornpo-
nenis’, Quality &d ReEability Engineering International, 12.
165-171 (1996).
15. P. Nomikos and J. F. MacGregor. ‘Multivariate SPC charts
ACKNOWLEDGEMENT for monitoring batch processes? Technomerrics, 37, ( 1 ) 41-
59 (1995).
b.Mastrangelo was supported for this work, in 16. G . C. Runger and F. B. Alt, ‘Choosing principal components
Part, by NSF under grant DMI-9501903. for multivariate statistical process control’. to appear in Corn-
municarions in Statistics-Theory and Methods.
17. G . C. Runger, ‘Multivariate statistical process control for
REFERENCES autocorrelated processes’. to appear in International Journal
of Production Research.
1. L. C. Alwan. ‘Autocorrelation: fixed versus variable Control 18. G . E. P. Box and G. M. Jenkins, Time Series Analysis:
limits’, Quality Engineering, 4. (2) 167-188 (1991). Forecasting and Control, Holden Day, Oakland. California,
2. L. C. Alwan, ‘Effects of autocorrelation on control chart 1976.
performance’. Communications in statisrics: Theory and
Methods, 21, 1025-1049 (1992).
3. L. C. Alwan and H. V. Roberts, ‘Time series modeling for
quality statistical process control’, Journal of Business and Authors ’ biographies:
Economic Statistics, 6, ( 1 ). 87-95 ( 1988).
4. D. S. Emer, ‘A control chart for dependent data’, ASQC Christina M. Mastrangelo is an Assistant Professor in
Technical conference Transactions, Atlanta, GA. 1980. the Department of Systems Engineering. She holds BS, MS
5. D. C. Montgomery and C. M. Mastrangelo, ‘Some statistical
and Ph.D. degrees in industrial engineering, and conducts
process control methods for autocorrelated data’. Journal of
Quality Technology, 23, (3). 179-193 (1991). research in statistical process control and empirical stoch-
6. c. A. Lowry and D.C. Montgomery. ‘A review of multivari- astic modeling. She is a recipient of the Ellis R. Ott
ate control charts’, IIE Transactions. 27. 800-810 (1995). and Shewell Awards. She has several years of industrial
7. H. Hotelling, ‘Multivariate quality control-illustrated by the experience in aerospace manufacturing and is a member
air testing of sample bombsights’ in C. Eisenhart, M. W. of IIE and ASQC.
Hastay and W. A. Wallis (eds) Techniques of Statistical
Analysis, McGraw Hill, New York, NY. 1947. George C. Runger. Ph.D. is a faculty member in Manage-
8. J. J. Pignatiello and G. C. Runger. ‘Comparison of multivari- ment Science and Statistics at the University of Maryland.
ate CUSUM charts’, Journal of Qualify Technology, 22, 173-
186 (1990). His research interests are in process control and exper-
9. R. B. Crosier, ‘Multivariate generalizations of cumulative imental design strategies. He has published in journals
sum quality control schemes’, Technometrics. 30, 291-303 such as the Institute of Industrial Engineering Transactions,
(1988). Journal of Quality Technology, Quality Engineering, Jour-
210 C. M. MASTRANGELO, G. C. RUNGER AND D. C. MONTGOMERY
nal of the American Statistical Association, International in the development and application of statistical method-
Journal of Reduction Research, IEEE Transactions, and ology for problems in engineering and the sciences. Dr.
the Journal of Multivariate Analysis. He is a co-author of Montgomery is an author of 10 books and numerous
the text Applied Statistics and Probability for Engineers. technical papers. He is a recipient of the Ellis R. Ott
Formerly, he was a senior engineer at IBM and on the Award and both the Shewell Award and Brumbaugh
faculty at Rensselaer Polytechnic Institute. He holds Award from the American Society for Quality Control.
degrees in Industrial Engineering and Statistics. He is currently the editor of the Journal of Quality Tech-
nology. Dr. Montgomery is a Fellow of the American
Douglas C. Montgomery is Professor of Engineering at Statistical Association, a Fellow of the American Society
Arizona State University. He received the Ph.D. in engin- for Quality Control, and a Fellow of the Institute of
eering from Virginia Tech. His professional interests are Industrial Engineers.