Professional Documents
Culture Documents
Using Run-Length Distributions of Control Charts To Detect False Alarms
Using Run-Length Distributions of Control Charts To Detect False Alarms
Using Run-Length Distributions of Control Charts To Detect False Alarms
MANAGEMENT 1994
USING
HERBERT
Krannert GraduateSchoolof Management,Purdue University, WestLafayette, Indiana 47907, USA Krannert GraduateSchoolof Management,Purdue University, WestLafayette, Indiana 47907, USA David EcclesSchoolof Business, University of Utah, Salt Lake City, Utah 84112, USA
Run-length distributions for various statistical process-control charts and techniques for computing them recently have been reported in the literature. The real advantages of knowing the run-length distribution for a process-control chart versus knowing only the associated average-run length of the chart have not been exploited. Our purpose is to use knowledge of the run-length distribution as an aid in deciding if an out-of-control signal is a true signal or merely a false alarm. The ability to distinguish between true and false signals is important, especially in operations where it is costly to investigate the causes of out-ofcontrol conditions. Knowledge of the run-length distribution allows us to compute likelihood ratios, which are simple to calculate and to interpret and which are used to determine the odds of obtaining an out-of-control signal at a particular run length when a shift in the process mean actually has occurred vis-a-vis no such shift. We extend our analysis in a Bayesian sense by incorporating prior information on the distribution of the shift size of the process mean, combined with the likelihood ratio obtained from the run-length distribution, to determine if a shift largerthan a critical size has occurred. We give examples for the Shewhart chart, the exponentially weighted moving-average chart, and the special-cause control chart for processes with autocorrelated observations. The examples show that the current recommended usage of the average-run length alone as a guide for determining whether a signal is a false alarm or otherwise can be misleading. We also show that the performance of the traditional charts, in terms of their average-run length, can be enhanced in many instances by using the likelihood-ratio procedure. (STATISTICAL PROCESS CONTROL; AVERAGE-RUN LENGTH; FALSE ALARMS)
1.
Introduction
The use of statistical process-control (WC) charts, and in particular Shewhart charts, has become widespread in recent years because of an increasing emphasis on improving quality so as to improve product and service competitiveness. Traditional SPC charts are simple to implement and are very effective in helping alert manufac* Received April 1992; revision received January 1993; accepted May 1993. 217 1059-1478/94/0303/217$1.25
Copyright 0 1994, Production and Operations Management Society
218
HERBERT
MOSKOWITZ,
ROBERT
D. PLANTE,
AND
DON
G. WARDELL
turers to the presence of quality anomalies in their processes. Shewhart charts are basically set up as hypothesis tests on the quality characteristic being measured. Control limits on these charts are determined so as to control the probability of obtaining a point outside the control limits when the process is in control (i.e., to control the probability of a type I error, a). It is common practice in the United States to set the limits of a Shewhart chart at +3 standard deviations, giving an (Y value of 0.0026. When an observed value falls outside the control limits, the process is generally considered to be out of control and a search for an assignable cause is initiated. One of the most important properties associated with any SPC chart is the run length. The run length is the number of observations required to obtain an observation outside of the control limits for a given shift in the mean (Wadsworth, Stephens, and Godfrey 1986). The count of the run length is initiated at the beginning of a production run or at the first observation after an out-of-control signal has occurred. When the process is in control, the run length should be long but is not infinite because there is a probability ((Y) of a point being outside the limits even when the process is in control. This is referred to as a false alarm. In many cases investigating a false alarm can be very costly, especially when the process must be shut down or if highly skilled workers must be paid to conduct the investigation. When an observation falls outside of the control limits and the cause of the out-of-control signal is not obvious, the operator is faced with the problem of deciding whether a point is a true signal or is simply a false alarm. In this paper, we show how the run-length distribution can be used to assist such a determination, thereby enabling more informed use of control charts by process operators. 1.1.
Several researchers have been concerned explicitly with the need to distinguish between actual signals and false alarms, particularly in the area of the economic design of control charts (Montgomery 1980). Such economic designs trade off the costs of investigating false alarms against the costs of producing defective items, taking into account the probability of a false alarm (a) as well as the probability of detecting an out-of-control state given it has actually occurred (called 1 - /3, where /3 is defined as the probability of a type II error, i.e., the probability of obtaining a point within limits given that the process is not in control). Such designs also make use of the fact that the run length of the Shewhart chart is a geometric random variable with mean 1/( 1 - 8) ; however, this is only used to find the average-run length ( ARL), because the objective is to minimize the expected cost per unit time. Using a similar idea, Mayer ( 1983) discussed the issue of trading off the type I and II errors associated with control charts in a discussion on the validity of using +3 standard deviations for control limits in every process environment. He argued that when the cost of investigating a false alarm is relatively high, then wider limits should be used, whereas if the cost of producing defective items is very high, then narrower limits should be used. The exact width of the limits, he suggested, should be determined by experience and good judgment on the part of the user; however, he gave no specific guidelines. A second area where there is special interest in deciding whether a signal is a false alarm is in knowledge-based systems applied to quality control. Hosni and Elshennawy ( 1988) developed an expert system that uses dialogue with the user to select an
RUN-LENGTH
219
appropriate control chart and then interprets its signals. They accomplished this by constructing a matrix that they used to link various out-of-control signals to possible assignable causes. Although they mention the possibility of getting a false alarm, their expert system treats every out-of-control signal as a true alarm. Evans and Lindsay ( 1988) developed a similar system; they note that it is difficult to discern whether an out-of-control point is a random outlier or a genuine shift in the process level. However, in most instances their system also treats signals as true alarms, except in some cases in which it waits for more information to make a decision, based on the type of out-of-control signal experienced. Finally, Robinson and Ho ( 1978), Crowder ( 1989), and Lucas and Saccucci ( 1990) use the ARL of the exponentially weighted moving-average (EWMA) chart to design control schemes that are intended to control the frequency of false alarms while simultaneously increasing the chance of detecting a specified size shift once it occurs. This is possible with the EWMA chart because there are two parameters that can be manipulated, namely, the smoothing constant, X, and the multiplier of the standard deviation, defined as L. Both Lucas and Saccucci ( 1990) and Crowder ( 1989) show that by appropriately setting the values of h and L, one can often specify a control chart that gives a high ARL when the process is in control and a relatively small ARL when the process has shifted by a specified magnitude.
220
HERBERT
MOSKOWITZ,
ROBERT
D. PLANTE,
AND
DON
G. WARDELL
The run-length distribution of a particular control chart depends on the type of assignable cause. In all cases, we assume that the type of assignable cause to be detected results in a step change in the process mean. The run-length distribution of the control charts is then actually a conditional distribution contingent on the value of the shift in the process mean.
RUN-LENGTH
221
where Nis the run length or the number of subgroups taken until a signal is obtained. If, as is usually assumed, the means of the subgroups are normally distributed, then ~0) is p(6) = 1 - @(L - 6) + (P(-L - 6),
(2)
where 9 is the standard normal distribution function and L is the number of standard deviations used to find the upper and lower control limits (usually assumed to be 3).
Qt = (1 - x)9(-1 + ti,
where ZZ is the sample mean and Ql is the value of the EWMA after observation we let the initial EWMA value, QO, be u, then we have (Crowder, 1987) P(N=llu,s)=l-* and h-(1-X)u+ x _ ax +~ -h-(1-X)u -6
(3) t. If
P(N= UIU, 6) =;
s h fYN=u-
llY,S)(P
y-(1
1 [ Au, 1 1
- X)u dy, x _
0.x
(4)
forv22,
(5)
where h = the control limits of the EWMA chart = Laifx/2 ui = the standard deviation of the sample means X = the smoothing constant in the EWMA model 50 = the normal density function.
222
HERBERT
MOSKOWITZ,
ROBERT
D. PLANTE,
AND
DON
G. WARDELL
where
+*
and a, = the ua = the @i = the 8i = the
L-65
1
aa [
1+~(1-8rl)]]
(7)
standard deviation of the observations standard deviation of the residuals autoregressive parameter in the ARMA ( 1, I ) model moving average parameter in the ARMA ( 1,l) model.
The value of the likelihood ratio, A, ranges from 0 to cc. If A is equal to 1, the informationless case, we have a 50-50 chance of obtaining a signal at v when a shift has. occurred. If the ratio is greater than 1, then it is more likely that a shift has occurred. On the other hand, when the ratio is less than 1, the signal is more likely a false alarm than a true indication of a process shift. We assume for illustrative purposes only that any value of A below a cutoff ratio of 1 is judged a false alarm. The choice of the cutoff ratio (action limit), however, depends on the economics of the situation (i.e., on the relative costs of searching for a false alarm versus producing low-quality products), as well as on the managers risk attitude, and is described in Section 2.5. Using this likelihood ratio, A, we now describe a procedure to determine if a signal is a false alarm. First, the likelihood ratio in (8) is computed off-line (only one time) for each value of the run length, v, using the hypothesized value of 6*. These ratios can be printed directly on the control chart (see Figure 1b and the accompanying discussion in Section 4.1). We start the run-length clock at the beginning of a production run or at the first observation after an out-of-control signal has occurred. Each and every time an out-of-control signal is obtained, the clock must be restarted. We then plot the mean of the subgroups (or the residuals in the case of the see chart) using the appropriate control chart for the process under investigation. When a point on the control chart falls outside of the control limits, we determine if the likelihood ratio at that run length is greater than 1 (our action limit). If it is, we judge the signal more likely to be true than false and begin a search for an assignable cause. However,
RUN-LENGTH
DISTRIBUTIONS
OF CONTROL
CHARTS
223
if the ratio is less than 1, we conclude that such a signal is more likely to be a false alarm. After either kind of signal, we would restart the clock and keep track of the time until the next signal is given. In addition, for the EWMA chart, we must reset the forecast to QO to begin the process again, because the run-length distributions of the EWMA chart are derived assuming the starting point of QO. We should emphasize that in some cases the process will shift to the out-of-control state after a significant amount of time. When this occurs, the run length will be long, and the signal will be considered a false alarm using the likelihood ratio. However, because we restart the run-length clock immediately after the out-of-control signal, another signal will be given very soon thereafter. The new signal will be considered a true signal by the likelihood ratio. In practice, if false alarms are costly to investigate, it is often true that operators will not always act on an out-of-control signal but will instead wait for additional evidence. The approach described in the preceding paragraph formalizes the procedure for deciding when to wait for additional evidence and whether there is enough evidence to conclude that the process is out of control. We further discuss the effect of a delayed shift on chart performance in Section 5.2. 2.5. Determining the CutoflRatio
In Section 2.4 we used a cutoff ratio of 1 to illustrate the procedure of assessing the likelihood that a signal is a false alarm. As we mentioned, the choice of the cutoff ratio depends on the economics of the situation, as well as on the managers risk attitude. We use a simple decision analysis method to determine the cutoff ratio, which we refer to as A*. When an out-of-control signal is given, a conclusion must be made concerning the level of the process. This conclusion can be reached by framing the problem using a simple test of hypothesis. The null hypothesis, HO, is that the mean has not shifted, and the alternative hypothesis, H1, is that the mean has shifted to a new level. The cost of rejecting HO when it is true is the cost of a false alarm and is denoted cFA. The cost of rejecting H1 when it is true is the cost of concluding the process is in control when the mean has actually shifted and is denoted cD. If we wanted to incorporate the risk attitude of the manager, we could use utilities instead, and the approach would be similar. The decision to accept or reject HO depends on these costs, as well as on the odds ratio, R, defined as Q = P(Ho) P(HI) (9)
where P( Ho) is the probability that the mean is on target, and P( HI ) is the probability that it has shifted. The decision rule is to accept HI if 52 cFA/cD < 1 and to accept Ho otherwise ( Winkler and Hays, 1975 ) . The odds ratio can be either a prior odds ratio or a posterior odds ratio. In this case we are basing the decision as to the state of the process on whether or not an out-of-control signal is given. In other words, we are obtaining sample information that can be used to update the odds ratio. Therefore, the posterior odds ratio, C, depends on the prior odds ratio, G , and the likelihood ratio, A. If p is the prior probability that the process is on target, then
Q&P
1 -p
(10)
224 and
HERBERT
MOSKOWITZ,
ROBERT
D. PLANTE,
AND
DON
G. WARDELL
Q2= P -! 1 -ph. Using the posterior odds ratio, the decision rule becomes accept H1 if A > pcFA/( 1 - p)cD and accept HO otherwise. Therefore the cutoff ratio is
A* = ( 1F;)cD.
(12)
Hence, the cutoff ratio is a function of the costs of the two types of errors, as well as of the prior probabilities concerning the state of the system. The value of p would probably depend on the time the signal is given. As an example, we might model the process failure mechanism with an exponential distribution, with the mean time until the shift occurs equal to 7. If we obtain a signal at observation v, the value of p would be set to e-lr. Here our prior probability that the mean has shifted is small soon after the process is started or adjusted and becomes larger as the process continues. Note that with this type of prior probability, A*, is different for each value of v. However, just as is the case with A, it must only be computed one time, offline, and the relevant values can be given to the operator for easy reference. 3. Assigning a Prior Distribution to Possible Process Shifts
Suppose that a manager is not so much interested in the odds that a specified process shift has occurred but more so in the odds that the process may have shifted above a specified value. Then, because the run-length distribution is conditional, we can extend the analysis by assuming some distribution on 6, namely, that 6 is a nonnegative continuous random variable with probability density function f( 6). Knowledge of this distribution enables us to determine the probability of an out-ofcontrol signal given that the shift in the mean is above or below some critical shift, 6*. 1.n this case we need to find the probabilities of obtaining an out-of-control signal at subgroup v given that a process shift is either less than or greater than 6*. That is, we want P(N = v 16 > 6*) and P(N = v I 6 5 6* ) as follows, where F( 6) represents the distribution function of 6:
With the two conditional probabilities, ( 13 ) and ( 14)) we can compute the likelihood ratio of obtaining an out-of-control signal at subgroup v given that we are interested in detecting any shift greater than size 6* as follows: A(N= via*) = P(N= P(N= vl6> a*) VI6 I a*) (15)
RUN-LENGTH
DISTRIBUTIONS
OF
CONTROL
CHARTS
225
We can also compute the posterior probability that a shift of size 6* or greater has occurred given the run length at which an out-of-control signal is detected. To do this, we must first obtain the unconditional probability of N: P(N = v) = j->o P(N = vl6)f(6)dS. Now we can determine
I(6 > 6*IN (16)
The choice of a density function forf( 6) obviously affects the likelihood ratio, A. In the next section, we investigate, among other issues, the robustness of the likelihood ratio method with respect to choice of a prior distribution on the shift size. 4. Effectiveness of Shift Detection via Likelihood Ratios
We investigate the effectiveness of using the run-length distribution for the Shewhart, EWMA, and see charts to identify false signals. We first discuss the usefulness of the run-length distributions assuming no prior information on possible process shifts. We then use the Shewhart chart to examine ( 1) the robustness of the likelihood ratio with respect to the choice of a prior distribution for shift size and (2) the effect of the average shift size of the prior distribution on the posterior distribution of 6*. To facilitate discussion, we assume that the critical or cutoff value of A is 1. That is, if the likelihood ratio is greater than 1, the signal is considered to be valid; however, if the ratio is less than 1, the signal is a false alarm.
226
HERBERT
MOSKOWITZ,
ROBERT
D. PLANTE,
AND
DON
G. WARDELL
100
Likelihood
-41.I.I. 1
j. 4 Observation
I. 5 Number
! 6
.i. 7 8
a. LikelihoodRatiosfor the Shewhart Chartfor 3 Different Shiftsin the Mean. b. A Sample Shewhart Chartwith LikelihoodRatiosfor 1 Standard DeviationShiftsIndicated.
the operator can simply observe the ratio at that point and use it to decide if the signal should be considered a true or false alarm. 4.2. The
EWMA
Chart
We now illustrate the usefulness of the run-length distribution for the EWMA chart by considering some examples given in the literature. The first example is from Lucas and Saccucci ( 1990) in which they illustrate the EWMA control scheme with X = 0.25 and L = 3.0, where a shift of one standard deviation is introduced after the 10th observation. Again the standard deviation referred to is a~ = a,/ 6, although often for the EWMA chart iz = 1. Figure 2 shows the EWMA chart and the associated
RUN-LENGTH
DISTRIBUTIONS
OF CONTROL
CHARTS
227
-2-
10
15
20
0 FIGURE 2. Application
10 of the Likelihood
20
30
40
50
a. The EWMA Chart for the Lucas Example. b. Likelihood Ratios for the EWMA Chart with X = 0.25 and L = 3.0.
likelihood ratios for X = 0.25 and L = 3.0, for shifts of one, two, and three standard deviations. As shown in Figure 2a, the first occurrence of a point plotted outside of the control limits is obtained after 16 observations. Examining the plot in Figure 2b, we seethat if we were interested in detecting a shift of 1a~, then we would conclude that the signal was legitimate. On the other hand, if we were looking for a shift of
228
HERBERT
MOSKOWITZ,
ROBERT
D. PLANTE,
AND
DON
G. WARDELL
two or three standard deviations, we would conclude that the signal was a false alarm and restart the procedure, including the EWMA calculations, starting again with Q0 = 0. A second example is from Crowder ( 1989)) in which he uses the in-control and out-of-control ARLS to design an EWMA control scheme. It is stated that the desired in-control ARL is 250 and that we wish to design a chart that will quickly detect a shift of size one standard deviation. The chosen parameters are then X = 0.15 and L = 2.65. Figure 3 shows the plot of the likelihood ratios for this case. What is important to note is that although the chart was designed to give a false signal on average every 250 observations, the SRL is so large that the likelihood ratio test would suggest that after 22 observations, an out-of-control signal is more likely a false alarm. Hence, the likelihood ratio test gives control chart designers and users another tool for designing and using EWMA charts that is not available by simply specifying the desired ARLS. 4.3. The see Chart
The final set of examples are for the scc chart for correlated data. For the scc chart, the sample size is always one, so the shift is now in terms of a,, the standard deviation of the observations. Wardell, Moskowitz, and Plante ( 1994) showed that when the data can be described by an ARMA ( I,1 ) model, processes that are negatively autocorrelated have much smaller ARLS than those that are positively autocorrelated. For instance, when 4, = 0.0 and 0r = 0.9, the first autocorrelation coefficient is -0.497 and the ARL at a shift of one standard deviation is 2.76; in contrast, when
-2
-3
10
20 Run Length
30
40
50
FIGURE
3.
Likelihood
Ratios
Chart
With
RUN-LENGTH
DISTRIBUTIONS
OF CONTROL
CHARTS
229
#Q = 0.95 and 0, = 0.45, the first autocorrelation coefficient is 0.824 and the ARL is 274.7 for a shift of one standard deviation. For both cases, the in-control ARL is 370.4. Figure 4 shows the likelihood ratios for these two examples. The results show that in both cases we would conclude that after the fifth or sixth observation any out-of-control signal is more likely a false alarm than a true signal. We also see in this example that knowledge of both the ARL and the likelihood ratio is useful. When the ARL is small (Figure 4a), the distinction between false alarms and true signals is very clear since the likelihood ratio drops quickly below 1, but when the ARL is large (Figure 4b), the distinction becomes much less discernible, as is observed by the flatness of the curve near a likelihood ratio of 1.
Size of the shift in standard 1 deviations
8 Run Length
12
16
20
12 Run Length
16
20
FIGURE
4.
Likelihood
Ratios
for Two
ARMA(
1,l)
Models.
Autocorrelation).
230
HERBERT
MOSKOWITZ,
ROBERT
D. PLANTE,
AND
DON
G. WARDELL
The exponential density assumption for the shift in the mean appears plausible, because it is presumably more likely that a shift in the mean of a process is small (Duncan, 197 1) . The assumption that 6 follows an exponential density also allows us to examine the effect of having a system subject to small changes (where the average shift, 1 /p, is small) versus one where the average shift in the mean is large. We only present results for the Shewhart chart, as many of the results here are extensions of the results reported in earlier sections. Figure 5 shows three examples for which the average shift is equal to 1 and where we desire to detect a shift (a*) of less than 0.5, 1, and 2 standard deviations. It can be seen that as we increase 6*, the run length at which we determine a signal to be a false alarm decreases. This makes sense because large shifts should produce a signal immediately; if we are expecting large shifts, we would ignore signals caused by smaller shifts. Thus, unless we obtained a signal immediately, we would conclude that the signal was caused by a smaller shift than what we were concerned about and looking for. On the other hand, if we were interested in detecting smaller shifts, we would expect that later signals would be legitimate, because it takes longer to detect smaller shifts. These results agree with the cases discussed in Section 4.1 where no prior distribution was assumed.
TO PRIOR EXPONENTIAL PARAMETER. 4.4.1. SENSITIVITY plots in which 6* remains constant at one standard deviation,
I -
ko.5 a*=1
&*=2
20 Run Length
40
60
FIGURE
5.
Likelihood
Ratios
Chart
With
Prior
Exponential
Parameter
p = 1.
RUN-LENGTH
DISTRIBUTIONS
OF CONTROL
CHARTS
231
I=2
p=l pO.5
6.
Sensitivity
of Likelihood
Ratio
to Prior
Exponential
Parameter
p for 6* = 1.
of the exponential distribution is changed. From Figure 6 it is clear that as the average shift in the mean increases (that is, as p decreases), the run length at which we determine a signal to be a false alarm decreases. This again is consistent with the results where no prior distribution was assumed. When the average shift in the mean increases, we are more likely to obtain a signal sooner and less likely to obtain a signal later.
TO PRIOR DISTRIBUTION: EXPONENTIAL VERSUS UNIFORM. 4.4.2. SENSITIVITY Figure 7 shows a comparison of the likelihood ratios when the distribution of the shift in the mean is assumed to be exponential, compared with when it is assumed to be uniform. Figure 7a shows that when p = 1 and 6* = 1, there are only minor differences in the results, especially near the point where a signal is deemed to be a false alarm. Figure 7b shows another example with different parameters (p = 2, 6* = 0.5 ) . Again the results are similar. Thus, it appears that the time at which we begin to conclude signals are false alarms is not very sensitive to the prior distributions or parameters chosen. Overall the likelihood ratios appear to be reasonably robust with respect to the selection of a prior distribution.
4.5.
Posterior Probability
Revision
The following set of examples uses ( 17 ) and ( 18 ) to compute the posterior probability that a shift in the process mean has occurred that is above or below a specified value 6* for a given run length. For these examples we use a Shewhart chart and assume that the prior distribution of possible process shifts is exponential with parameter p. For p .= 1, Figure 8 shows the probability that a shift greater than size 6* has occurred at a given run length, for three different 6* values. For 6* = 1, Figure 9 shows the probability that a shift greater than 6* has occurred given the run length for three different values of p. When the posterior probability falls below 0.5, it is more likely that an out-of-control signal is a false alarm. The graph in Figure 8 is similar to Figure 5 in the sense that when 6* increases, the run length for which a signal is considered a false alarm decreases. Interestingly, the graph in Figure 9 depicts much different behavior from that shown in Figure 6. As shown in Figure 9, when the average shift in the mean ( 1 /p) increases, the run length before an observation
232
HERBERT
MOSKOWITZ,
ROBERT
D. PLANTE,
Exponential Uniform
AND
DON
G. WARDELL
.-
12
16
20
24
28
32
36
40
Run Length
8 0 ; d -8 8 c = s -1 I 6 5 4 3 2 1
16
24
32
40
Run Length
Ratios with Average Shift = 1 and 6* = 1 for Exponential Ratios With Average Shift = 2 and 6* = 0.5 for Exponential
is considered a false alarm increases. This is true because the prior probability that 6 > 1 increasessignificantly as p increases.
5. Performance of the Likelihood Ratio Test
We evaluated the effectiveness of the proposed procedure by simulating the ARL of the control charts when using the likelihood ratio. In all cases,the size of the critical shift, 6*, was 1aZ, and the critical or cutoff value of A was 1. We show only the results for the casewhen no prior distribution is assumed on the sizeof the mean shift. We examined the following two situations: ( 1) when a shift occurs immediately and (2) when the shift occurs at some random time in the process. 5.1. ARL for an Immediate Shift Table 1 compares the simulation results, in terms of the ARL, for the Shewhart chart with the likelihood ratio test versus when the test is not used for an immediate
RUN-LENGTH
233
0.8
0.6
0.4
0.2
0.0
0
20
40
Run Length, N
60
80
100
FIGURE 8. Posterior Probabilities of Being Out of Control for the Shewhart Chart with p = 1.
process shift. The results show that the likelihood ratio test is effective. When using the likelihood ratio test, the ARL when there is no shift in the mean is increased from 370 to almost 1,600 observations. Thus, on average, the frequency of false alarms is reduced by a factor of four. However, when a shift of 1ai occurs, the ARL is slightly larger than with the Shewhart chart alone. For shifts smaller than the critical shift of la,, the ARL is larger than with the Shewhart chart alone. Further, when shifts are larger than 1a*, the ARL is essentially the same, because nearly every signal occurs before the likelihood ratio falls below I. The results for the EWMA chart are very similar (Table 1b). Here we used the second EWMA chart described above in Section 4.2, with X = 0.15 and L = 2.65. Table 1b shows that the ARL, when there is no shift, becomes very large, whereas the ARLS for shifts at or above the critical shift remain about the same. Because the results for the EWMA chart are similar to those of the Shewhart chart, we give further results only for the Shewhart chart.
0.8
E x $ z
0.6
p=2
p=l pd.5
20
40
60
80
100
234
HERBERT
MOSKOWITZ,
ROBERT D. PLANTE,
ARL
TABLE 1 With Versus Without the Likelihood Simulated ARL With the Test
ARL
ARL
a. Comparison ofthe 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Size of the Mean Shift (in terms of IJ~) 370.38 158.73 43.89 14.97 6.33 3.24 2.00 Without the Test
of the Shewhart chart 1443.28 308.76 47.29 14.88 6.21 3.28 2.02 Simulated ARL With the Test 48.08 11.16 1.51 0.39 0.16 0.08 0.04 95% Confidence Limits for Simulation
ARL
of the
ARL
of the
EWMA
chart 237.10 3.77 0.35 0.12 0.07 0.05 0.04 95% Confidence Limits for Simulation
247.3 26.87 8.75 5.05 3.59 2.82 2.36 Without the Test With Limits of *3u* the.limits
3735.83 52.85 9.18 5.08 3.58 2.80 2.39 Simulated ARL With the Test and Limits of k2.62~~
Size of the Mean Shift (in terms of Us) c. Effect of narrowing 0.0 0.5 1.0 1.5 2.0 2.5 3.0
When the shift is small, or even when the shift is at the critical value, the ARL using the likelihood ratio test is longer than with the Shewhart chart alone. This is potentially undesirable but can be compensated for by selecting a critical value that is somewhat smaller than what is actually desired or by narrowing the limits of the chart so that the frequency of false alarms increases somewhat, but shifts are detected earlier. As an example, Table lc shows the results when the Shewhart limits are changed to k2.62, and the likelihood ratio is used. Here we have chosen the control limits of the Shewhart chart such that when the likelihood ratio method is used to test for false alarms, the ARL is about 370 when no shift has occurred. This is the same ARL as that of the standard Shewhart chart (Table 1a). Notice, however, that when shifts actually do occur, the chart with narrower limits combined with the likelihood ratio test detects the shifts more quickly on average. In this case, using the likelihood ratio test to check for false alarms has allowed us to narrow the limits
RUN-LENGTH
DISTRIBUTIONS TABLE
OF CONTROL 2
CHARTS
235
Simulated oJtheShewhart ARL Chart UsedWith theLikelihoodRatio TestWhenthe Time Until the MeanShift is Delayed Accordingto an Exponential Distribution
Mean Shift (in terms of a2 ) Average Number of Observations Until the Shift Occurs 50 100 200 500
Average Number of Observations Until the Shift Occurs 50 100 200 500
ARL
c. Simulated values ARL whenthe underlying distribution theprocess of failure mechanism* used is to determine likelihoodratio the 0.0
0.5 1.0 1.5 2.0 2.5 3.0 370.38 158.73 43.89 14.97 6.33 3.24 2.00 837.670 26 1.869 52.224 16.979 7.126 3.649 2.240 distribution. 367.142 145.349 36.824 12.725 5.678 3.115 1.984
* The distribution is assumed to be the exponential until the shift occurred was 100.
without increasing the frequency of false alarms. Simultaneously, the narrower limits allow the chart to detect true shifts more quickly.
5.2. ARL for a Delayed Shift
The above case of an immediate shift is the most favorable condition when the likelihood ratio test is used to check for false alarms. For example, if the shift does not occur until after the point where the likelihood ratio falls below 1, any signal will be deemed a false signal. Our procedure acknowledges this possibility, and that is why the run-length clock is restarted after any out-of-control signal is observed.
236
HERBERT
MOSKOWITZ,
ROBERT D. PLANTE,
To evaluate the effect of delayed shifts, we assumed that the time of the shift in the mean followed an exponential distribution. Table 2a shows the simulated ARL values as a function of the mean number of observations until the shift occurs, where the control limits of the Shewhart chart were set at *3u,. We have not included the confidence intervals for the simulation, but, as with earlier results, they are about three to four percent of the ARL. As before, the frequency of false alarms is very low. However, when shifts occur, the average time to detect those shifts increases as the mean time to shift increases. For most decision makers, this increase in time to detect shifts would be unacceptable. However, this does not invalidate the method. As discussed previously, it is possible to narrow the limits to help detect shifts more rapidly, while not appreciably increasing the frequency of false alarms. Table 2b shows the results when the control limits are reduced to k2.62~~. As before, when there is no shift, the ARL is about the same as that of the Shewhart chart without the likelihood test. But when small shifts occur, using the narrow limits with the likelihood ratio test detects the shift more quickly, even when the average time until the shift occurs is high. The average time to detect large shifts (2uf or larger) is higher than with the Shewhart chart alone. However, this is not viewed as a major drawback of the likelihood ratio method. Our procedure has been conceived and designed to assist operators in circumstances when they are unsure whether a signal is a false alarm or not. When the shift is large, the observation will likely exceed the control limits significantly, thus providing an obvious signal that the process is out of control. When small shifts occur, however, observations that exceed the control limits will do so marginally, and more guidance is needed in these circumstances for determining whether the signal is a false alarm. The proposed likelihood ratio test performs well under these conditions. Again, we could establish the limits between k2.62~~~ and &3uz, which would yield a longer in-control ARL but would not detect shifts as quickly. The decision on where to set such limits would depend upon the economic trade-off of obtaining more false alarms versus detecting shifts sooner. If the time that the process shifts follows an exponential distribution, and its mean can be estimated, it is possible to derive a new run-length distribution for the Shewhart chart when a shift occurs (see the Appendix). This new run-length distribution can then be used in the numerator of (8) to determine new likelihood ratios. For example, if the underlying distribution of the process failure mechanism is exponential, with mean time until failure equal to 100 observations, the likelihood ratio is as shown in Figure 10. In this case, for a shift of one standard deviation, the ratio does not fall below 1 until the 212th observation. is used to Table 2c shows the ARL values when the new run-length distribution find the likelihood ratio, when the time until the mean shift follows an exponential distribution with mean 100, for Shewhart limits of +3u, and k2.85~~. The second case gives an in-control ARL of about 370 again and can thus be compared with the Shewhart chart per se. When ?3u, limits are used, the frequency of false alarms is decreased by more than two times. However, there still exists the problem of detecting shifts more slowly. Again, this can be offset by narrowing the control limits. For example, when the limits are tightened to +2.85u,, the in-control ARL is the same as that of the Shewhart chart alone, but shifts are detected more quickly. Hence, when there is a delay in the process shift, the performance of the likelihood ratio method improves by explicitly incorporating the underlying distribution of the process failure mechanism into the likelihood ratio calculation.
RUN-LENGTH
DISTRIBUTIONS
OF CONTROL
CHARTS
231
-2
-3
50
150
200
250
6.
Discussion
We have introduced a straightforward but useful method for assessing the validity of out-of-control signals given by SPC charts. In its simplest form, this method requires no information on the prior distribution of the size of the shift in the process mean. However, it is possible to use a prior distribution, which allows us to compute the posterior probability that a process is truly out of control given that an out-of-control signal is obtained at a particular run length. Further, the results appear to be robust with respect to an assumed prior distribution of possible shifts in a process mean but not with respect to the average shift size derived from the prior distribution. We have shown that the proposed procedure performs well in terms of ,+RL under most circumstances when compared with the traditional Shewhart chart used alone. The proposed procedures use basic statistical concepts to aid in making processcontrol decisions. More advanced decision-making methodologies could be used to enhance the procedure, such as incorporation of the risk attitude of the decision maker. Also, this method could be incorporated into a decision support/knowledge-based system that could compute the likelihood ratios and make suggestions (with explanations) to the user about the possibility of a signal being a false alarm. Although the likelihood ratio method we have developed is explicitly statistical, we plan to more formally develop an economic model in the future. We have shown that with Shewhart limits of +3 standard deviations, the likelihood methodology significantly reduces the frequency of false alarms. However, at the same time, the time to detect shifts increases. We also have shown that we can tighten the limits, which improves the time to detect shifts considerably but increases the frequency of false alarms to a small degree. An economic model will give specific guidelines for setting the control limits to optimally balance the time to detect shifts and the frequency of false alarms.
238
HERBERT
MOSKOWITZ,
ROBERT
D. PLANTE,
AND
DON
G. WARDELL
Appendix.
Incorporating the Distribution of Shift Time into the Run-Length Distribution of the Shewhart Chart
Here we show how the underlying distribution of the process failure mechanism can be combined with the geometric run-length distribution of the Shewhart chart to obtain a new run-length distribution. In this case we are defining the run length as the number of observations required to obtain an out-of-control signal from the time the process is either initiated or adjusted to target. We assume that the underlying distribution of the process failure mechanism is the exponential distribution. Let the random variable T represent the time at which the mean shifts, and the mean value of T be T. Again we let N be the run length, as defined above. The probability of obtaining a signal outside of the control limits depends on whether or not the mean has shifted. Therefore, we can write the probability as
P(N=v)=P(N=vlT-cv)P(T<v)+P(N=vlT>v)P(T>v).
(Al)
P( T -C V) is the probability that the mean has already shifted, and P( T > v) is the probability that it has not. The random variable T is obviously continuous. However, we discretize it for our purposes. If the shift occurs at any point between v and v + 1, we assume that it occurred at observation v. If we make this assumption, then we can write p( T = t) = 1 If we let r = e-l, i e-ldt = e-t/T( 1 - e-/~).
(AZ)
(A3)
Note that this discrete version of the exponential bution. We can now write (A 1) as v-1
P(N= v) = 2 P(N=
i=l
distri-
vlT=
i)sr+
5 P(N=
i=u
vlT=
i)sr.
(A4)
We first simplify the second term of (A4). The conditional probability in this term is the probability that the observation has exceeded its limit, given that the shift has not yet occurred. This is the probability of obtaining a false alarm at observation v and is P(N=
vlT= i) =poq6-,
where p. is the probability of obtaining the mean is on target, and qo is 1 - po. The conditional probability in the first term of (A4) can be given as
P(N = VI T = i) = q&plq;--i,
fori
v,
for i < v,
(A61
where pI is the probability of obtaining a signal outside of the Shewhart limits when the mean has shifted, and q1 is 1 - pl. This expression shows that the probability of getting an observation outside of the limits on the vth observation, given that the mean shifted at time i, is the joint probability of the following events: that no false alarm was given before the mean shifted (represented by qk), that no signal was given v - 1 - i observations after it shifted (represented by q;-I-), and that a signal was given on observation v.
RUN-LENGTH
DISTRIBUTIONS
OF CONTROL
CHARTS
239
Substituting
(A5 ) and (A6) into (A4) and simplifying P(N = u) = poq6-r + sp,q;- 1 -R I-R
[ 1
where R = qor/ql. This expression can now be compared with the probability of obtaining a signal when no shift has occurred to obtain the likelihood ratio in (8). References
ALWAN, L. C. AND H. V. ROBERTS ( 1988), Time-Series Modeling for Statistical Process Control, JournalofBusiness Economic and Statistics,6, 1, 87-95. Decisions, Journalof theRoyaIStatistical BARNARD, G. A. ( 1954), Sampling Inspections and Statistical Society,Series 16, 2, 151-165. B, Analysis,Forecasting Control, 2nd ed., and Box, G. E. P. AND G. M. JENKINS (1976), Time Series Holden Day, San Francisco. CHAMP, C. W. AND W. H. WOODALL ( 1987), Exact Results for Shewhart Control Charts With Sup 29, plementary Runs Rules, Technometrics, 4, 393-399. CROWDER, S. V. ( 1987), A Simple Method for Studying Run Length Distributions of Exponentially Weighted Moving Average Charts, Technometrics, 4, 401-407. 29, ( 1989), Design of Exponentially Weighted Moving Average Schemes, Journalof Quality Technology, 3, 155-162. 21, DUNCAN, A. J. ( 197 1 ), The Economic Design of f-charts When There is a Multiplicity of Assignable Causes, Journalof theAmericanStatistical Association, 333, 107- 12 1. 66, EVANS, J. R. AND W. M. LINDSAY ( 1988), A Framework for Expert System Development in Statistical Quality Control, Computers IndustrialEngineering, 3, 335-343. and 14, HOSNI, Y. A. AND A. K. EL~HENNAWY ( 1988), Knowledge-Based Quality Control System, Computers andIndustrialEngineering, 1-4, 33 l-337. 15, LUCAS, J. M. AND M. S. SACCUCCI( 1990), Exponentially Weighted Moving Average Control Schemes: 32, Properties and Enhancements, Technometrics, 1, 1-12. 16, MAYER, R. R. ( 1983), Selecting Control Chart Limits, Quality Progress, 9, 24-26. MONTGOMERY, D. C. ( 1980), The Economic Design of Control Charts: A Review and Literature 12, Survey, Journalof Quahty Technology, 2, 75-87. ROBINSON, P. B. AND T. Y. HO ( 1978), Average Run Lengths of Geometric Moving Average Charts 20, by Numerical Methods, Technometrics, 1, 85-93. for WADSWORTH, H. M., K. S. STEPHENS, AND A. B. GODFREY (1986), Modern Methods Quality ControlandImprovement, John Wiley and Sons, New York. WALDMANN, K. H. (1986a), Bounds for the Distribution of the Run Length of Geometric Moving Average Charts, AppliedStatistics,35, 2, 15I-158. ( 198613)) Bounds for the Distribution of the Run Length of One-Sided and Two-Sided CUSUM 28, Quality Control Schemes, Technometrics, 1, 61-67. WARDELL, D. G., H. MOSKOWITZ, AND R. D. PLANTE ( 1994), Run Length Distributions of Special36, Cause Control Charts for Correlated Processes, Technometrics, 1, 3- 17. Probability,Znference Decision, ed., Holt, and 2nd WINKLER, R. L. AND W. L. HAYS ( 1975),Statistics: Rinehart and Winston, New York. WOODALL, W. H. ( 1983), The Distribution of the Run Length of One Sided CUSUM Procedures for 25, Continuous Random Variables, Technometrics, 3, 295-301.