lEEE TRANSACXIONS ON RELIABILITY, VOL. 44. NO. 3.
1995 SEPTEMBER
Maximum Likelihood Voting for Fault-Tolerant Software
with Finite Output-Space
Yiu-Wing Leung
The Hong Kong Polytechnic University
Key Words - Software reliability, Software fault-tolerance,
Multiversion software, Voting.
Summary & Conclusions - When the output space of a
multiversion software is finite, several software versions can give
identical but incorrect outputs. This paper proposes a maximum
Likelihood voting (MLV) strategy for multiversion software with
finite output-space under the assumption of failure independence.
To estimate the correct result, MLV uses the reliability of each
software version and determines the most likely correct result. In
addition, two enhancements are made to MLV. 1) Impose a re-
quirement m* that the most likely correct output must have prob-
ability at least a*.2) The voter can estimate when it has received
one or more outputs from the software versions. If the probability
that the estimated result is correct is at least a*,then it immediately
gives this estimated output. Since the voter need not wait for all
the outputs before it can estimate, the required mean execution
time can be reduced. The numerical results show that these MLV
strategies have better performance than consensusvoting and ma-
jority voting, especially when the variation of software version
reliability is large. Enhancement #2 can appreciably reduce the
mean execution time, especially when the software versions have
larger execution time standard-deviation.
1. INTRODUCTION
As the processing power of computers increases, and their
size & cost decrease, the size & complexity of application soft-
ware increase. For a large complex software, it is very difficult
to remove all the faults [l]. If the consequences of software
failures are costly or disastrous, software fault tolerance can
be an effective approach to increase the software reliability.
A well-known software fault-tolerant method is multiver-
sion programming [ 1 - 101, which is similar to N-fold modular
redundancy for hardware fault tolerance. This method requires
separate s-independent preparation of multiple versions of a soft-
ware. Figure 1 shows a multiversion software with Nversions.
The N versions are executed and the outputs of these N ver-
sions are sent to a voter. The voter executes a strategy to estimate
the correct output.
Acronyms & Abbreviations
cv consensus voting
MLV maximum-likelihood voting
MLV-Ek MLV with enhancement Ek (k= 1, 2)
MV majority voting.
0018-9529/95/$4.00 01995 IEEE
419
The simplest voting strategy is MV [111. Usually N is odd,
so if at least liub( % N ) software versions give the same output,
MV estimates this output as the correct result.
i
cannot reach
consensus
Figure 1. Multiversion Software
CV [2] is designed for multiversion software with small out-
put space in which case several software versions can give iden-
tical but incorrect outputs [2]. For example, if the softwareis im-
plemented to solve a yes-no decision problem, the output space
is binary. Then, all the incorrect outputs are identical. As a result,
the incorrect outputs can agree. CV selects the output that the most
software versions give. For example, suppose there are 6 soft-
ware versions and 3 possible outputs. If 2 versions give output
#1,3 versions give output #2, and 1 version gives output #3, then
CV estimates that the correct result is output #2.
Neither MV nor CV considers the software reliabilities of
the software versions when they estimate. In other words, when
these voting strategies estimate, they do not consider how
reliable each software version is and how faithfid its output is.
In practice, the reliability of software versions can differ.
If multiversion software is developed, the project manager
should impose a minimum software-versionreliability require-
ment; software reliability can be measured using statistical
methods [12 - 161. Since the software versions are developed
s-independently, different versions can have different software
reliabilities while they satisfy the software reliability require-
ment. For example, if the software reliability requirement is
0.90, then version 1 might have reliability 0.99, version 2 might
have reliability 0.95, etc. The voter can use this reliability in-
formation to make a more accurate estimate.
Example
There are 11 software versions.
The output space is binary (0, 1).
5 A-versions have reliability 0.99; 6 B-versions have soft-
ware reliability 0.95.
420 IEEE TRANSACTIONS ON RELIABILITY. VOL. 44, NO. 3. 1995 SEPTEMBER

For a given input, the A-version outputs = 0, and the B- 3b. A failed software version gives 1 of the R- 1 incor-
version outputs = 1; this event is denoted E. According to rect outputs with equal probability. 4
MV or CV, the correct result is estimated to be 1 because
the 6 B-versions (of 11 total versions) give 1. However, given
the 11 outputs and assuming s-independence of version 3. MLV
failures,
The main idea of MLV is to choose, based on how reliable
Pr {correct output is 0 I [} = 0.995-(1-0.95)6/Pr{[} each software version is and how faithful its output is, the out-
put which is most likely correct.
= -
1.49 10-'/Pr { [} ; From assumption #3, a software version gives a particular
incorrect output with probability ( 1 - ri)/ ( R - 1) .
Pr{correct output is 1 I [ } = (1 -0.99)5.0.956/Pr{[} R
Pr{s} = Pr{s n (correct result is output j ) }
= 7.35.10-"/Pr{[). j=1

R N
Since Pr{correct output is 0 IE} > Pr {correct output is 1 I E},
it is more likely that-the correct output is 0, although only 5
of the 11 versions give 0.
This paper proposes:
Aj(i) =
a MLV strategy for multiversion software with finite output (l-ri)/(R-l), otherwise.
space. Based on the outputs and the reliabilities of the soft-
ware versions, this strategy selects the output that is most like- xi is found as follows:
ly correct.
two enhancements to MLV. xj = Pr{correct result is output j l s }
We analyze & compare the performance of these MLV strategies
= Pr{s f l (only the versions giving output j are
with those of MV & CV.
correct)}/Pr {s)
2. ASSUMPTIONS & NOTATION R N
(3)
Notation k=l i=l

N number of software versions MLV Algorithm

R output-space cardinality Inputs: r, s
ri, si [reliability, output] of software version i
r (rl, r2, ..., rN)
Output: Estimated correct output
S (SI, s2, SN) Steps:
Pr{correct result is output j Is}
xj,
a acceptance requirement probability 1. Compute xi for all j = 1,2,. .. ,R;
* set: versions that have delivered output to voter
set: versions giving output j
2. Arrange the xj in decreasing sequence such that there are
x maxima (x 1 1):
3 total required execution time, a r.v.
Fm, Rzv [unreliability, reliability] under ZV Xj, = Xi, = ... = Xj, > x,, for all j ' # j,, k E [ l , x ] .
ZV = CV, MLV, MV, MLV-El, MLV-E2.
3. Choose output jk,k E [ l j ] , with probability l/x.
Other, standard notation is given in "Information for Readers
& Authors" at the rear of each issue. / * If x = 1, then the unique maximum is chosen with prob-
ability 1. */
Assumptions (same as in [2]) EndAlgorithm 4

1. All software versions are functionally equivalent and Since the denominators of all xj are the same, see (3), the
mutually s-independent. voter need only find & compare the numerators of the xj, and
2. R is finite. thus reduce the computation time.
3a. For 1 input: 1 of the R possible outputs is correct; R- 1 MLV does not pose any restriction on the ri, i E [ 1,N.
of them are incorrect. Therefore, MLV still applies when ri = a constant for all i, and -
 ~

LEUNG: MAXIMUM LIKELIHOOD VOTING FOR FAULT-TOLERANT SOFTWARE WITH FINITE OUTPUT-SPACE 421

when R 1 2, MLV & CV are equivalent. MLV-E1 Algorithm

when R = 2, MLV, CV, MV are equivalent.
Inputs: r, s, CY*

Output: Estimated correct output or a warning message

4. MLV-E1
Steps:
(Enhancement #1) 1. Compute xj for all j = 1,2,...,R;
2. Arrange the xj in decreasing sequence such that there are
In MLV, the estimated output is the most likely correct x maxima (x2 0) satisfying both conditions:
one. Even though it is most likely correct, the probability that
it is incorrect might still be unacceptably large for some critical
applications.
for allj’ # j,, k E [l,x].
Example
N=5,R=5, 3. IF x > 0 THEN choose output jk,k E [ l , x ] , with prob-
ability l l x ;
/ * If x = 1, then the unique maximum is chosen with prob-
r = (0.93, 0.92, 0.95, 0.94, 0.93), ability 1 * /
ELSE output a warning message;
s = ( 5 , 2, 4, 3, 1). ENDIF
EndAlgorithm 4
Since, 1) each software version gives a distinct output, and 2)
software version 3 is the most reliable, MLV estimates 4 to be
5. MLV-E2
the correct output. However, this estimate could be less reliable
because only one version gives this output. In this case, it might (Enhancement #2)
be better to give a warning message so that the software users
can determine the appropriate course of actions (eg, Output the In many practical situations, the voter does not receive all
message: LOW PROBABILITY AGREEMENT). 4 outputs from all software versions simultaneously.
The main idea of MLV-E1 is to impose a requirement a*
such that the voter returns the most likely correct output iff the Example 1
probability of this output is at least CY*.Figure 2 illustrates the Some software versions might require different amounts
voting strategy. of execution time [171 because they are developed by different
teams using different development methods. Therefore, the voter
first receives the output of the fastest version, then it receives
the output given by the next fastest version, etc. 4

Example 2

Maximum likelihood voting

The software versions are executed on a set of geograph-
ically distributed processors, the processors are connected by
an estimated a communication network and the software-versionoutputs are
result sent to a central voter through the communication network. The
time to transmit an output from a processor to the central voter
depends on the physical distance and the current traffic load
between the two communicating points [ 181, Therefore, the
voter might not receive all software-version outputs
simultaneously. 4
The main idea of MLV-E2 is that the voter need not wait
for all software-version outputs before it can estimate. The voter
begins to estimate as soon as it receives the first output. It con-
output warning
result message tinues estimating until Pr {estimated output is correct} 2 a*,
at which point it stops. This strategy can reduce the mean ex-
ecution time required by the multiversion software while satis-
fying the requirement a*. The xi are computed by (4).

End
(4)
Figure 2. Maximum Likelihood Voting with Enhancement 1
422 IEEE TRANSACTIONS ON RELIABILITY. VOL. 44. NO. 3. 1995 SEPTEMBER

6. PERFORMANCE ANALYSIS

This section analyzes the performance of MV, CV, MLV,

MLV-El, and MLV-E2.
qE9\Stj implies: q is an element of Q, but not of Qj

< start )
Notation

ti output of software version i

t ( t l , t2, ..., t N )
received one or more new outputs
lrom the soltware versions 4 number of software versions giving output j , 0 If,
I N
result f c f i t f2, ...? fR)
Ti elapsed time before the voter receives output of
software-version i, a r.v.
ai ( t ) Cdf {Ti}.
versions?

Assumptions

output Ihe estlmaled output waming

4. The voter is free of faults.
5. The execution time for voting is negligible,
6. For a given input, output 1 is the correct result.
7. The Ti are mutually s-independent for all i. 4

CEnd) Assumption #4 is reasonable because the voter performs

relatively simple operations and can be tested exhaustively.
Figure 3. Maximum Likelihood Voting with Enhancement 2
Assumption #5 is reasonable because voting requires only a few
operations while a software that requires a method to mask out
its remaining faults is presumably complex and requires a much
MLV-E2 Algorithm longer computation time. Assumption #6 simplifies the nota-
tion without loss of generality.
Inputs: r, a* Since each software version gives one of the R possible
Output: Estimated correct output or a waming message outputs, the total number of output combinations is RN. If R
& N are not large, then it is feasible to compute the performance
Steps: based on the analytic expressions derived in sections 6.1 - 6.5.
la. @ := 0 Otherwise, the number of output combinations is so large that
lb. x := 0 we have to resort to computer simulation.

2. FOR each incoming set of y software-version outputs ( y 2 1 Pr ( t } = Pr {t n (only the versions giving output 1 are correct)}
is a r.v.) DO
N
3a. Q, := 9 U { j k : k ~ [ l , y ] } = 6(i);
3b. FOR k= 1 TO y DO Q , : = Q, U {jk}; END-FOR i=l
3c.x:=x y +
4. Compute all the xi according to (4); arrange them in 6(i) =
decreasing sequence such that there are zmaxima (z L 0) satis- ( 1 -ri)/(Z?- l ) , otherwise.
fying both conditions:
The f is found from t:

A = Ofor 1 I i I R;
5. IF z > 0 THEN choose output j k , k E [l,z], with prob-
ability ltz; F O R i = l T O N D O f , : = f ,+ l ; END-FOR.
/ * If z= 1, then the unique maximum is chosen with prob-
ability 1 * / Execution Time
GoTo EndAlgorithm; E N D l F
6. IF x = R THEN Output a warning message; GoTo (For all voting except MLV-E2.)
EndAlgorithm; E N D J F The voter waits for all the outputs before it estimates.
7. END-FOR
EndAlgorithm 4 33 = max{?i: i E [l,n]); (6)
 ~

LEUNG: MAXIMUM LIKELIHOOD VOTING FOR FAULT-TOLERANT SOFTWARE WITH FINITE OUTPUT-SPACE 423

For MLV-E2, the voter need not wait for all outputs before
it estimates. Section 7 uses computer simulation to study the
mean testing time required. and (fi > fk for k ,... ,ji-l)}
# lJ1 (16)

6.1 MV CV makes an incorrect estimate when the number of soft-

ware versions giving output 1:
The MV is designed for odd number of software versions.
The voter can give a correct result if at least liub(%N) soft- is not the largest, or
ware versions give output 1. is the largest and equals that giving output jk, k E [ l ,i- 11,
but the voter does not choose output 1.
N
tEn F~~ = Pr{$} + Pr{EinC}
i=2
= {t: fl 1 liub(%N)}. (9)

6.2 CV

Notation
6.3 MLV
C event: {output 1 is chosen}
Ei fi
event: { f i = fjl = J 2 = ... = Ji-,, > fk for all k Fi= {output 1, output jl, ..., output ji- have the same largest
f 1, j 1 , ..., i~[l,Rl probability of correctness)
4 opposite of any event $.
The E;, i E [1,R] are mutually exclusive. For El, the number
of software versions giving output 1 is the largest. Therefore,
consensus voting chooses output 1 with certainty. for all k z 1, jl, ..., j i - l }
The Fk,k E [l ,RI are mutually exclusive.
Pr{E, n C} = Pr(t};
R
tEQl
RMLV = Pr{Fi f l C}
= {t: fl > fk for all k f l } i=l

R
For E2, f i = fjl; hence CV chooses output 1 with probability
'h. = (l/i)* Pr{t}, (18)
i=l f E f!

3;. = (t: (XI = xj, = ... = Xj,-,)

The MLV fails when the most likely correct result is not
and cfi > fk for k # l,jl)} (13) output 1, or the most likely correct result is: output 1, or out-
put j , or .. . or output j i - 1, but the voter does not choose out-
Pr{Ei f7 C}, i E [3,R] can be found in a similar fashion. put 1.

Rcv = Pr((E1 U E2 U ... U ER) n C}

6.4 MLV-EI
Since the Eiare mutually exclusive,
This strategy fails when:
R
Rcv = Pr(E; n C } output 1 is not the most likely correct one while the most likely
i=l correct one satisfies the requirement a*,or
424 IEEE TRANSACTIONS ON RELIABILITY. VOL. 44, NO. 3. 199.5 SEPTEMBER

output 1, output j,, .. ., and output ji- are the most likely
system reliability
correct results and all of them satisfy the requirement a*,
but the voter does not choose output 1. 0.9 %
i-1
FMLV-E~ = pr{t> + 7 ~r{t>, (2W
t E i-T i=2 tE c
{r = {t: x1 < max{xj} and max{xj} 2 a*},
I I
(21b)

{T 3 {t: (x, = xj, = ... = xj,-l)

6.5 MLV-E2
4 6 8 10 12 14 16 18 20 22
1
This strategy fails when the voter has received one or more R
outputs, and:
[N = 5; r = (0.3,0.4, 0.5,0.6,0.7}]
outputj U # 1) is found to be the most likely correct result
Figure 4. System-Reliability vs R
and it satisfies the accuracy requirement a*,or
output 1, output j,, ..., and output ji-lare found to be the average system reliability
most likely correct results, all of them satisfy the require- 1.ooo
ment a*,but the voter does not choose output 1.
It is difficult to express the failure probability in closed form; 0.995
it is measured by computer simulation in section 7.
0.990

7. NUMERICAL RESULTS AND DISCUSSION

0.985

This section uses numerical examples to compare the per-

formance of MV, CV, MLV, MLV-El, and MLV-E2. 0.980

N = 5

r = (0.3,0.4, 0.5, 0.6, 0.7).

0'975
0.970 I
t13
a

5 7 9 11
Figure 4 shows system reliability vs R. N
MLV always results in larger system reliability than CV
or MV. When R increases, the probability of getting identical [R = 3; ri is uniformly distributed between 0.8 & 1.01
and incorrect outputs becomes smaller and hence the reliabili-
ty of both MLV & CV increases with R. Figure 5. System-Reliability vs N
To study the effect of N on system reliability, let
We now illustrate the effects of the variation of software
R = 3, version reliability on the system reliability. Consider the three
distributions of software version reliability:
the reliability of each software version be uniformly distributed uniformly distributed -
between 0.8 and 1.0.
between 0.5 and 0.7,
We generate 5000 random cases, calculate the system
reliability for each, and then obtain the average system between 0.3 and 0.9,
reliability. Figure 5 shows the average system reliability between 0.2 and 1.0.
vs N . MLV provides the largest system reliability. As
N increases, the multiversion software becomes more reliable, These distributions have the same mean, but the first distribu-
and the reliability difference among the three voting strategies tion has the smallest standard deviation while the third distribu-
become smaller. tion has the largest. Figures 6 - 8 show the average system
 ~

LEUNG: MAXIMUM LIKELIHOOD VOTING FOR FAULT-TOLERANT SOFTWARE WITH FINITE OUTPUT-SPACE 425

reliability vs N for the three distributions. If the variation of
software version reliability is larger, MLV has better perfor-
mance than either CV or MV.
average system reliability
p= 1.0, and a* = 0.99. The relative reduction in mean execu-
tion time is larger:
as N increases, because the other voting strategies require a
longer time to wait for all the N outputs, or
as the reliability of the software versions increases because
the MLV-E2 needs to wait only for fewer outputs to satisfy

l'O
0.9
I the same requirement a*,or
as the (T increases.

average system reliability

1.0 I
0.8

I
0.9
//

0.7 :
0.8
0.6 1 1
t-/
I

3 5 7 9 11
Y
N 0.7

[R = 3; ri is uniformly distributed between 0.5 & 0.71

Figure 6. System-Reliability vs N
3 5 7 9 1 1
1 I

N
average system reliability [R = 3; ri is uniformly distributed between 0.2 & 1.01
1 .o
Figure 8. System-Reliability vs N

reduction in mean execution time

0.9 70

60

0.8 50

40

0.7 30

20
0.6
3 5 7 9 11 10 - is uniformlydistributedover (0.9.1 .O)
. - - .rt ft IS unilormly distributed over (0.7. 1 0)
N
0 , " " " ' " " " ' * ' J .

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2(
[R= = 3; r, is uniformly distributed between 0.3 & 0.91
N
Figure 7. System-Reliability vs N
[R= 5; /A= 1 .o, CY* = 0.001
Figure 9. Relative Reduction in Mean Execution Time vs N
Compared with the other voting strategies, MLV-E2 re-
quires smaller mean execution time because it need not wait
for all the outputs. To illustrate, we let the execution time re- Figure 10 shows the relative reduction in mean execution
quired by each software version be s-normally distributed with time for three values of a* with R = 5; p = 1.0, B = 0.1;
mean p and standard-deviation B . Figure 9 shows the relative ri is uniformly distributed between 0.9 & 1.0. When a* is
reduction in mean execution time required vs N for R=5, higher, MLV-E2 must wait for more outputs in order to have
426 IEEE TRANSACTIONS ON RELIABILITY. VOL. 44, NO. 3, 1995 SEPTEMBER

a more accurate estimation, and hence the relative reduction ability is smaller than, but close to, 1-a*. Although FMLV-EI
in mean execution time is smaller.

reduction in mean execution time I

30

25 10-2
-

20
-

15
10‘~
-
10

-
5 -a*= 0.90
a*=0.99
10-61 ’
0 2 4
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 6 8 10 .I 2
N
N
[R=5; p = 1.0, u = O . I ; ri is uniformly distributed between 0.7 &
[R = 5; p = 1.O,0 = 0.1; r;.is uniformly distributed between 0.9 & 1.O]
1.O]
Figure 12. Failure Probability vs N
Figure 10. Relative Reduction in Mean Execution Time vs N

To illustrate the failure probability vs N , let R = 5 ; p = 1.O, REFERENCES

a=0.1; and ri is uniformly distributed between 0.7 & 1.0.
Figure 11 shows that FMLV-EI < FMLv < FCV. If a* is in- J.P.J. Kelly, T.I. Mcvittie, W.I. Yamamoto, “Implementing design diver-
creased, the failure probability becomes smaller. sity to achieve fault tolerance”, ZEEESoftware, vol8, 1991 Jul, pp 61-71.
D.F. McAllister, C.E. Sun, M.A. Vouk, “Reliability ofvoting in fault-
tolerant software systems for small output-spaces” , IEEE Truns. Reliabili-
ty, VOI39, 1990 Dec, pp 524-534.
Failure probability S . S . Brilliant, J.C. Knight, N.G. Leveson, “Analysis of faults in an N-
10 [31
version software experiment”, ZEEE Trans. Sofhvare Engineering, vol
16, 1990 Feb, pp 238-247.
10” - B. Littlewood, D.R. Miller, “Conceptual modeling of coincident failures
in multiversion software”, IEEE Trans. Sofhvare Engineering, vol 15,
- 1989 Dec, pp 1596-1614.
[51 J.C. Knight, N.G. Leveson, “An experimental evaluation of the assump
tion of independence in multiversion programming”, IEEE Trans. Sofi-
- ware Engineering, vol SE-12, 1986 Jan, pp 96-109.
M.A. Vouk, D.F. McAllister, K.C. Tai, “Identification of correlated
- failures of fault-tolerant software systems”, Proc. COMPSAC 85, 1985,
pp 437-444.
[71 P.G. Bishop, D.G. Esp, M. Barnes, er al, “PODS - A project on diverse
- software”, ZEEE Trans. Sofhvare Engineering, vol SE-12, 1986 Sep,
pp 929-940.
J. Kelly, D. Eckhardt, A. Caglayan, er al. “A large scale second genera-
tion experiment in multiversion software: Description and early results”,
2 4 6 8 10 12 Proc. FTCS 18, 1988 Jun, pp 9-14.
N 191 K.G. Shin, H. Lee, “Evaluation of error recovery blocks used for
cooperating processes”, ZEEE Trans. Sofhvare Engineering, vol SE-10,
[ R = 5 ;p = 1.O, a=0.1; r;.is uniformly distributed between 0.7 & 1985, pp 692-700.
1.O] J.D. Musa, W.W. Everett, “Soilware-reliabfity engineering: Technology
Figure 11. Failure Probability vs N for the 1990s”, IEEE Software, vol 7, 1990 Nov, pp 36-43.
R.J. Abbott, “Resourceful systems for fault tolerance, reliability, and
safety”, ACM Compuring Surveys, vol 22, 1990 Mar, pp 35-68.
M. Xie, Sofrware Reliability Modelling, 1991; World Scientific.
Figure 12 compares FMLV-EI & FMLV-EZ; the latter is
J.D. Musa, A. Iannino, K. Okumoto, Sojhvare Reliability: Measurement,
relatively independent of N because it immediately estimates Prediction, and Applicarion, 1987; McGraw-Hill.
the output when the probability that this estimated output is cor- J.D. Musa, “Tools for measuring software reliability”, IEEE Spectrum,
rect is just equal to or larger than a*.Therefore, its failure prob- 1989 Feb, pp 39-42.
LEUNG: MAXIMUM LIKELIHOOD VOTING FOR FAULT-TOLERANT SOFTWARE WITH FINITE OUTPUT-SPACE 427

S. Brocklehurst, B.Littlewood, “New ways to get accurate reliability Internet (e-mail): y .leungQieee .org
measures”, IEEE Sofnoare, vol 9, 1992 Jul, pp 34-42. Yiu-Wing Leung was born in Hong Kong. He received his BSc (1989)
F.T. Sheldon, K.M. Kavi, R.C. Tausworthe, eral, “Reliability measure- and PhD (1992) from the Chinese University of Hong Kong. He is an Assis-
ment: From theory to practice”, IEEE Sojiware, vol9, 1992Jul, pp 13-20. tant Professor in the Dept. of Computing, The Hong Kong Polytechnic Univer-
M.A. Vouk, A.M. Paradkar, D.F.McAllister, “Modeling execution time sity. His main research interests include information networks and hardware/sof’t-
of multi-stage N-version fault-tolerant software”, Fault-Tolerant SOB- ware reliability.
ware Systems: Techniques and Applications (H. Pham, Ed), 1992, pp
55-61; IEEE Press. Manuscript received 1995 April 1.
A.S. Tanenbaum, Computer Networks (2nded), 1988; Prentice-Hall.
Special Issue on Fault-Tolerant Software, ZEEE Trans. Reliability, vol IEEE Log Number 94-13708 4TRb
42, 1993 Jun.
AUTHOR
Dr. Yiu-Wing Leung; Dept. of Computing; The Hong Kong Polytechnic Univer-
sity; HONG KONG.

.4RWM.S ARWMS ARWMS ARWMS ARWMS ARWMS AIIWMS ARWMS AKWMS AKWMS ARWMS ARWMS ARWMS

1995 Annual Reliability and Maintainability Symposium

I
The P. K. McElroy Award for Best Paper
was bestowed upon
John K. McAnelly, PhD
for his paper.
“HACCP: A Total Quality System for Assuring Food Safety & Quality”
given at the 1994 Symposrum in Anaheim.
For more information, see the gold section of your copy of the 1995 Proceedzngs.
I

Each year the Symposzum presents the P . K. McElroy Award for the Best Paper at the previous
Symposzum. T h e P . K . McElroy Award consists of a plaque and a $1500 Honorarium. T h e two
criteria for Best Paper are:
T h e written paper is lucid, excellent, and important to the theory and/or practice of R&M
(reliabilitv & maintainability) engineering.
T h e presentation of the paper at the Symposzum is likewise lucid 8c excellent.
P. K. McElroy was an intensely practical person. T h e people & papers that receive the P. K.
McElroy Award must be able to make a difference to R8cM engineers and/or managers. It is not
enough that the paper’s content be competent & important: that competence & importance must
be obvious in both the written gaper and the presentation at the Symposzum.
Before the Symposzum, the content of each written paper is examined bv the Program Committee
for technical excellence and for clarity of‘exposition. T h e best of those papers a r e chosen and then
referred to a select group of past General Chair’n of the Symposzum. Each person in that group
attends each presentation; that group chooses the Best Paper to receive the P. K. McElroy Award.