Mathematical Reliability Theory: From the Beginning to the Present Time

Richard E. Barlow

College of Engineering University of California Berkeley, California 94720 USA barlow@newton.berkeley.edu

Abstract
It is argued that the mathematical theory of reliability as a separate discipline began in 1961 with the publication of Multi-component systems and their structures and their reliability by Birnbaum, Esary and Saunders. Prior to this time, mathematicians were just applying standard mathematical techniques such as queueing theory, statistics and probability to engineering reliability problems. We will describe how the 1965 book Mathematical Theory of Reliability came to be written. Some personal historical perspectives will follow on probabilistic concepts of aging. Finally, we will discuss more recent work on Schur functions and Bayesian implications for reliability research.

1. Coherent Systems
Reliability became a subject of great engineering interest in the 1950s due to the failure of American rockets as well as the failure of the first commercial jet aircraft; the British de Havilland comet. Life testing was part of this engineering interest. Epstein and Sobels 1953 paper studying the exponential distribution was a landmark contribution. However it was not until 1961 with the publication of Birnbaum, Esary and Saunders paper on coherent structures that reliability theory began to be treated as a separate subject. The emphasis in this paper is on theory. The Boeing 707 was under development at the time the de Havilland comets were crashing. It was partly for this reason that the Boeing Scientific Research Laboratories in Seattle began to emphasize reliability theory in their mathematics division. Z. W. Birnbaum from the University of Washington was a consultant to this group. Z. W. had a strong mathematical background. He studied under Steinhaus and Banach among others. He had a special talent for getting quickly to the nub of a problem, especially in his consulting. He was adept at formulating appropriate theoretical models to capture the essential aspects of applied problems. In a 1974 conference dedicated to Birnbaum, Sam Saunders introduced him as follows: I must remind you that Birnbaum is not a man to whom everyone looks up. In fact I estimate that 85% of all the men and 50% of all the women working in Reliability Theory today look down on him. The sad fact is, he is not a giant among the workers in his field. (Birnbaum was 5 4 tall.) A coherent system can be defined in terms of a binary function, (x) ; a system structure function which is non-decreasing in each vector argument and such that each component is relevant. (A

component is irrelevant if it doesnt matter whether or not it is working.) Such systems are called coherent. Not all systems of interest are coherent but the class is sufficiently large to be of considerable significance. Figure 1 is an example of a coherent system. The system is operational if there is a working path from source to terminal. Arcs are assumed to fail independently but may have different failure probabilities.

Figure 1: Directed Graph

Perhaps the most beautiful result concerning coherent systems is the IFRA closure theorem. (IFRA stands for increasing failure rate on the average.) The theorem says that the class of IFRA life distributions is the smallest class 1) containing the exponentials, 2) that is closed under the formation of coherent systems and 3) is closed under limits in distribution. Although life times are assumed independent, life distributions may differ. This theorem, due to Birnbaum, Esary and Marshall, was published in 1966. Sheldon Ross in 1972 provided a simplified proof of the theorem. Since coherent structures may be very complex, there has been a great deal of interest in their efficient probability calculation. A. Satyanarayana in 1978 introduced the idea of domination into the reliability literature. Starting with the minimal path sets, the signed domination is the number of odd formations of a coherent system minus the number of even formations. A formation is a set of minimal path sets whose union is the set of all components (or arcs in the case of networks). The domination is the absolute value of the signed domination. Using these ideas, Satyanarayana and Chang (1983) proved that the Factoring Algorithm is the most efficient algorithm for undirected networks based on series-parallel probability reductions and pivoting when 1) arcs fail independently and 2) nodes are deemed perfect. Figure 2 illustrates the factoring algorithm by a binary computational tree. The two leaves at the bottom of the tree are seriesparallel reducible. The domination coincides with the number of leaves at the bottom of the binary computational tree. The domination is 2 in this case. Arne Huseby in 1984 provided a more abstract unified theory of domination and signed domination with application to exact reliability calculation.

1 s 2 1 3

4 t 5 1

4 s 2 3 5 t s 2 3

4 t 5

Figure 2: The Binary Computational Tree 2. Mathematical Theory of Reliability published in 1965
The research monograph, Mathematical Theory of Reliability, was in part a product of the Cold War. I first met Frank Proschan in 1958 at the Electronic Counter measures Laboratory (EDL), a Cold War quick reaction facility in Mountain View, California. We were both Ph.D. students in Statistics at Stanford. Frank finished first in 1959. Frank had a poker face when telling the most outrageous stories. He was very witty even when delivering technical papers. I remember once when I was visiting Frank in Tallahassee sitting around a swimming pool in his apartment complex. A woman who lived in the complex began telling how wonderful it was since her recent divorce. After about 5 minutes of this, Frank said: Your divorce must have been made in heaven. Often, however, his humorous comments were about himself rather than other people. In the summer of 1960 Rudy Drenick, representing SIAM, came to EDL and proposed that we write a research monograph in reliability theory. He had first approached Z. W. Birnbaum who had declined but suggested Franks name. We thought that this was a great opportunity and quickly signed a book contract. Originally Larry Hunter was to join us but had to drop out. We submitted a monograph proposal to John Wiley & Sons, the publisher of the SIAM series. A reviewer of the proposal suggested that we include material about life distributions. This was missing in the proposal. Much of the literature on the failure rate function or the hazard rate was in the insurance literature. This inspired our interest in working on IFR (increasing failure rate) distributions. In 1960-61 I began working with Albert Marshall at Princeton while Frank went to the Boeing Scientific Research Laboratories. Together we began the study of IFR distributions. Franks Ph.D. thesis was in part concerned with optimal redundancy. At this time, spare parts allocation was of great interest to the military. The problem was to achieve an optimal allocation of redundancy; that is, maximize system reliability for the cost, weight, or volume, etc., allowed. Frank developed an algorithm for determining an undominated family of optimal solutions; that is each member of the family has the property that any allocation achieving higher reliability must be costlier, heavier or bulkier. His algorithm depended on the log concavity of the survival distribution corresponding to the convolution of n iid random variables. (The survival distribution of a single item is log concave iif it is IFR.) For this reason we spent a great deal of time trying to

prove the IFR convolution theorem. Using total positivity, Frank provided the neatest though not the first proof. The convolution result and other IFR results were published in a 1963 IMS paper. In a 1975 textbook with Frank we conjectured that IFRA distributions were also closed under convolution. In a 1976 paper, Block and Savits provided an elegant proof of this result. In the summer of 1961 I joined the GTE Research Laboratories in Menlo Park (now defunct). I discovered the crossing property for IFR survival distributions with respect to exponential survival distributions with the same mean. Continuing to work with Al Marshall, we published several papers on inequalities and bounds for IFR distributions starting in 1964. During the Cold War, the Russians were also very interested in reliability. In 1965, Gnedenko, Belyayev and Solovyev published Mathematical Methods of Reliability Theory in Russian. An English version appeared in 1969. They applied queueing theory ideas to reliability problems especially involving maintenance and replacement problems. In 1967 Frank visited Gnedenko in Moscow.

3. Reliability and Fault Tree Analysis

A conference on Reliability and Fault Tree Analysis occurred at Berkeley in 1974. In retrospect, perhaps one of the most influential papers in this volume leading to reliability research in the 1990s was the elegant paper by Proschan Applications of Majorization and Schur Functions in Reliability. In this expository paper, Frank introduces the ideas of majorization and Schur functions from inequality theory and uses them to obtain bounds, comparisons, and inequalities in reliability and life testing. Much later, Max Mendel and myself used these ideas to define multivariate IFR distributions for exchangeable random quantities. Subsequently, Fabio Spizzichino in 2001 published an excellent monograph further developing these ideas. Engineers who studied engineering systems in great detail, with little or no contribution by mathematicians, developed fault tree analysis. A possible explanation for this comes from the fact that the construction of the fault tree, a basic step in fault tree analysis, requires an intimate knowledge of the manner in which a system is designed and operated. A key feature of this approach is that it is failure oriented rather than success oriented. Often for this reason it can be very useful in improving the reliability of systems.

4. Concepts of Aging and Schur Concavity

Although a great deal of research dealt with IFR distributions, there were theoretical problems.

4.1 First difficulty with the IFR idea.

Consider the case of two components in parallel as in Figure 3.
Item 1

Item 2

Figure 3: Two items in parallel

Jim Esary at the Boeing Scientific Research Laboratories noticed that if each had an exponential life distribution, i.e. constant failure rate, but the failure rates were different, then the life distribution of the parallel system was not IFR. However the failure rate is increasing on the average; i.e. is IFRA. This observation motivated research leading to the IFRA closure theorem.

4.2 Second difficulty with IFR distributions

Suppose you index a distribution, say the exponential using the mean life, , e.g.
F( x | ) = 1 - e -x/ , for x 0.

Usually, in applications, you do not know the mean life. Suppose, however, that you have prior knowledge that can be encapsulated in a probability distribution. In this case your unconditional distribution for lifetime is
x / p() d . F( x | ) p ( )d = 1 e 0 0

The unconditional or predictive life distribution has a decreasing failure rate function. Likewise IFR distributions are not closed under mixing. This again suggests that something is wrong with the univariate IFR characterization of aging. 4.3 Aging Represented by Schur Concave Joint Survival Distributions New ideas are what make good research both possible and interesting. The new idea relative to answering the second difficulty was to consider not just a single lifetime but instead a collection of exchangeable lifetimes. Fabio Spizzichino (1992) said it best. In words, his mathematical result says that among any two items from n similar items (i.e. exchangeable items) that have survived a life test, the younger is the better, if and only if the joint survival function is Schur concave. Mathematically, he proved that F (x1 , x2 ,..., x n ) is Schur concave, if and only if, for any t > 0 and xi < x j ,
P X i > xi + t | X1 > x1 ,..., X n > xn P X j > x j + t | X1 > x1, ..., Xn > xn .

This is an intuitive restatement of the IFR idea only now for conditional joint survival distributions. If the conditional joint survival distribution F x1 , x2 , ...,xn | is Schur concave, then it is still Schur concave unconditionally since Schur concavity is defined in terms of an inequality on F . Using Schur concavity of the joint survival probability as our new definition of aging, the second difficulty with the univariate IFR definition is overcome. In the case of the exponential distribution, Schur constancy is preserved under mixing. The ideas in Spizzichinos 1992 paper were extended to a more general analysis of Bayesian multivariate aging using different stochastic comparisons of residual lifetimes for units having different ages. Bassan and Spizzichino (2001) define a notion of multivariate IFRA but not in terms of Schur concavity. The IFRA notion of aging seems to be very different from the IFR notion of aging or of its generalization in terms of Schur concavity.

5. Physical Foundations for Probability Distributions

Since reliability is concerned with uncertainty questions about engineering systems, it would make sense to derive appropriate probability distributions based on the engineering physics of a given problem. This would seem to make more sense than picking a mathematically convenient probability distribution more or less at random. It seems, however, that to do this we need to adopt a Bayesian approach to probability. That is, judgments such as indifference relative to certain basic random quantities must be made. For example, suppose we are interested in the stress, which causes yielding in a given material. Suppose furthermore that we believe Hookes Law is valid in this case. Starting with an indifference assumption regarding vectors of distortion energies with the same mean, we are led to the Weibull distribution for stress at yielding with shape parameter equal to 2, Barlow and Mendel (1994). A controversial figure, Max Mendel, appeared on the reliability scene in 1989. His MIT Ph.D. thesis in Mechanical Engineering concerned probability derivations based on engineering principles. Beginning in 1994, Mendel began exploring the use of differential geometry for the purpose of deriving probability distributions. This eventually led to the conclusion that lifetime spaces are not physical Euclidean spaces. The use of the hazard gradient, for example, to model multivariate hazard rates is therefore incorrect since it relies on the Euclidean metric. Shortle and Mendel (2001) argue as follows: Let L (1) L has a preferred orientation for its axes. (2) L has no natural notion of distance. Observe that Euclidean space is invariant under rotations, since rotations preserve the value of the inner product; i.e. there is no preferred orientation for the axes. We can characterize the physical structure of a space by the transformations that leave the space invariant. For Euclidean space, N these are translations and rotations. For L these are changes of units of the individual items. This is because physical properties about lifetimes should not depend on the units used to measure lifetimes. In the language of differential geometry, the correct representation for the space of lifetimes is a collection of fiber bundles. The new ideas in Shortle and Mendel (2001) suggest a surprising and exciting new line of research in mathematical reliability theory.
N N N

be the space of possible lifetimes for N

N

items. Euclidean space is not a good representation for L for two reasons:

References
Barlow, R. E. and A. W. Marshall (1964). Bounds for distributions with monotone hazard rate, I and II. Ann. Math. Statist. 35, 1234-1274. Barlow, R. E. and F. Proschan (1975). Statistical theory of reliability and life testing. Holt, Rinehart and Winston. Barlow, R. E. and M. B. Mendel (1994). The operational Bayesian approach. In P. R. Freeman and A. F. M. Smith (Eds.), Aspects of Uncertainty, pp. 19-28. Chichester: Wiley.

Barlow, R. E., A. W. Marshall and F. Proschan (1963). Properties of probability distributions with monotone hazard rate. Ann. Math. Stats. 34, 375-389. Barlow, R. E., and F. Proschan (1965). Mathematical Theory of Reliability. J. Wiley & Sons. Reprinted (1996) SIAM, Philadelphia, PA. Bassan, B. and F. Spizzichino (2001). Dependence and multivariate aging: the role of level sets of the survival function. In Hayakawa, Y., Irony, T. and Xie, M. (Eds.), System and Bayesian reliability, pp. 229-242. World Scientific. Birnbaum, Z. W., J. D. Esary and A. W. Marshall (1966). Stochastic characterization of wearout for components and systems. Ann. Math. Stats. 37, 816-825. Birnbaum, Z. W., J. D. Esary and S. C. Saunders (1961), Multi-component systems and structures and their reliability. Technometrics 3 (1), 55-77. Block, H. and T. H. Savits (1976). The IFRA closure problem. The Annals of Probability 4, 1030-32. Epstein, B. and M. Sobel (1953), Life testing. J. Amer. Statist. Assoc. 48, 486-502. Gnedenko, B. V., Yu. Belyayev and A. D. Solovyev (1969). Mathematical Methods of Reliability Theory. New York: Academic Press. Huseby, A. B. (1989). Domination Theory and the Crapo beta-invariant. Networks 19, 135149. Proschan, F. (1975). Applications of majorization and Schur functions in reliability and life testing. In R. E. Barlow, J. B. Fussell and N. D. Singpurwalla (Eds.), Reliability and Fault Tree Analysis, pp. 237-258. Philadelphia: Society for Industrial and Applied Mathematics. Ross, S. M. (1972). Introduction to probability models with optimization applications. Academic Press. Satyanarayana, A., and A. Prabhakar (1978). New Topological Formula and Rapid Algorithm for Reliability Analysis of Complex Networks. IEEE Trans. Reliability R-27, 82-100. Satyanarayana, A., and M. K. Chang (1983). Network reliability and the factoring theorem. Networks 13, 107-120. Shortle, J. F. and M. B. Mendel (2001). Physical foundations for lifetime distributions. In Hayakawa, Y., Irony, T. and Xie, M. (Eds.), System and Bayesian reliability, pp. 257-266. World Scientific. Spizzichino, F. (1992). Reliability decision problems under conditions of ageing. In J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith (Eds.), Bayesian Statistics, 4, Oxford University Press. Spizzichino, F. (2001). Subjective probability models for lifetimes. Chapman & Hall/CRC.