Professional Documents
Culture Documents
Remaining Useful Life Prediction For A Nonlinear Heterogeneous Wiener Process Model With An Adaptive Drift
Remaining Useful Life Prediction For A Nonlinear Heterogeneous Wiener Process Model With An Adaptive Drift
0018-9529 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
688 IEEE TRANSACTIONS ON RELIABILITY, VOL. 64, NO. 2, JUNE 2015
, The backward smoothed expectation, i.e., Brownian motion (BM) with a linear drift, becomes one
and variance of at the th CM of the most popular SP models. On one hand, the WP model
point, where can model a degradation state as an innitely divisible process
The backward smoothed covariance [20], as well as characterize a practical degradation trajectory
of two consecutive estimated that may vary bi-directionally with time. On the other hand,
degradation drifts at the th CM the explicit distribution of the rst passage time (FPT) of
point, where the WP model is available, which is well known as the IG
Convergence threshold of the EM distribution [21]. Whitmore [22] applied a WP model subject
algorithm to measurement noises to model the declined tendency of the
Random variable of RUL gain of a transistor. Peng and Tseng [23] adopted a WP model
Failure threshold with a random drift coefcient and measurement noises to
, PDF, and CDF of predicted RUL at estimate the mean-time-to-failure (MTTF) of the lasers. Xu et
the th CM point al. [24] considered a real-time reliability prediction problem
for a dynamic system with a hidden degradation process. They
utilized particle ltering to identify the system's state, and
I. INTRODUCTION employed the WP model to predict the reliability. Doksum et
al. [12] and Whitmore et al. [13] applied a WP model with a
model and Si's model are linear. It leads to the fact that both of by incorporating the historical degradation data into the estima-
their models cannot trace the dynamics of nonlinear degradation tion of . Once the degradation drift was estimated, they used a
trajectories well. linear function with respect to time to predict the evolution of
From the above review of the related work, it can be found the degradation level.
that, though there is some research focusing on nonlinear degra- However, in reality, nonlinear degradation trajectories are
dation trajectories or degradation models adaptive to historical encountered frequently, such as the degradation trajectories of
degradation data for RUL prediction, separately, few works con- bearings [32], batteries [33], and gyros [34]. Thus, the linear
centrate on both of them simultaneously. As stated before, an prediction used in the model described by (1) and (2) cannot
adaptive drift helps to take historical degradation data into ac- trace nonlinear degradation trajectories well. Additionally,
count, and a nonlinear structure can make predicted degradation the conventional WP model is a time homogeneous process,
trajectories close to actual degradation trajectories. The combi- which means that the transition probability between two given
nation of an adaptive drift and a nonlinear structure contributes degradation states at any two CM points depends only on the
to promoting the accuracy of RUL prediction. Accordingly, we time interval. But not all degradation processes satisfy this
make three major contributions in this paper. First, we propose constraint. For example, the typical lifetime of the bearings
a novel nonlinear heterogeneous Wiener-process-based model [35] can be roughly divided into two stages: one is the normal
with an adaptive drift for RUL prediction, and utilize a state- stage, and the other is the defective stage. At the rst stage, the
space-based method to characterize our model. Second, we de- bearings degrade slowly, whereas the defects propagate quickly
velop an on-line ltering algorithm based on Bayes' theorem at the second stage. This example indicates that equally long
to update the hidden degradation drift, and then apply the ex- time intervals may give rise to different degradation increments.
pectation-maximization (EM) algorithm, and a novel Bayes'- To take nonlinearity and heterogeneity into consideration, we
theorem-based smoother to estimate the unknown parameters incorporate a nonlinear structure into the WP model, given as
once a new measurement becomes available. Last, we derive
an explicit analytical distribution for the predicted RUL incor- (3)
porating the complete distribution of the estimated degradation
drift. where is usually assumed to be 0. The specic form of
The rest of this paper is organized as follows. In Section II, can be determined by domain experts, or inferred from
we formulate the problem. In Section III, we develop an on-line the lifetime data of other identical systems. It is notable that
ltering algorithm to estimate the degradation drift, and apply there are three advantages in (3). First, in contrast with the con-
the EM algorithm to re-estimate the unknown parameters. In ventional WP model, (3) can track nonlinear degradation trajec-
Section IV, we derive an explicit analytical distribution for the tories better by adjusting to t the historical degradation data.
predicted RUL. In Section V, a simulation, and a case study are Second, the nonlinear function can characterize a time
used to demonstrate the effectiveness of our model, and com- heterogeneous process. Third, as pointed out in [13], (3) retains
pare the performance of our model with that of Wang's model, the nice mathematical property of the WP model that the explicit
and Si's model. Section VI concludes this paper. distribution of the predicted RUL can be obtained analytically
once the form of is specied.
Remark 1: There are two reasons for choosing a time-trans-
II. PROBLEM FORMULATION formation function instead of a more general one, e.g., a non-
linear drift used in [36], to characterize nonlinear degradation
To take the complete historical degradation data into consid- trajectories. First, the explicit distribution of the predicted RUL
eration, as well as maintain the nice properties of the WP model, for the former can be obtained analytically, while the distribu-
we consider Wang's model, given as [30] tion of the predicted RUL for the latter is related to solving
the Fokker-Planck-Kolmogorov equation with boundary con-
(1) straints, and the explicit analytical solution can be hardly ob-
tained. Second, members of a population of identical systems
(2)
have some common properties, so the specic form of
can be inferred from the lifetime data of other identical systems.
In Wang's model, (2) was the discretized conventional WP
Subsequently, can approach an individual degradation
model, and the authors constructed an updating procedure for
trajectory through the adjustable . Consequently, a time-trans-
the degradation drift using a random walk model as in (1),
formation function sufces to characterize nonlinear degrada-
with the initial assumption that followed a s-normal distri-
tion trajectories.
bution. Hence, evolved as a time-dependent random variable.
To better characterize practical degradation processes and ob-
Because was unobservable, it had to be estimated once a new
tain the predicted RULs more accurately, we take nonlinearity,
measurement became available. Each time a new measurement
heterogeneity, and the complete historical degradation data into
came out, they updated the posterior distribution of using the
consideration simultaneously by combining (1) with (3). Ac-
prior distribution and the new measurement by a recursive lter.
cording to the Euler discretization, our complete model at the
When the next measurement was available, the posterior distri-
th CM point is formulated as
bution at the current CM point became the prior distribution for
the next CM point. Thus, they established a link between the
(4)
historical degradation data and the predicted degradation levels
690 IEEE TRANSACTIONS ON RELIABILITY, VOL. 64, NO. 2, JUNE 2015
(5)
,
HUANG et al.: REMAINING USEFUL LIFE PREDICTION 691
(12)
(10)
(11)
692 IEEE TRANSACTIONS ON RELIABILITY, VOL. 64, NO. 2, JUNE 2015
Once , and are obtained, and can be 8: Calculate by (20), and search for
derived as the optimal parameter set to maximize
.
(18) 9: until
10: .
(19)
11: end for
Substituting , , and into (11), it can be rewritten as
IV. RUL PREDICTION
Thus far, we have estimated the degradation drift, and the
unknown parameters. In this section, we will discuss the RUL
prediction utilizing the estimated degradation drift and parame-
ters. In literature, there exist two methods relating the predicted
degradation level to the predicted RUL at the th CM point,
which dene the RUL as , and
. As pointed out in [31], [39],
the rst denition neglects the possible hitting events within
, which can result in the under-maintenance, and is
(20) unacceptable, especially for some safety-critical systems. Con-
sequently, we adopt the second denition, i.e., the denition of
2) M Phase: After deriving , we obtain the pa- FPT, as the link between the predicted degradation level and the
rameter estimate at the current iterative step, i.e., , by max- RUL.
imizing (20) with respect to . As , , and in the rst line In conformity with our model, at the th CM point, the evo-
of (4) are s-independent of and in the second line of (4), , lution of the degradation level is written as
, and have analytically optimal solutions.
(21) (24)
So the RUL at the th CM point is dened as the time that
(22) lapses from the current time to the FPT that the degrada-
tion level rst hits a specic threshold if has never
reached before . Due to the introduction of the adaptive drift
and the nonlinear structure, the RUL prediction based on our
(23) model cannot be directly characterized by the IG distribution as
The remaining parameters and can be estimated numeri- the conventional WP model. Thus, Theorem 3 is proposed to ad-
cally by maximizing (20). The E phase and the M phase are ex- dress this issue. Before introducing Theorem 3, we rst give the
ecuted iteratively until the parameter estimates converge. The following Lemma 1 [31] to simplify the derivation of Theorem 3.
estimation of the degradation drift and the unknown parameters Lemma 1: If a random variable satises that
is summarized as Algorithm 1. , then the expectation can be formulated
as
Algorithm 1 Estimation of the Degradation Drift and the
Unknown Parameters (25)
(27)
where .
Remark 2: Wang's model, which ignores the uncertainty of
, simply supplants with its expectation in (24). Further-
more, in both Wang's model and Si's model equal , so
their models cannot trace the evolution of nonlinear degradation
trajectories well. Our model considers the complete distribution
of , and incorporates a nonlinear structure into . Both
of them make our model to trail practical degradation trajecto-
ries better, and attain more accurate RUL prediction in compar-
ison with Wang's model and Si's model.
A. A Numerical Simulation
The degradation drift and the unknown parameters need to be
estimated in our model, so the accuracies of the estimated drift
and parameters should be tested. However, their explicit values
may be difcult to be specied in practical systems. Hence, to
demonstrate the convergent property and the predictive capa-
bility of our model, a numerical simulation is provided.
For illustration, we consider the nonlinear structure
as , so the unknown parameter equals . Specically, the
following model is exploited to generate the simulation data.
Fig. 4. The raw vibration signal, and the RMS feature of bearing #1 3. (a) The
Fig. 3. PDFs of the RULs based on our model and the WP model against the raw vibration signal of bearing #1 3. (b) The RMS feature of bearing #1 3.
actual RULs.
and the upper limit of the degradation drifts, respectively. The the on-line CM information. In contrast with other bearing
dashed line represents the actual degradation drifts. It shows that experimental platforms, the data engendered by PRONOSTIA
the estimated degradation drifts can track their true values well. correspond to normally degraded bearings, which indicates
So far, we have demonstrated that we can estimate both the that the defects are not initiated on the bearings; and that the
hidden degradation drift and the unknown parameters quite well degraded bearings contain almost all the types of defects,
by Algorithm 1. Subsequently, we will utilize both of them to including faults in balls, rings, and cages.
predict the RULs at each CM point. What's more, the predicted The sampling frequency of the vibration signal is 25.6 kHz,
RULs based on the WP model are also calculated for compar- but the experimenters do not record each sample. They record
ison. The PDFs dependent on our model and the WP model at 2560 consecutive samples (i.e., the samples during 0.1 s) every
the 15th, 18th, 21st, and 24th CM points are plotted in Fig. 3. 10 seconds. The raw signal is pre-processed by extracting the
It shows that the PDFs of the predicted RULs provided by our root mean square (RMS) feature for the recorded data. After the
model are more compact than those given by the WP model, transformation, the RMS feature sequence forms a new time se-
which indicates that the uncertainties of the predicted RULs ries of degradation data whose sampling interval is 10 seconds.
from our model are lower than those from the WP model. In ad- To avoid propagation of damages to the whole experimental
dition, our model can obtain more accurate RULs than the WP platform, the test is ceased when the amplitude of the vibra-
model does. The predicted RULs from the WP model are far tion signal overpasses 20 g, where 20 g is short for twenty-fold
away from the actual RULs, especially at the early CM points. gravity acceleration. The raw horizontal vibration signal of one
In contrast, the actual RULs are always covered by the predicted of the bearings, called bearing #1 3, and its corresponding
RULs from our model with large probability densities. RMS feature, are shown in Fig. 4.
In conformity with the denition of failure, once a raw signal
overpasses 20 g, the tested bearing is considered to reach the
B. A Practical Case Study
failure threshold, so the amplitude of the corresponding RMS
In this subsection, we utilize a practical example to compare is dened as the failure threshold in the RMS feature domain
the performance of the RUL prediction from our model with that in this paper. Taking bearing #1 3 for example, at its 2342nd
from Wang's model, and Si's model. The bearing dataset [35] sampling set, the raw vibrational signals overpass 20 g, so its
comes from a laboratory experimental platform (PRONOSTIA) corresponding RMS value, 4.7145, is dened as the threshold
which enables the accelerated degradation of bearings in the RMS feature domain, and the degradation trajectory is
under constant or variable operating conditions, and gathers truncated after the 2342nd CM point.
HUANG et al.: REMAINING USEFUL LIFE PREDICTION 695
TABLE I
ESTIMATED PARAMETERS OF ALL DEGRADATION MODELS
Fig. 6. The contrasting MSEs of RUL prediction. (a) The contrasting MSEs of
our model and Wang's model. (b) The contrasting MSEs of our model and Si's
model.
model are more compact than those from ours, not all actual
RULs are covered by Si's model. It implies that the predicted
RULs from Si's model deviate from the true RULs, especially at
Fig. 5. The comparison of the performances of RUL prediction. (a) Compar- the early CM points. The deviation may result from the strong
ison with Wang's model. (b) Comparison with Si's model. tracking ltering (STF) algorithm used in Si's model, because
the STF algorithm is more appropriate for the degradation pro-
cesses that have jumps, or sudden changes, but the degradation
Bearing #1 3 is used to compare the accuracies of the pre- trajectory in this experiment is a smooth one as seen in Fig. 4(b).
dicted RULs from our model with those from Wang's model, and To further compare the accuracies of the RUL prediction, we
Si's model. For illustrative purpose, we consider . calculate the mean square error (MSE) at each CM point. The
When all the degradation data are processed, the average con- contrasting MSEs are shown in Fig. 6. The dashed line, and the
sumed time, and the longest consumed time are 2.2147 s, and solid line in Fig. 6(a) represent MSEs of Wang's model, and
4.6403 s, respectively. Both of them are much smaller than the our model, respectively. The dashed line, and the solid line in
sampling interval, 10 s, which demonstrates that the proposed Fig. 6(b) represent MSEs of Si's model, and our model, respec-
approach is suitable for the real-time application. The estimated tively. It can be seen that, with the increase of the operating time,
parameters of all degradation models are shown in Table I. It is the MSEs of both Si's and our models, rather than Wang's model,
noteworthy that the estimated parameter in our model is not approach zero gradually. Note that the variations of the MSEs
equal to 1, which indicates that the degradation process is not a of Wang's model and Si's model are much larger than those of
linear path. our model, which means that the accuracies of the RUL predic-
The comparative performances of RUL prediction from our tion from their models uctuate obviously. In addition, most of
model, Wang's model, and Si's model are revealed in Fig. 5. the MSEs of their models are larger than those of ours, and the
Compared with Wang's model, the PDFs of the predicted RULs total MSEs (TMSEs) are listed in Table II. It is evident that the
from our model are much more compact, which means the un- TMSE of our model is half of that of Wang's model, and one
certainties of our model are much smaller. Each actual RUL fth of that of Si' model. In conclusion, all of the above results
is completely covered by both Wang's model and our model. indicate that the performance of RUL prediction from our model
In Fig. 5(b), though the PDFs of the predicted RULs from Si's surpasses that from Wang's model and Si's model.
696 IEEE TRANSACTIONS ON RELIABILITY, VOL. 64, NO. 2, JUNE 2015
VI. CONCLUSION
APPENDIX
In practical systems, nonlinear degradation trajectories exist
extensively, and the components may degrade slower or faster at Proof of Theorem 1:
different times, which indicates that the degradation processes
are not homogeneous all the time. Moreover, the conventional Proof: At the th ( ) CM point, we have the prior
WP model depends on a strong Markovian assumption which knowledge ,
implies that the degradation level prediction relies only on the , and
current degradation level rather than the historical degradation . According to the law of total probability, and
data. It may give rise to an inaccurate prediction. To correct Bayes' theorem, the PDF of the estimated current degradation
these deciencies, we concentrate on a nonlinear heterogeneous drift is derived as shown in (29) at the bottom of the page,
WP model with an adaptive drift. To update the distribution of where
the estimated degradation drift, an online ltering algorithm
based on Bayes' theorem is proposed. Additionally, the EM
algorithm is adopted to re-estimate the parameters, while the
backward smoothed degradation drift is calculated by a novel
smoother based on Bayes' theorem. Subsequently, the PDF ,
and CDF of the predicted RUL incorporating the complete ,
distribution of the estimated degradation drift are obtained an- and . Invoking the integral
alytically. A simulation is provided to verify the effectiveness , we have (30), shown at
of the online ltering algorithm and the parameter estimation the bottom of the next page.
method. In addition, it shows that our model can obtain more In comparison with the PDF of the Gaussian distribu-
accurate predicted RULs than the conventional WP model does tion, it is not difcult to verify that
for nonlinear degradation trajectories. Furthermore, a practical is s-normally distributed with the expectation
example is presented to compare the performance of RUL , and the vari-
prediction from our model with that from Wang's model and ance .
Si's model, and the experimental results indicate that our model At the rst iteration, i.e., , we have the prior knowl-
surpasses both of them. edge , ,
(29)
HUANG et al.: REMAINING USEFUL LIFE PREDICTION 697
. Similar to the derivation of other iterations, the degra- According to the state estimation and the structural infor-
dation drift at the rst iteration is formulated as mation of our model, we have ,
, and . In
addition, is assumed to satisfy
. Substituting these quantities into (32), we
have
(31)
(35)
Comparing the rst three expressions in (33) with (34), the co-
variance matrix of the joint distribution, , is for-
mulated as
(36)
(32)
(30)
698 IEEE TRANSACTIONS ON RELIABILITY, VOL. 64, NO. 2, JUNE 2015
so
(37) (44)
(39)
(45)
To seek the recursive relationship between the consecutive
expectations, we compare the fourth expression of (33) with Subsequently, the PDF of the predicted RUL conditional on
(34) to achieve and can be obtained by taking the derivative of (45)
with respect to the residual lifetime , given as
(40)
(41)
(46)
(42)
Substituting (42) into (38), the initialization of the covariance is Considering the complete distribution of the estimated degra-
given as dation drift, the PDF, and CDF of the predicted RUL are respec-
tively calculated as
(43)
Proof of Theorem 3:
Proof: According to [13], at the th CM point, the FPT
crossing the threshold conditional on and follows the (47)
IG distribution, that is,
(48)
HUANG et al.: REMAINING USEFUL LIFE PREDICTION 699
(50)
[26] L. I. Pettit and K. D. S. Young, Bayesian analysis for inverse Gaussian Zeyi Huang received his B.Eng. from the College of Automation Science
lifetime data with measures of degradation, J. Statist. Computat. Sim- and Engineering at South China University of Technology in 2012. He is
ulat., vol. 63, no. 3, pp. 217234, 1999. currently a master candidate with the State Key Laboratory of Industrial
[27] N. Z. Gebraeel, M. A. Lawley, R. Li, and J. K. Ryan, Residual-life dis- Control Technology, and the Department of Control Science and Engineering,
tributions from component degradation signals: A Bayesian approach, Zhejiang University.
IIE Trans., vol. 37, no. 6, pp. 543557, 2005. His research interest covers residual life time prediction.
[28] M. E. Robinson and M. J. Crowder, Bayesian methods for a
growth-curve degradation model with repeated measures, Lifetime
Data Anal., vol. 6, no. 4, pp. 357374, Dec. 2000.
[29] Z. Ye, N. Chen, and K.-L. Tsui, A Bayesian approach to condition Zhengguo Xu (M09) received the B.Eng. degree in electrical engineering from
monitoring with imperfect inspections, Qual. Rel. Eng. Int., to be pub- Changsha Electric Power University, Changsha, China, in 2002; and the Ph.D.
lished. degree in control science and engineering from Tsinghua University, Beijing,
[30] W. Wang, M. Carr, W. Xu, and K. Kobbacy, A model for residual life China, in 2009.
prediction based on Brownian motion with an adaptive drift, Micro- He is currently an Associate Professor with the State Key Laboratory of In-
electron. Rel., vol. 51, no. 2, pp. 285293, Feb. 2011. dustrial Control Technology, and the Department of Control Science and Engi-
[31] X.-S. Si, W. Wang, C.-H. Hu, M.-Y. Chen, and D.-H. Zhou, A Wiener- neering, Zhejiang University, Hangzhou, China. His research interests include
process-based degradation model with a recursive lter algorithm for control and safety for railway transportation, reliability analysis for complex
remaining useful life estimation, Mech. Syst. Signal Process., vol. 35, systems, and fault diagnosis.
no. 12, pp. 219237, Feb. 2013.
[32] W. Wang, A two-stage prognosis model in condition based mainte-
nance, Eur. J. Oper. Res., vol. 182, no. 3, pp. 11771187, Nov. 2007.
[33] L. Feng, H. Wang, X. Si, and H. Zou, A state-space-based prognostic
model for hidden and age-dependent nonlinear degradation process, Wenhai Wang received the B.Eng., M.Sci., and Ph.D. degrees from Zhejiang
IEEE Trans. Automat. Sci. Eng., vol. 10, no. 4, pp. 10721086, Oct. University, Hangzhou, China, in 1989, 1992, and 1996, respectively.
2013. He is currently a Professor with the State Key Laboratory of Industrial
[34] X.-S. Si, W. Wang, M.-Y. Chen, C.-H. Hu, and D.-H. Zhou, A degra- Control Technology, and the Department of Control Science and Engineering,
dation path-dependent approach for remaining useful life estimation Zhejiang University. His research interests include railway transportation,
with an exact and closed-form solution, Eur. J. Oper. Res., vol. 226, distributed control systems, and industrial automation.
no. 1, pp. 5366, Apr. 2013.
[35] P. Nectoux, R. Gouriveau, K. Medjaher, E. Ramasso, B. Morello, N.
Zerhouni, and C. Varnier, PRONOSTIA: An experimental platform
for bearings accelerated degradation tests, in Proc. IEEE Int. Conf. Youxian Sun received the B.Eng. degree from Zhejiang University, Hangzhou,
Prognostics and Health Management, Denver, CO, USA, 2012. China, in 1964.
[36] X.-S. Si, W. Wang, C.-H. Hu, D.-H. Zhou, and M. Pecht, Remaining Since then, he has been with Zhejiang University, where he was promoted
useful life estimation based on a nonlinear diffusion degradation to Associate Professor in 1982, and Professor in 1988. From 1984 to 1987, he
process, IEEE Trans. Rel., vol. 61, no. 1, pp. 5067, Mar. 2012. received the Humboldt Fellowship to be a Visiting Professor with Stuttgart Uni-
[37] A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood versity, Stuttgart, Germany. He is currently the Director of the Institute of In-
from incomplete data via the EM algorithm, J. Royal Statist. Soc.. Ser. dustrial Process Control, and the Director of the National Engineering Research
B (Methodol.), vol. 39, no. 1, pp. 138, 1977. Center of Industrial Automation. He also serves as President of the Chinese As-
[38] C. F. J. Wu, On the convergence properties of the EM algorithm, sociation of Automation, and Vice President of the China Instrument and Con-
Ann. Statist., vol. 11, no. 1, pp. 95103, Mar. 1983. trol Society. He previously served as Vice Chairman of the IFAC Pulp and Paper
[39] J. I. Park and S. J. Bae, Direct prediction methods on lifetime distri- Committee. He has authored or coauthored six books, and published more than
bution of organic light-emitting diodes from accelerated degradation 400 papers. His research interests include modeling, control, and optimization
tests, IEEE Trans. Rel., vol. 59, no. 1, pp. 7490, Mar. 2010. of complex engineering systems; robust control theory and applications; and
distributed control systems.