See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/318917280

Remaining Useful Life Prediction for a Machine With Multiple Dependent

Features Based on Bayesian Dynamic Linear Model and Copulas

Article  in  IEEE Access · August 2017

DOI: 10.1109/ACCESS.2017.2735966

CITATIONS READS

4 346

4 authors, including:

Fuqiang Sun Xiao-Yang Li

Beihang University (BUAA) Beihang University (BUAA)
32 PUBLICATIONS   75 CITATIONS    110 PUBLICATIONS   406 CITATIONS   

SEE PROFILE SEE PROFILE

Wei Zhang
Beihang University (BUAA)
18 PUBLICATIONS   135 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Reliability Modeling of the dependent competing failure systems with multiple degradation processes and random shocks View project

Relative entropy based Bayesian design for accelerated degradation testing View project

All content following this page was uploaded by Fuqiang Sun on 13 September 2017.

The user has requested enhancement of the downloaded file.

SPECIAL SECTION ON COMPLEX SYSTEM HEALTH MANAGEMENT BASED
ON CONDITION MONITORING AND TEST DATA

Received June 1, 2017, accepted July 18, 2017, date of publication August 4, 2017, date of current version September 6, 2017.
Digital Object Identifier 10.1109/ACCESS.2017.2735966

Remaining Useful Life Prediction for a Machine

With Multiple Dependent Features Based on
Bayesian Dynamic Linear Model and Copulas
FUQIANG SUN, NING WANG, XIAOYANG LI, AND WEI ZHANG
Science and Technology on Reliability and Environmental Engineering Laboratory, School of Reliability and Systems Engineering,
Beihang University, Beijing 100191, China
Corresponding author: Wei Zhang (zhangwei.dse@buaa.edu.cn)
This work was supported by the National Natural Science Foundation of China under Grant 61603018 and Grant 51405009.

ABSTRACT Degradation modeling and remaining useful life (RUL) prediction for products with multiple
degradation features are hot topics in the prognostic and health management. The key to this problem is
to describe the dependence among multiple degradation features effectively. In this paper, a multivariate
degradation modeling approach based on the Bayesian dynamic linear model (BDLM) is proposed to
calculate the RULs of degradation features, and the Copula function is employed to capture the dependence
among RUL distributions. A combined BDLM is used to establish the multivariate degradation model,
which includes two typical BDLMs, namely, the linear growth model and seasonal factors model. After
the model parameters get calibrated by the maximum likelihood estimation, the model can predict the
degradation process of features. Once the failure thresholds are given, the probability density function
and cumulative distribution function (CDF) of RUL for each degradation feature can be obtained. Since
these RUL distributions are not independent of each other, the Copula function is adopted herein to couple
the CDFs. Finally, some practical testing data of a microwave component, which has two degradation
features, are utilized to validate our proposed method. This paper provides a new idea for the multivariate
degradation modeling and RUL prediction.

INDEX TERMS Multiple dependent features, Bayesian dynamic linear model, remaining useful life,
copulas.

I. INTRODUCTION
Prognostic and health management (PHM) has been inten-
sively studied as an effective tool for managing life cycles of
machines and reducing maintenance cost. Degradation mod-
eling and remaining useful life (RUL) prediction is the core
of prognostics [1]. An accurate RUL prediction could provide
the basis for maintenance decisions, spare parts provision,
and preventing fatal failures.
The existing RUL prediction methods can be categorized
into two broad classes, i.e., physics of failure (PoF) models
and data-driven approaches [1]. The PoF models are achieved
on the basis of acquired knowledge of the failure mechanisms.
However, it is difficult, if not impossible, to understand the
physical failure mechanisms of a complex component or sys-
tem. On the other hand, the data-driven approaches do not
rely on the failure mechanisms of machines, which only rely
on available past observed data and statistical models [2].
They are low cost approaches and have the advantage
of better applicability [3]. Due to these advantages, data-
driven approaches have become the popular RUL predic-
tion methods. There have been excellent literature reviews
over the last decade on data driven approaches for RUL
prediction. Si et al. [2] reviewed the developments of the
RUL estimation models based on statistical data driven
approaches. Javed et al. [3] performed a comprehensive sur-
vey on prognostics approaches and discussed the application
point of view for each category. They classified data-driven
approaches as machine learning and statistical learning
approaches, and elaborated their close links. These reviews
papers cover most of RUL prediction methods. However, they
share one thing in common in that they only discussed the
cases involving a single degradation feature.
However, the machine degradation is usually reflected
by several interrelated features. Such features may include

2169-3536
2017 IEEE. Translations and content mining are permitted for academic research only.
VOLUME 5, 2017 Personal use is also permitted, but republication/redistribution requires IEEE permission. 16277
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features

not only the functional and/or performance parameters of

the machines, but also some indirect degradation features
extracted from raw sensory signals [4], such as vibration,
force and acoustic signals, temperature and voltage. There-
fore, a machine may have multiple features and they may
be not independent of each other. For the high dimensional
degradation data, the principal component analysis, singu-
lar value decomposition, self-organizing map, or clustering
method are often used to implement feature dimension reduc-
tion and determine the most suitable feature to conduct degra-
dation modeling and RUL prediction [5]. These studies often
assume that the high dimensional features could transform
into single feature or some low dimensional independent
features through feature dimension reduction, which is only
suitable for the situations that multiple features are linear cor-
relation. However, in the field of PHM, little effort has been
focused on the mathematical modeling of multiple dependent
features.
The degradation data of multiple features over time could
constitute a set of time series. Dynamic linear model (DLM)
provides a new way for the degradation modeling with mul-
tiple dependent features, which is widely used in the time
series analysis [6]–[8]. The dynamic linear model was first
proposed by West and Harrison [9]. In order to deal with
FIGURE 1. The overview of the proposed method.
the multivariate problems, Triantafyllopoulos [10] developed
a Bayesian dynamic linear model (BDLM) to forecast the
multivariate time series based on Wishart and singular mul- dynamic linear model and copulas is proposed. The overview
tivariate beta distributions, which considers the dependency of the proposed method is shown in Fig. 1. The BDLM,
among multivariate time series. Lau et al. [11] used the consists with linear growth model and seasonal factor model,
multivariate DLM to monitor influenza activity throughout is introduced to describe the degradation processes with mul-
epidemics based on multivariate data. Lee et al. [12] adopted tiple dependent features. Model parameters are estimated
a kind of multivariate DLM to dig the concealed phonotactic using maximum likelihood estimation (MLE). Kalman fil-
structure out of the phone-likelihood vectors. It can be seen tering is used for handling the iterative updating based on
that the DLM has been successfully applied in many fields the observed degradation data. Using the updating results,
to model multivariate time series, however, scarce studies it can predict the degradation trend and bounds. Then, after
have been conducted on the degradation modeling and RUL setting the failure threshold for each degradation feature,
prediction of multiple dependent features. a random sampling procedure is employed for the prediction
Using aforementioned multivariate degradation model, of marginal RUL distribution. Copulas are used to link the
the RUL distribution of each feature can be estimated. Then, dependency among multiple RUL distributions and provides
another problem is to estimate the RUL joint distribution the RUL joint distribution of a machine.
of a machine with multiple dependent features, which is The advantage of the proposed method is that it can
helpful for the future maintenance schedule planning. In the take the comprehensive consideration of dependency. In the
literature, two popular methods have been utilized to identify degradation process, the BDLM captures the dependency
a multivariate joint probability, namely, multivariate normal among multiple features during degradation process. Besides,
distribution [13] and the copulas method [14]–[20]. In prac- the Copulas describe the dependency of RUL distributions for
tice, assuming a multivariate normal distribution may not be multiple degradation features. By this way, the dependency
suitable for all conditions. As a useful alternative, the copu- can be better described, which can improve the precise of the
las can estimate the multivariate joint distribution by com- RUL prediction.
bining the marginal distributions using a joint dependence The reminder of this paper is organized as follows.
structure, and do not impose constraints on the univariate Section 2 presents the theory of Bayesian dynamic linear
marginal distributions [21]. Due to the flexibility, the copulas model, elaborates the methods of multivariate degradation
method has been extensively applied in the field of reli- modeling and remaining useful life prediction. The Copula
ability analysis [14], [15], degradation modeling [16]–[18], function is described in Section 3, and is used for calculating
warranty claims and prognostics [19], [20], [22]. the joint RULs distribution. Section 4 performs a case study
In this paper, a novel multivariate degradation model- to verify the validity of the proposed model. Finally, some
ing and remaining useful life prediction method based on conclusions are drawn in Section 5.

16278 VOLUME 5, 2017

F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features
II. A BAYESIAN DYNAMIC LINEAR MODEL FOR
RUL PREDICTION
A. BAYESIAN FORECASTING FRAMEWORK
The Bayesian forecasting establishes a relationship between
the prior information Dt−1 and the posteriori distribution
considering the new measurements from the updated infor-
mation Dt . It can be represented by Bayes theorem as [23]
random changes of state vector, N (0, W ) is the multivariate
normal distribution with mean 0 and covariance variance W .
Both vt and wt are white-noise errors following normal dis-
tribution with a priori known variance matrices V and W ,
respectively.
Next, two typical DLMs, namely linear growth and sea-
sonal factors model, are briefly introduced.
p ŷt  dt ∝ p ŷt  dt−1 × p yt | ŷt , dt−1




(1)
1) LINEAR GROWTH MODEL
where ŷt is the predicted value
in time t, yt is the obser- Linear growth model is a commonly used class of dynamic
vation data at time
 t,
p ŷt
 dt is the posteriori distribution linear model, which assumes that the current level changes
given Dt , p
ŷt  dt−1 is the prior distribution given Dt−1 , linearly through time [7]. The state vector of linear growth
p yt | ŷt , dt−1 is the distribution of yt given ŷt and Dt−1 . model is θ t = [µt , βt ]0 , where µt and βt are the parameters
Equation (1) use the historical data and observation data of state vector θ t at time t, which are related to the corre-
for prediction, which is a standard Bayesian forecasting link- sponding state parameters at t − 1 by the state equation. The

priori information Dt−1 and posteriori distribution
ing the observation matrix Z and state matrix T can be expressed as
p ŷt  dt using the new observation data yt from the updated Z = [1, 0], T = [1, 1; 0, 1]. Then, according to (2) and (3),
information Dt [23]. the linear growth model can be expressed as

B. DYNAMIC LINEAR MODEL Observation equation: yt = µt + vt (4)

Dynamic linear model is an important class of state space State equation: µt = µt−1 + βt−1 + wt,1 (5)
model, where the state vector is specified by a normal prior State equation: βt = βt−1 + wt,2 (6)
distribution. Markovian dependence is arguably the simplest
form of the dependence structures among the observation where vt is the observation error, wt,1 and wt,2 are the errors
data [24]. However, the Markovian structure assumption for of state vector.
the observations may not be appropriate in many applications.
State space models build on the relatively simple dependence 2) SEASONAL FACTORS MODEL
structure of a Markov chain to define more complex models Usually, there are some cyclical behavior in the time series
for the observations [24]. data. Suppose the period of seasonal factors model is s, then
Formally, the state space model mainly consists of two the state vector is θ t = [α1 α2, , . . . , αs ]0 , where αi is the
parts: one is an unobservable Markov chain θt , called state seasonal factors. The observation matrix Z is a s dimensional
process; the other is yt , which is an imprecise measurement vector, Z = [1, 0, . . . , 0]. The state matrix T is a s × s matrix,
of θt and depends on θt only. The information flow of a state which are expressed as
space model is represented in Fig. 2 [24].  
0 0 ··· 1
1 0 ··· 0
T = .
 
 .. 

0 0 1 0
The seasonal factors αi must P be imposed a identifiability
FIGURE 2. The dependence structure for a state space model.
constrain, a common choice is sj=1 αj = 0 [24]. So there
As a state space model, the dynamic linear model (DLM) are s − 1 free factors effective. On this basis, the observation
was developed by West and Harrison [9] in the Bayesian fore- matrix Z is represented by s − 1 dimensional vector Z =
casting framework. The DLM consists of two basic equations: [1, 0, . . . , 0], and the state matrix T is a (s−1) ×(s−1) matrix,
state equation and observation equation [24].
 
−1 −1 · · · −1
Observation equation  1 0 ··· 0 
T = .
 
yt = Z θ t + vt , vt ∼ N (0, V ) (2)  .. 

and state equation 0 0 1 0 (s−1)×(s−1)
θ t = T θ t−1 + wt , wt ∼ N (0, W ) (3) C. BIVARIATE DEGRADATION MODEL
where yt is the observation data at time t, θ t is the state vector Bivariate degradation process is a special case of the multi-
at time t, Z and T are observation matrix and state matrix variate situations, and it is the foundation of the multivariate
respectively. vt is the observation error, N (0, V ) is the multi- degradation research. This study focuses on the RUL predic-
variate normal distribution with mean 0 and covariance vari- tion in the situation of bivariate dependent features. The linear
ance V , also known as the observational variance. wt is the growth model and seasonal factors model are combined to

VOLUME 5, 2017 16279

 F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features

establish a bivariate Bayesian dynamic linear model, in order where T2,i is state matrix of seasonal factors model for each
to model the bivariate dependent degradation process. degradation feature, i = 1, 2,
 
1) BIVARIATE LINEAR GROWTH MODEL −1 −1 ··· −1
(1)  1 0 ··· 0 
In a bivariate linear growth model, yi,t is the ith observa-
T2,i = ..
 
(1)
tion data at time t, i = 1, 2. θ t is the state vector at .

 
time t. Z1 and T1 are the observation matrix and state matrix 0 0 1 0 (s−1)×(s−1)
respectively.
The observation equation is 3) COMBINED BIVARIATE BDLM
"
(1)
# The bivariate linear growth model and bivariate seasonal
y1,t (1)
(1) = Z1 θ t + vt (7) factors model are combined to establish the bivariate BDLM,
y2,t yi,t is the ith observation data at time t, i = 1, 2.
The state vector θ t consists of the state vector of linear
where vt is the observation error, vt ∼ N (0, V ), (1)
growth model θ t and the state vector of seasonal factors
(2)

1 0 0 0
 model θ t , and can be expressed as θ t = [µ1,t , β1,t ,
Z1 = . µ2,t , β2,,t , α1 α2, , . . . , αs−1 , τ1 , τ2, , . . . , τs−1 ]0 . Then the
0 0 1 0
observation matrix Z and state matrix T can be
The state equation can be expressed as reformulated as
(1) (1)
θt = T1 θ t−1 + wt (8) Z = [Z1 , Z2 ] (11)
 
T1 0
(1) 0 T = (12)
where θ t = µ1,t , β1,t , µ2,t , β2,t , wt

is the state error, and 0 T2
 
1 1 0 0 Then, the observation equation can be represented by
0 1 0 0
wt ∼ N (0, V ), T1 =  .  
0 0 1 1 y1,t
0 0 0 1 = Z θ t + vt (13)
y2,t

2) BIVARIATE SEASONAL FACTORS MODEL Finally, the state equation can be expressed as
In a bivariate seasonal factors model, suppose the periods for
both of the degradation features are s, then a pure seasonal θ t = T θ t−1 + wt (14)
factors model without fluctuation is used to model the peri-
(2)
odic data. yi,t is the ith observation data at time t, i = 1, 2. D. PARAMETER ESTIMATION AND ITERATIVE UPDATING
(2)
θ t is the state vector at time t. Z2 and T2 are the observation 1) PARAMETER ESTIMATION
matrix and state matrix respectively. Maximum likelihood estimation (MLE) is used for the esti-
The observation equation is mation of model parameters, namely the covariance variance
V and W , which can be denoted as 9 = [V , W ].
(2)
" #
y1,t (2) The joint log-likelihood function for the measurement
(2) = Z2 θ t (9)
y2,t sequence y1:t and the state sequence θ1:t can be formulated
as [25]
where
  L ( y1:t , θ 1:t | 9)
1 0 ··· 0 0 0 ··· 0
.
hYt Yt
Z2 =
i
0 0 ··· 0 1 0 ··· 0 2×(2s−2)
= log p ( y1:i | θ 1:i , 9) p ( θ 1:i | θ 1:i−1 , 9)
i=1 i=1
t t
The state equation can be expressed as = const − log |V | − log |W |
2 2
(2) (2) t
θt = T2 θ t−1 (10) −
1 X
(yi − Z θ i )T V −1 (yi − Z θ i )
2
(2) i=1
where θ t = [α1 , α2 , · · · , αs−1 , τ1 , τ2 , · · · , τs−1 ]0 , αj and τj t
are the seasonal factors of each degradation feature, j = 1X
− (θ i − T θ i−1 )T W −1 (θ i − T θ i−1 ) (15)
1, 2, . . . , s − 1. T2 is the state matrix of bivariate seasonal 2
i=1
factors model,
  where p(·) represents the normal distribution.
T2,1 0 Then, the model parameters could be estimated by maxi-
T2 =
0 T2,2 mizing the joint log-likelihood function (15).

16280 VOLUME 5, 2017

F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features

2) ITERATIVE UPDATING If all information has been known as Dall , and θ t+1 | Dall ∼
After constructing the degradation model using Bayesian N (st+1 , St+1 ), then θ t | Dall ∼ N (st , St ), where
dynamic linear model, it is significant to estimate the degra-
dation state. Once the new observation data are available, t+1 (st+1 − at+1 )
st = mt + Ct T 0 R−1 (23)
we could utilize the forward-recursive algorithm Kalman fil- St = t+1 (Rt+1
Ct − Ct T 0 R−1 − St+1 ) R−1
t+1 TCt (24)
tering and backward-recursive algorithm Kalman smoothing
to update the model parameters. This step is implemented until t = 0, then all the smooth-
ing data can be obtained.
a: KALMAN FILTERING
E. RUL PREDICTION OF EACH FEATURE
Kalman filtering can update the BDLM based on observed
data under Bayesian framework. It is a suitable choice for Using the established BDLM, it is convenient to predict the
the state estimating, due to the observation data yt and state degradation model, and calculate the corresponding RUL
vector θt follow normal distribution for any t ≥ 1 [9]. distribution for each degradation feature.
For a normal distribution, it can be determined once the The remaining useful life is defined as the time from the
mean and variance are given. The details of the filtering in current moment to the end of its useful life [2]. Commonly,
the Bayesian framework are presented as follows [24]. the end of useful life is the degradation curve passes through
Given the initial information: θ 0 |D0 ∼ N (m0 , C0 ), where a predefined failure threshold in the first time. The principle
m0 and C0 are the initial mean and standard deviation of state of the RUL prediction is illustrated as Fig. 3.
vector θ0 , θ 0 |D0 represents the initial knowledge given the
information D0 .
Step 1 initialization: At time t = 0, given the informa-
tion D0 , the initial parameters of θ is specified as

( θ 0 | D0 ) ∼ N (m0 , C0 ) (16)

where m0 and C0 are the prior values.

Step 2 a priori estimation: The posteriori distribution
at time t − 1 is used to estimate the prior distribution
θ t |Dt−1 ∼ N (at , Rt ) at time t, i.e.

at = E ( θ t | Dt−1 ) = T mt−1 (17)

0
Rt = Var ( θ t | Dt−1 ) = TCt−1 T + W (18)

Step 3 forecast: The mean and variance of the priori

distribution θ t |Dt−1 is used to predict the mean ft and FIGURE 3. Principle of RUL prediction.
variance Qt of yt ,
The flow chart for estimating the RUL distribution of
ft = E ( yt | Dt−1 ) = Z at (19)
0 each feature based on BDLM is shown in Fig. 4. Firstly,
Qt = Var ( θ t | Dt−1 ) = ZRt−1 Z + V (20) the bivariate BDLM degradation model is established based
on linear growth model and seasonal factors model, and
Step 4 update: After the observation data yt is available,
Kalman filtering is implemented under Bayesian forecasting
update the posteriori distribution of θt , the updated mean and
framework for model estimation. Secondly, the result of the
variance under Dt are
Kalman filtering is used as the initial distribution, and the
0
mt = E ( θ t | Dt ) = at + Rt Z Q−1 (21) degradation trend and bounds for future time are predicted.
t et
0 Finally, the RUL distribution of each degradation feature is
Ct = Var ( θ t | Dt ) = Rt − Rt Z Q−1
t ZRt (22)
predicted using sampling method.
where et = yt − ft . Suppose the failure threshold of a degradation feature as M ,
Step 5 : Remove to the next time point, and implement then the remaining useful life can be calculated by
these Steps from Step 2 until to the end of observation data. RUL = inf {r : yt+r > M |yt < M } (25)
b: KALMAN SMOOTHING So it is necessary to predict the yt+r . The means and
In some cases, one may want to retrospectively reconstruct variances of the state vector θ t+k and observation yt+k
the behavior of system to study the underlying informa- for the future time t + k are calculated as follows [26],
tion. So the backward-recursive algorithm can be applied to k = 1, 2, . . . N , where N is a positive integer.
achieve this goal. The Kalman smoother is also performed in When the information Dt at time t is known, the Kalman
the dynamic linear model [24]. filtering is used to update the posteriori distribution of θ t at

VOLUME 5, 2017 16281


FIGURE 4. The flow chart of RUL distribution estimation.
time t, namely θ t |Dt ∼ N (mt , Ct ), which is used as the initial
distribution for the prediction.
Let at (0) = mt , Rt (0) = Ct be the initiations for predic-
tion, then for k ≥ 1, θ t+k |Dt ∼ N (at (k), Rt (k)), which could
be calculated by
at (k) = T at (k − 1) (26)
Rt (k) = TRt (k − 1) T + W 0
(27)
The yt+k with given information Dt follows normal distri-
bution yt+k |Dt ∼ N (ft (k), Qt (k)).
ft (k) = Z at (k) (28)
Qt (k) = ZRt (k) Z 0 + V (29)
On this basis, the probability density function (PDF) and
cumulative distribution function (CDF) of the RUL can be
estimated based on a sampling approach with the following
steps.
Step 1: A random sampling is implemented based on
yt+k ∼ N (ft (k), Qt (k)) to generate yt+k , k = 1, 2, . . . N ,
which is used as a simulated degradation process.
Step 2: For a given failure threshold M , a RUL can be
calculated based on (25) and the simulated degradation data
in Step 1.
Step 3: Repeat Step 1 and Step 2 for K times (K ≥ 1000),
then one can get K RULs.
Step 4: Kernel smoothing function estimation method [27]
is used to fit the CDF F(x) and PDF f (x) of RULs.
III. THE ESTIMATION OF JOINT RULs DISTRIBUTION
Section II has predicted the RUL distribution for each degra-
dation feature. While, the machine’s RUL prediction is more
significant and meaningful in PHM, which reflects the con-
dition of a system. Since the multiple degradation features
are not independent with each other, it is not appropriate to
ignore the dependency among multiple RUL distributions.
16282
F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features
Copulas provide a convenient approach to describe
the dependency and identifying multivariate probability
distribution.
A. COPULA THEORY
Copulas model separately the marginal distributions and
the joint-dependence structure [21]. The theoretical basis of
Copula function is Sklar ’s theorem [28], which proves the
existence and uniqueness of Copula function.
Sklar ’s theorem: Let H be a bivariate joint distribution
function with margins F(x) and G(y). Then there exists a
copula C such that for all x, y in R,
H (x, y) = C(F(x), G(y)) (30)
If F(x) and G(y) are continuous, then C is unique. Con-
versely, if C is a copula and F(x) and G(y) are distribution
functions, then the function H (x, y) defined by (30) is a joint
distribution function with margins F(x) and G(y).
A two-dimensional copula is a function C from I2 to
I = [0, 1] with the following properties [29]:
(1) Boundary conditions: for every u in I,
C(u, 0) = C(0, u) = 0 (31)
and
C(u, 1) = C(1, u) = u (32)
(2) Which is 2-increasing on I2 , i.e., for every u1 , u2 , v1 ,
v2 in I such that u1 ≤ u2 and v1 ≤ v2 ,
C(u2 , v2 ) − C(u2 , v1 ) − C(u1 , v2 ) + C(u1 , v1 ) ≥ 0 (33)
Furthermore, let c(u, v) = ∂ 2 C(u, v)/∂u∂v be the density
of copula function C, then the PDF corresponding to joint
distribution function H (x, y) can be calculated as
h(x, y) = c(F(x), F(y)) · f (x) · g(y) (34)
where, f (x) andg(y) are the PDFs of margins F(x) and G(y)
respectively.
In this paper, five commonly used Copula families are
selected for analysis, as shown in TABLE 1 [16].
In order to select the most suitable Copula family to
describe the joint distribution function, the Akaike infor-
mation criterion (AIC) is used to quantify the fitting effect
of Copulas. The Copula family with smaller AIC value fits
better. The AIC is defined as
AIC = 2q − 2 ln (L) (35)
where q denotes the number of unknown parameters in the
model, ln(L) is the maximum value of the log-likelihood
function.
B. THE PDF OF THE MACHINE’S RUL DISTRIBUTION
Suppose Mi is the failure threshold for ith degradation fea-
tures, the remaining useful life RULi for ith degradation
VOLUME 5, 2017
F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features

TABLE 1. Several commonly used Copula families.

FIGURE 5. The monitoring data for power gain A and B.

feature is calculated by (25), i = 1, 2. The RUL for a machine

can be expressed as
RUL = min {RUL1 , RUL2 } (36)
In this paper, the CDF of RULi can be calculated using the
proposed method in Section II, which is denoted by Fi (t),
i = 1, 2. If the degradation features are independent, then
the CDF of machine’s RUL F(t) is calculated as
F (t) = P (RUL ≤ t) FIGURE 6. Scatter plots of a specimen’s two degradation features.
= P (min (RUL1 , RUL2 ) ≤ t)
= 1 − P (min (RUL1 , RUL2 ) > t)
= 1 − P (RUL1 > t, RUL2 > t)
= 1 − P (RUL1 > t) · P (RUL2 > t)
= 1 − (1 − P (RUL1 ≤ t)) · (1 − P (RUL2 ≤ t))
= 1 − (1 − F1 (t)) · (1 − F2 (t))
= F1 (t) + F2 (t) − F1 (t) · F2 (t) (37)
And the PDF of machine’s RUL f (t) is
f (t) = f1 (t) · (1 − F2 (t)) + f2 (t) · (1 − F1 (t)) (38)
To capture the dependency among the RULs of multiple
degradation features, the Copulas are employed to establish
the machine’s RUL distribution,
FIGURE 7. The filtering and smoothing results for power gain A.
F (t) = C (F1 (t) , F2 (t)) (39)
Then the PDF of the machine’s RUL distribution can be
calculated as of the machine, power gain A and power gain B, are mon-
itored during the operation process, the monitoring data are
f (t) = c (F1 (t) , F2 (t)) · f1 (t) · f2 (t) (40) shown in Fig. 5.
Fig. 6 depicts the scatter plots of a specimen’s two degrada-
IV. CASE STUDY tion features. It is clearly to see that two degradation features
A case study on a microwave machine is carried out have obvious dependency.
to demonstrate the effectiveness of the proposed method. Bayesian Dynamic linear model is used to establish bivari-
In order to predict its remaining useful life, two key features ate degradation model for the microwave machine. The model

VOLUME 5, 2017 16283

 F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features

FIGURE 8. The filtering and smoothing results for power gain B.

FIGURE 10. The prediction of power gain B. (a) The prediction result.
(b) Partial enlarged view.

The covariance variance of observation error V and state

error W are estimated using MLE method,
 
0.0024 0.0021
V =
0.0021 0.0033
 
0.0003323 0 −7.773e − 5 0
 0 2.864e − 176 0 0
W =  −7.773e − 5

0 0.0002668 0 
0 0 0 0
The Kalman filtering and Kalman smoothing algorithm are
applied to the bivariate BDLM, and the results for two degra-
dation features are shown in Fig. 7 and Fig. 8 respectively.
Besides, based on the bivariate BDLM degradation model,
FIGURE 9. The prediction of power gain A. (a) The prediction result. the 500 degradation value and variance are predicted
(b) Partial enlarged view.
using (28) and (29), where t = 174, k = 500. The
failure thresholds for two degradation features are set to
be 22 and 20 respectively, the prediction results are shown
consists of two parts: one is the linear growth model, the other in Fig. 9 and Fig. 10.
is seasonal factors model, whose period is set to 7. The The proposed sampling approach is used and the PDF of
proposed degradation model is the superposition of the two the fitted distribution for two degradation features are shown
class of model. The observation matrix and state matrix for in Fig. 11.
the bivariate Bayesian dynamic linear model are (11) and (12) The CDF of fitted distributions are used as the marginal
respectively, where s = 7. distribution, several commonly used Copula functions are

16284 VOLUME 5, 2017

F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features

FIGURE 13. The PDF of remaining useful life of power gain A.

FIGURE 11. The distribution of remaining useful life of two degradation
features.

FIGURE 14. The PDF of remaining useful life of power gain B.

FIGURE 12. The PDF of joint distribution function.

used to describe the dependency of RUL for two degradation

features. AIC is used to choose the best Copula function. The
Gumbel Copula is proved to be the best one with the AIC
value of −122.8222, and the PDF of joint distribution func-
tion based on Gumbel Copula is shown in Fig. 12. In addition,
when assuming power gain A and B are independent with
each other, the degradation models for power gain A and B are
established using univariate BDLM, respectively. The RUL
distributions for two features are evaluated using the random
sampling method in this paper. And the machine’s RUL distri- FIGURE 15. The PDF of joint remaining useful life distribution.
bution is obtained based on (37) and (38), as shown in Fig. 12.
It is clearly to see that the independent RUL expectation
is lower than dependent RUL expectation. In other words, new model parameters. In each time, a PDF of remaining
making the independent assumption may be under-estimate useful life for each degradation feature obtained and shown
the system reliability. in Fig. 13 and Fig. 14.
In order to show the changes of remaining useful life of Five Copula families are also used to couple the CDFs
the product, the first 100 predict value of each degradation of two features’ RUL under different monitoring time. And
feature are added to the original data, the degradation model is the AIC is adopted to calculate the goodness-of-fit scores of
reestablished and the parameters are re-estimated. The model different Copula families. On the monitoring time of 174 day
reestablishing is repeated 4 times, the first 100 predict value and 274 day, Gumbel Copula is proved to be the best one
of last model is added to the original data to estimate the with the AIC value of −122.8222 and −165.2588. On the

VOLUME 5, 2017 16285


monitoring time of 374 day, 474 day and 574 day, Gaussian
Copula is selected with the minimum AIC of −13.5743,
−13.3552, and −588.9102. Then, the joint PDFs of the
microwave machine’s RUL distribution at different monitor
point are calculated and illustrated in Fig. 15, which can be
further used for maintenance decision-making.
V. CONCLUSIONS
This paper proposes a multivariate degradation modeling
approach based on BDLM to calculate the RULs of degrada-
tion features. And the Copula method is employed to capture
the dependency among RUL distributions. A practical testing
of the microwave machine with bivariate degradation features
are utilized to validate the method. Some conclusions can be
drawn.
In the proposed method, the dependence of different degra-
dation features is considered both in multivariate degradation
modeling and RUL prediction, which can account for the
dependence sufficiently and effectively. It provides a good
alternative for the degradation modeling with multiple degra-
dation features.
The current study just selects two typical BDLMs, linear
growth model and seasonal factors model. In the future, more
types of BDLM should be investigated.
REFERENCES
[1] M. Pecht, Prognostics and Health Management of Electronics. New York,
NY, USA: Wiley, 2008.
[2] X.-S. Si, W. Wang, C.-H. Hu, and D.-H. Zhou, ‘‘Remaining useful life
estimation—A review on the statistical data driven approaches,’’ Eur. J.
Oper. Res., vol. 213, pp. 1–14, Aug. 2011.
[3] K. Javed, R. Gouriveau, and N. Zerhouni, ‘‘State of the art and taxonomy
of prognostics approaches, trends of prognostics applications and open
issues towards maturity at different technology readiness levels,’’ Mech.
Syst. Signal Process., vol. 94, pp. 214–236, Sep. 2017.
[4] S. Kumar and M. Pecht, ‘‘Modeling approaches for prognostics and health
management of electronics,’’ Int. J. Performability Eng., vol. 6, no. 5,
pp. 467–476, Sep. 2010.
[5] J. Lee, F. Wu, W. Zhao, M. Ghaffari, L. Liao, and D. Siegel, ‘‘Prognostics
and health management design for rotary machinery systems-reviews,
methodology and applications,’’ Mech. Syst. Signal Process., vol. 42,
pp. 314–334, Jan. 2014.
[6] L. M. M. Lima, E. Popova, and P. Damien, ‘‘Modeling and forecasting of
Brazilian reservoir inflows via dynamic linear models,’’ Int. J. Forecasting,
vol. 30, no. 3, pp. 464–476, 2014.
[7] S. Zhang and W. Zhou, ‘‘Bayesian dynamic linear model for growth of
corrosion defects on energy pipelines,’’ Rel. Eng., Syst. Safety, vol. 128,
pp. 24–31, 2014.
[8] S. Bouaddi, A. Ihlal, and A. Fernández-García, ‘‘Soiled CSP solar reflec-
tors modeling using dynamic linear models,’’ Solar Energy, vol. 122,
pp. 847–863, Dec. 2015.
[9] M. West and J. Harrison, Bayesian Forecasting and Dynamic Models,
2nd ed. New York, NY, USA: Springer-Verlag, 1997.
[10] K. Triantafyllopoulos, ‘‘Multivariate stochastic volatility with Bayesian
dynamic linear models,’’ J. Stat. Planning Inference, vol. 138, no. 4,
pp. 1021–1037, Apr. 2008.
[11] E. H. Lau, C. K. Cheng, D. K. Ip, and B. J. Cowling, ‘‘Situational awareness
of influenza activity based on multiple streams of surveillance data using
multivariate dynamic linear model,’’ PLoS One, vol. 7, no. 5, p. e38346,
2012.
[12] H.-S. Lee, Y.-C. Shih, H.-M. Wang, and S.-K. Jeng, ‘‘Subspace-based
phonotactic language recognition using multivariate dynamic linear mod-
els,’’ in Proc. 38th IEEE Int. Conf. Acoust., Speech, Signal Pro-
cess., (ICASSP), Vancouver, BC, Canada, May 2013, pp. 6870–6874.
[13] S. Lu, H. Lu, and W. J. Kolarik, ‘‘Multivariate performance reliability pre-
diction in real-time,’’ Rel. Eng., Syst. Safety, vol. 72, pp. 39–45, Apr. 2001.
16286
F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features
[14] J. Zhang, X. Ma, and Y. Zhao, ‘‘A stress-strength time-varying correlation
interference model for structural reliability analysis using copulas,’’ IEEE
Trans. Rel., vol. 66, no. 2, pp. 351–365, Jun. 2017.
[15] X. Zhang and A. Wilson, ‘‘System reliability and component importance
under dependence: A copula approach,’’ Technometrics, vol. 59, no. 2,
pp. 215–224, 2017.
[16] F. Sun, L. Liu, X. Li, and H. Liao, ‘‘Stochastic modeling and analysis of
multiple nonlinear accelerated degradation processes through information
fusion,’’ Sensors, vol. 16, no. 8, p. 1242, Aug. 2016.
[17] W. Peng, Y.-F. Li, Y.-J. Yang, S.-P. Zhu, and H.-Z. Huang, ‘‘Bivariate
analysis of incomplete degradation observations based on inverse Gaussian
processes and copulas,’’ IEEE Trans. Rel., vol. 65, no. 2, pp. 624–639,
Jun. 2016.
[18] Y. Wang and H. Pham, ‘‘Modeling the dependent competing risks with
multiple degradation processes and random shock using time-varying cop-
ulas,’’ IEEE Trans. Rel., vol. 61, no. 1, pp. 13–22, Mar. 2012.
[19] S. K. Gupta, S. De, and A. Chatterjee, ‘‘Some reliability issues for
incomplete two-dimensional warranty claims data,’’ Rel. Eng., Syst. Safety,
vol. 157, pp. 64–77, Jan. 2017.
[20] Z. Xi, R. Jing, P. Wang, and C. Hu, ‘‘A copula-based sampling method
for data-driven prognostics,’’ Rel. Eng., Syst. Safety, vol. 132, pp. 72–82,
Dec. 2014.
[21] H. Joe, Multivariate Models and Multivariate Dependence Concepts,
1st ed. Boca Raton, FL, USA: CRC Press, 1997.
[22] X. Wang, N. Balakrishnan, and B. Guo, ‘‘Residual life estimation based on
nonlinear-multivariate Wiener processes,’’ J. Stat. Comput. Simul., vol. 85,
pp. 1742–1764, Jun. 2015.
[23] X. Fei, C. C. Lu, and K. Liu, ‘‘A Bayesian dynamic linear model approach
for real-time short-term freeway travel time prediction,’’ Transp. Res. C-
Emerg. Technol., vol. 19, pp. 1306–1318, Dec. 2011.
[24] G. Petris, S. Petrone, and P. Campagnoli, Dynamic Linear Models With R.
New York, NY, USA: Springer-Verlag, 2009.
[25] X. Xu and N. Chen, ‘‘A state-space-based prognostics model for lithium-
ion battery degradation,’’ Rel. Eng., Syst. Safety, vol. 159, pp. 47–57,
Mar. 2017.
[26] R. Prado and M. West, Time Series Modeling, Computation, and Inference.
New York, NY, USA: Taylor & Francis, 2010.
[27] A. W. Bowman and A. Azzalini, Applied Smoothing Techniques for Data
Analysis, 2nd ed. New York, NY, USA: Oxford Univ. Press, 1997.
[28] A. Sklar, Fonctions de Répartition à n Dimensions et Leurs Marges, vol. 8.
l’Institut de Statistique de L’Université de Paris, 1959, pp. 229–231.
[29] R. B. Nelsen, An Introduction to Copulas, 2nd ed. New York, NY, USA:
Springer, 2007.
FUQIANG SUN received the Ph.D. degree in
systems engineering from Beihang University,
Beijing, China, in 2012, where he is currently an
Assistant Professor with the Science and Technol-
ogy on Reliability and Environmental Engineer-
ing Laboratory, School of Reliability and Systems
Engineering. His research interests include accel-
erated degradation/life testing, prognostics, com-
peting risk, and multivariate statistical analysis.
NING WANG received the B.S. degree in product
quality engineering from China Jiliang University
in 2015. He is currently pursuing the master’s
degree with the Science and Technology on Relia-
bility and Environmental Engineering Laboratory,
School of Reliability and Systems Engineering,
Beihang University. His research interests focus on
life prediction and accelerated test modeling.
VOLUME 5, 2017
F. Sun et al.: RUL Prediction for a Machine With Multiple Dependent Features
XIAOYANG LI received the Ph.D. degree in
aerospace systems engineering from Beihang Uni-
versity, Beijing, China, in 2007. She is currently an
Associate Professor with the School of Reliabil-
ity and Systems Engineering, Beihang University.
Her research interests include life modeling and
optimal test design for accelerated testing, and
quantification method of uncertainty.
VOLUME 5, 2017
View publication stats
WEI ZHANG received the Ph.D. degree in engi-
neering from Clarkson University in 2012. He was
with Arizona State University as an Assistant
Research Scientist. He is currently an Associate
Professor of reliability and systems engineer-
ing with Beihang University. His research areas
include fatigue and fracture mechanics, structure
reliability, and life prediction.
16287