See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/313735176

The State of the Art of Hidden Markov Models for Predictive Maintenance of
Diesel Engines: HMM for predictive maintenance of diesel engines

Article  in  Quality and Reliability Engineering · January 2017

DOI: 10.1002/qre.2130

CITATION READS

1 990

4 authors:

António Simões Jose Viegas

Instituto Politécnico de Coimbra International Transport Forum, Paris
7 PUBLICATIONS   10 CITATIONS    83 PUBLICATIONS   1,318 CITATIONS   

SEE PROFILE SEE PROFILE

José Torres Farinha Inácio Fonseca

University of Coimbra Instituto Politécnico de Coimbra
59 PUBLICATIONS   111 CITATIONS    40 PUBLICATIONS   133 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

MSc Dissertation View project

Time replacement optimization models for urban transportation buses with indexation to fleet reserve View project

All content following this page was uploaded by António Simões on 09 May 2017.

The user has requested enhancement of the downloaded file.

Case Study
(wileyonlinelibrary.com) DOI: 10.1002/qre.2130 Published online in Wiley Online Library

The State of the Art of Hidden Markov Models

for Predictive Maintenance of Diesel Engines
António Simões,a,d*† José Manuel Viegas,b José Torres Farinhaa,d and
Inácio Fonsecac,d
The maintenance of diesel Engines is usually scheduled according to the maintenance procedures defined by manufacturers.
However, the state of the art shows that the condition monitoring maintenance associated with adequate prediction
algorithms allows performance improvement both by increasing the intervals between interventions and by helping to
maintain reliability levels. There are many types of variables that can be used to measure equipment condition, as is the case
of several types of pollutant emissions such as NOx, CO2, HC, PM, and NOISE, among others. This is a typical problem that can
be solved through a hidden Markov model, taking into account the specificity of this type of equipment. The paper describes
two algorithms that can help to increase the quality of assessment of engine states and the efficiency of maintenance
planning. Those are the Viterbi and Baum–Welch algorithms. The importance of how to calculate the performance index
of the model by the use of the perplexity algorithm is also emphasized. In this paper, a new paradigm is proposed,
designated as ecological predictive maintenance. Copyright © 2017 John Wiley & Sons, Ltd.

Keywords: ecological predictive maintenance; diesel engines; effluents; hidden Markov model

1. Introduction
he vehicles maintenance unit is a cost center in many companies, but it also is a center of performance improvement of the

T organization. The most common maintenance strategies for diesel engines are the periodic or scheduled ones, although quite
often there is no planned maintenance. Predictive maintenance supported on condition based maintenance has its references
on the natural world.
Production systems suffer degradation before the occurrence of dysfunctions. This approach as a basis for maintenance is still
quite incipient in the case of diesel engines.
Maintenance actions should not be launched only following a fault in the equipment or process, but rather by an intelligent
decision based on the foreseen degradation.1
Predictive maintenance is growing with the objective of extending equipment life cycle. The limited use of control variables (of
which the oil properties are the most common) to infer the system state has restricted the range of results. Other variables are not
common because of the complexity analyses, as is the case of effluents, although their potential is recognized for increasing
maintenance intervals and helping guarantee equipment reliability. With the use of control variables aligned with an ecological
perspective, we believe that a new frontier is being opened.
The main aspects reported in this paper are the following ones: Why to use ecological predictive maintenance? What is the
additional value in using hidden Markov model (HMM)? Which is the visible observations sequence that best explains the states?
Which is the option that most likely produces the best states sequence, given a set of multiple HMM and the real observations
sequence? How to update the model parameters in recurrent use, in the sense of best fit to the new observations? How to evaluate
the perplexity index?2–4

a
Department of Mechanical Engineering, Polytechnic Institute of Coimbra/Coimbra Institute of Engineering, Rua Pedro Nunes, Quinta da Nora3030-199, Coimbra,
Portugal
b
Department of Civil Engineering, Architecture and Geo-resources, University of Lisbon/Instituto Superior Técnico, Avenida Rovisco Pais1049-001, Lisbon, Portugal
c
Department of Electrical Engineering, Polytechnic Institute of Coimbra/Coimbra Institute of Engineering, Rua Pedro Nunes, Quinta da Nora3030-199, Coimbra,
Portugal
d
CEMUC – Centre for Mechanical Engineering of the University of Coimbra, Pinhal de Marrocos3030-788, Coimbra, Portugal
*Correspondence to: António Simões, Polytechnic Institute of Coimbra/Coimbra Institute of Engineering, Department of Mechanical Engineering, Rua Pedro Nunes,
Quinta da Nora, 3030-199 Coimbra, Portugal.
†
E-mail: assimoes@isec.pt

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
 A. SIMÕES ET AL.

2. State of the art

The detective maintenance or maintenance intelligent system brings together the prediction and prevention options. It is based on a
continuous measurement, instead of a sporadic or periodic one, through the use of embedded equipment. The data from sensors are
processed using degradation models and intelligent knowledge, for the estimation of the present system condition, allowing the
prediction of its future state – for example, Watchdog Agent.5
Maintenance, in general, and predictive maintenance, in particular, try to combine improvements of equipment reliability with a
reduction of its costs, both direct and indirect.6
The conventional equipment’s health indicators, used as condition monitoring indicators, are not always managed having into
account the environmental impacts. This innovative approach of condition monitoring maintenance, presented in this paper, is based
on variables that are managed having into account the environment and, because of this, are called ecological variables.
The evaluation of the health condition of the equipment based on the spectrum of emissions of a vehicle requires the knowledge
of the operating regime of the engine (torque and speed). However, the operating regime can be estimated through the knowledge
of the vehicle dynamics.
Vehicle specific power7 is a very useful variable for the estimation of engine operation conditions, and it is defined as the
instantaneous power per unit of vehicle mass, being calculated by the following formula8:
d
ðKE þ PE Þ þ F R v þ F L v
VSP ¼ dt (1)
m
where
VSP = vehicle specific power (kW/ton)
KE = kinetic energy
PE = potential energy
FR = rolling resistance force
FL = aerodynamic drag force
v = vehicle speed
m = vehicle mass

European emission standards define the acceptable limits for exhaust emissions of new vehicles sold in European Union and
European Economic Area member states. The emission standards are defined in a series of European Union directives staging the
progressive introduction of increasingly stringent standards. The standard lies between the levels of Euro 0 and Euro VI.
Until 1992, the industry was regulated by the Euro 0. It was replaced by Euro I, which dealt solely with NOx, HC, and CO. In 1996,
Euro II saw the first adoption of a particulate matter limit, and all engines were required to meet the same limit levels. Euro III came in
2001. In 2006, with Euro IV, after treatment was required for the first time, using either selective catalytic reduction or exhaust gas
recirculation (EGR), and on-board diagnostics required to monitor emission control.
With EURO IV application, the heavy vehicles had suffered reductions of pollutant emissions. This mitigation was of such order that
it ensured the compliance with emission standards related to CO and HC levels (this did not happen with the emissions of PM10 and
NOx) proposed by EURO V, without the use of any device of additional post-treatment.
In 2008, when EURO V emission standards were introduced; the concerns were focused on control, not only of the particulate
matter pollution that was 10 μm in diameter or less, called PM10, but also of NOx levels.9
Then, in 2014, the Euro VI came into effect, with the tightest emissions levels to date. In meeting the regulations, European
automotive original equipment manufacturers have resorted to both selective catalytic reduction and EGR, combined with a diesel
particulate filter. The way how these technologies were adopted and integrated will give some indication as to whether original
equipment manufacturers will have an agreement to Euro VII targets.
In the study of Brahma,10 an important evaluation of the technological innovations is accomplished. It describes that, during the
last decade, modern technologies like direct injection, EGR, variable geometry turbocharger, and the fuel injection of high pressure
common rail have been reducing the difference between diesel engines and spark ignition engines in terms of environmental
impacts.
The most recent great developments in diesel engines were focused on the development of new piezo-electric injectors and
higher injection pressures which, in conjunction, explain the decrease of consumption by up to 10–15%.
More recently, great developments appeared in the area of petrol engines with a new generation of phase variators. Toyota
Valvematic system is very similar to BMW’s Valvetronic, allowing reductions from 10% to 26% in consumption and emissions and,
at the same time, a power increase between 14% and 20%.
NOISE represents one of the negative externalities of transports activities. One of the recommendations of International
Association of Public Transport (UITP) refers that lower noise levels can be achieved through regular maintenance of the fleets, use
of silent technologies, incorporation of absorbent materials, and application of restriction measures to the use of private vehicles.11
This is the background for the incorporation of noise in our mathematical model.
The methodology and technology for measuring emissions include the following: immobilized vehicle, with engine running
without load; engine disconnected of the vehicle, mounted on a test bench and connected to a dynamometer; vehicle placed on
a chassis dynamometer equipped with means of load and inertia simulation, which is used in torque and power measurement;

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
A. SIMÕES ET AL.

emissions monitoring technique in roadway, also named as remote sensing device, used as pollutants detection; and vehicle with
on-board diagnosis systems. The procedures in the tests are standardized or recommended.12
It is referred that the maintenance pattern can generally influence the inter-variability between vehicles emissions.13 This study
leads to the following conclusions: when the repairs carried out are not in consonance with the manufacturers’ specifications, the
emissions are high; when the service is not executed in agreement with the manufacturers’ plans leads to engine and after-treatment
systems deteriorations that create high levels of pollutant emissions.
A Drive Clean Program study14 refers that the curve of cumulative distribution function of rate of faults, which is no more than the
accumulation of the probability density function of rate of flaws with the fleet age, can be presented in the sigmoidal form (the shape
of the letter S).
Among traditional condition indicators, vibration appears in the vanguard. In a case study,15 a model that identifies, monitors, and
improves economic impact of the maintenance based on the vibration is presented. This model supplies an additional possibility to
identify “where,” “why,” and “how much” capital should be invested.
An intelligent maintenance management system must have functions that allow answering the following questions: Where is the
dysfunction symptom located (component or sub-system)? What is the cause of the dysfunction symptom? What time does mediate
until the occurrence of critical operation states? What are the consequences of the dysfunction? What is the recommended
maintenance action?
Several prediction and recognition models are available. The most used are the HMM and the artificial neural network. The first one
corresponds to the option presented in this paper, because it allows the perfect adjustment of the model to the real emissions vehicle
behavior. The artificial neural network has been demonstrating an enormous potential but only in specific applications.
In addition to the individual analysis of the variables, in the evaluation of the effluents with environmental impact, an
environmental function can be used, integrating all impacts, as it can be seen in the formula hereafter:

  




m_ OPAC m _ NOx  m_ CO2 m_ NOISE m_ CO m_ HC
EF ¼ a þ b þ c þ d þ e þ f (2)
_ OPACref
m _ NOxref 
m _ CO2 ref
m _ NOISEref
m _ COref
m _ HCref
m

The World Health Organization combines the whole impact of the gases emission in what it designates as SCORE, balancing them
according to the their influence in the health of the populations, through the following equation:
CO
SCORE ¼ 20PM þ NOx þ HC þ (3)
10
The best maintenance practices in urban public transport operators are not very innovative: The Federal Transit Administration
currently recommends a 20% spare ratio as the optimum spare factor – this number is specified in the Circular C 9030 1A, issued
in 1987.
Federal Transit Administration made a survey in a group of 36 North American and Canadian companies. The results are compiled
by the author4 in Table I:
The Coimbra Municipality Urban Transport Services (SMTUC) is classified in the group of the small companies. The company
includes in its management panel a group of indicators compiled by the author4 as specified in Table II:
Table II shows the contrast between SMTUC and North American and Canadian reference practices.

3. Ecological predictive maintenance

The subject of this paper is to apply the concept of HMM in condition monitoring maintenance of diesel engines. Condition
monitoring maintenance relies on the capability to make interventions before any control variable reaches its limit value.
Ecological predictive maintenance is based on ecological indicators.
The maintenance of diesel engines through condition monitoring maintenance methodologies allows planning more efficient
timing of interventions, based on the following condition variables: NOx, PM10, CO2 and NOISE. In planning maintenance with such
variables, a HMM has been developed. This model has shown to be adequate to manage the complexity associated to those variables.
The model uses the Baum–Welch and the Viterbi algorithms.16

Table I. Macro exploitation indicators

Number of Spare ratio Velocity Km/ Average fleet age
Class of company companies (%) (km/h) (year.bus) (years)
Small 8 19 20.0 54768 7.7
Medium 8 22 20.6 59901 8.1
Large 12 21 20.9 65026 8.0
Very large 8 17 19.3 62771 9.3
All 36 20 20.3 61141 8.3

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
 A. SIMÕES ET AL.

Table II. Management indicators in SMTUC

Year Total active fleet Supply (Medium number of buses) Downtime percentage (*) Average fleet age (years)
2009 108 83 3.2 10.87
2010 111 83 3.2 11.12
*Measures downtime due to maintenance and reactionary repair, excluding accidents.

It is important to detect easily and economically the indicators values that are strongly correlated to the levels of physical
degradation of the system (engine and vehicle). The state indicators values of the urban buses are picked up through a diagnosis
connector.
The state degradation curve of a vehicle is an important tool to evaluate its condition. Many failure modes show signs of warning
(symptoms) as they are about to occur. If, during an inspection, the organizational structure in charge of maintenance finds evidence
that any of the vehicle systems is reaching the end of its life, then it is possible to intervene promptly and to avoid the failure.
In the state degradation curve of a vehicle, the Weibull method is the most common statistical way to describe the fault behavior
(to estimate the P–F interval, i.e., the time interval between the point of detection of symptoms (P) and the point of occurrence of
failure (F)). Figure 1 shows a curve illustrating such equipment behavior.
The curve shows that when a failure starts appearing, the equipment state suffers a deterioration until the point at which it is
possible to detect the corresponding symptoms (P). Using the current detection technology, P is the initial point at which an evolving
failure can be observed. If that small failure is not detected and mitigated, deterioration continues until a serious failure occurs (F). The
time range between P and F, commonly called the P–F interval, is the window of opportunity during which an inspection can possibly
detect the imminent failure and address.17,18 P–F intervals can be measured in any unit associated to the exposure to stress (running
time, miles, cycles, pollutant emissions, kilometers, etc.).
The functional failures that have multiple symptoms of the impending failure offer a great deal of flexibility in the choice of tasks to
perform for detecting the impending failure. In this study, the first and principal symptom may be the pollutant emissions level,
detected by inspection and monitoring around eight weeks before failure. The second symptom may be unusual vibration, detectable
by vibration analysis, around 6 weeks before failure. The third symptom may be sound detectable at 4 weeks before failure using
airborne ultrasound. The fourth symptom may be increased heat detectable at 2 weeks prior to failure. According to Figure 2, it
can be seen that the first symptom in P–F interval tends to last at least 6 weeks, because that the point P, corresponding to the

Figure 1. State degradation curve of a vehicle (P–F interval)

Figure 2. Inspection intervals and dysfunction delay

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
A. SIMÕES ET AL.

detection of the environmental dysfunction, cannot be detected in the first inspection, occurred 8 weeks before failure. The most
unfavorable situation happens around 2 weeks before the beginning of the dysfunction.
A cause–effect assessment to different sub-systems of heavy vehicles with a compression ignition engine, using diesel as fuel,
allows establishing the qualitative associations specified in Table III.
With the aim of eliminating the dysfunctions detected in engines, according to the degradation state in which correction happens,
different intervention levels can be defined, as described in Figure 3.
Typical engine repairs associated to high levels of soot and/or high-pollutant gaseous emissions, occurred during 2 years in
SMTUC, with Mercedes CITARO O530 series, are compiled in Table IV.
There are several studies evaluating the impacts of transport on environmental pollution and fuel consumption.19–21
The variation levels of carbon dioxide emissions, related to the circulation speed, in vehicles of small, average, and large
dimensions, can be seen in Figure 4.21
It is possible to infer that the minimum value of emissions per distance unit occurs for speeds between 64 and 84 km/h. This
analysis shows the importance of assuring good traffic flowing in urban transport to minimize the environmental impact.
Under this context, a new paradigm of maintenance planning for diesel engines is proposed. This new paradigm takes as reference
the attributes of environmental impact instead of miles or hours of functioning. These are the bases from which the ecological
predictive maintenance emerges. The tool that supports this concept detects symptoms of dysfunctions before the control variables
reach their limits, based on environmental impacts, as can be seen in Figure 5.22
The monitoring of the environmental indicators can aid the detection of pieces with waste, bronzes, or bearings deteriorated or
with defect, clearances in mechanical fits or in the gears, or still occurrences of fractures.2
In Figure 6, the ecological predictive maintenance is supported on the correlation between ecological indicators that are visible
indicators and the degradation state that corresponds to hidden indicators. These correlations are identified in Figure 6. In the
drawing, environmental visible indicators are represented by dashed arrow, and continuous evolution of the state is represented
by solid arrow.

4. Hidden Markov models

Following this approach, a new set of forecasting algorithms should be used. One of them is the Markov chain and, in particular, the
hidden Markov chain.16,23
Markov process models are a class of probability models used to study the evolution of a stochastic system over time. Transition
probabilities are used to identify how a system evolves from one period of time to the next one. In our case, a Markov chain tries to
characterize the system behavior over time, as described by the transition probabilities matrix, emissions matrix, and the initial state
probability.

Table III. More common faults in internal combustion engines

Part of the engine Nature of Failure Pollution

FI – Fuel injection Fuel injected High

Injectors Fuel injected High
Fuel pumps Low pressure Low
EGR – Exhaust Gas Recirculation Enrich mixture NOx High
Intake manifold and air filter Blockades High
Turbocharger Performance High
Valves Deterioration High
Injection Point Delayed or advanced High
Timing Belts and Chains Valves High
Oil leaks Rupture or waste High
Water pump/ducts Heating Medium
Fan Heating Medium
Thermostat Heating Medium
Bronze bushes/Bearings Heating Medium
Electrical and electronic components or sensors Several dysfunctions Low to High
AIR – Air injection Lack of secondary air Medium
CAT – Catalyst or particulate filter Aging or rupture Low to High
Clutch, gearbox, differential, axes Waste or rupture Medium
Maxillaries and disks Squeezes Low to High
Pressure pump of assisted steering Effort Low
Links Effort Low
FI, fuel injection; EGR, exhaust gas recirculation; AIR, air injection; CAT, catalyst or particulate filter.

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
 A. SIMÕES ET AL.

Figure 3. Levels of intervention

Table IV. Repairs of bus engines with impacts in the emissions (Mercedes CITARO O530 series)
Corrective repair type Percentage of dysfunctions
Control of the fuel/air relation 1.4
Air filter/gasoil filters 7.8
Diesel injection pump adjustments 4.9
Components of Diesel injection pump 2.8
Valvetrain system adjustments 3.5
Components of valvetrain system 4.9
Components of injectors/port-injectors 8.5
Engine head revision 2.1
Valves adjustments 3.5
Turbocharger 6.3
Cooling system 26.1
Lubrication system 11.3
Engine revision 13.4
Engine rebuilding 3.5

Figure 4. Carbon dioxide emissions versus speed

In order to be possible to represent a stochastic process through a HMM, the following requisites must be satisfied:
• The transition probability of present state to any other state is independent of how the process reached its current state (no
memory). So, for computation purposes, the only necessary state is the previous one;
• The “true” states are hidden, that is, not directly observable;
• The observable variables (emissions, in this case) are probabilistic functions of the hidden states;
• The hidden states behave as a first order Markov model.
An HMM is an extension of a first order Markov model. It consists of two sets of correlated variables (states and emissions in this
paper) and three sets of probabilities (Figure 7):
• The hidden states – the true states of a system – may be described by a Markov process;
• The observable variables (emissions). These are the indicators, the “visible face” of the process;
• The initial probabilities, for hidden states;

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
A. SIMÕES ET AL.

Figure 5. Evolution of the degradation of the system

Figure 6. Correlation between ecological indicators and state of degradation of the system

Figure 7. Connections in a hidden Markov model of five states and four emissions classes

• The transition probabilities matrix, for hidden states;

• The model “state-symbol” probability matrix, where each element represents the probability of outputting an emission symbol,
having into account that the model is in a specified state. It is the observation probabilities matrix. Symbols are combinations of
observed emissions. In this study, symbols are emissions spectra.
The model integrates an initial state probability vector whose elements represent the probabilities of the system to start in
different states (or classes).
In defining the classes, it becomes necessary to adopt a set of limits, international standards, environmental specifications, or any
other norms, according to each specific situation. In fact, the emission of any diesel engine vehicle is a random process, depending on
many variables, such as the driver behavior, exploitation conditions, and temperature.

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
 A. SIMÕES ET AL.

Usually, the system is planned to make cycles between interventions, beginning at the zero instant (usually when the vehicle is
new). It evolves along the time with the on-duty service, passing from several maintenance interventions and states. Several
maintenance interventions and states are identified.
The symbols in Figure 7 represent possible emissions classes or emissions spectrums or combinations of environmental impacts
associated to different health states.
In principle, it would be advantageous to be able to use HMM with continuous observation densities.
Having as objective the use of a continuous observation density, some restrictions have to be placed in the form of the
model probability density function in order to ensure that the parameters of such a function can be re-estimated in a
consistent way.
The continuous HMM, as the discrete one, has similar properties:
• The residence time in a state (memoryless process);
• The next state only is affected by the instant of transition and only depends on the current state and not on how it got there.
However, the following two reasons do not permit the validation of a continuous HMM conditions:
1) The residence time in each state does not necessarily follow an exponential distribution. In the context of this paper, it depends
on acceleration, weather, and many variables that continuously change;
2) Currently, it is not economical, in practice, to continuously measure the emissions. Only intermittent sampling was used.

4.1. Emissions and HMM states

In synthesis, in running the model, it is necessary to create the following matrixes:
• Q  {q1…qN} – set of values for hidden states, designated as states vector or states library;
• V  {V1…VM} – set of values (symbols) possible in observations, designated as observations vector or emissions library.
The followings sets represent a generic library of states and emissions spectra in the ambit of diesel buses:
• A = {aij} – states transition probabilities matrix. Each element of matrix A must be computed. The procedure is as follows:

  number of transitions from qi to qj

aij ¼ P Stþ1 ¼ qj jSt ¼ qi ¼ (4)
number of times in state qi
• B = {bik} – states observation per symbol probabilities matrix designated as emissions matrix.

number of times in state qj with the symbol v k

bj ðk Þ ¼ (5)
number of times in state qj

Figure 8 shows the temporal sequence for each one of the described matrixes, transition matrix, and emissions matrix.
• ∏ = {∏i} – initial state probability matrix. It is the states vector probability in the instant t = 1.

π i ¼ P½S1 ¼ qi ; 1≤i≤N (6)

In the beginning of a life cycle (t = 1), the equipment is assumed as new or just after a (deep) maintenance operation.
For several cycles,
Number of times in state qi at time t ¼ 1
πi ¼ (7)
Number of times in all states at time t ¼ 1

Figure 8. Temporal sequence in a hidden Markov model

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
A. SIMÕES ET AL.

Next states evolve until the limits imposed by international standards,24 environmental rules, and/or by other requisites of each
specific situation.
The HMM model is given by

fQ; V; ∏; A; Bg (8)

Q corresponds to a library of states. States observed or predicted in different instants (t1, t2, t3, t4 ……tT) are represented by St. On
the other hand V corresponds to a library of emissions or symbols. Observed symbols in different instants (t1, t2, t3, t4 ……tT) are
represented by Ot.
The HMM implementation is supported by the following steps, described here at conceptual level:
1. Using the forward–backward algorithm, the probability of the observations sequence is computed;
2. With the observations sequence, the most likely hidden states sequence are computed;
3. With an observations sequence and on a set of possible models, it is possible to answer the question: “which is the model that
most closely fits the data?”
Given an observations sequence and a model, the probability of the observations sequence is given by

O ¼ ðo1 :::oT Þ; λ ¼ ðA; B; ΠÞ (9)

Then, it is necessary to compute the probability of the observations sequence, given the model, P(O| λ). The objective of doing this
is to consider that the probability of an observations sequence is the sum of the probabilities of all possible states sequences that
converge to that state. However, naive computation is very expensive, because given T observations and N states, there are NT
possible states sequences, an excessive number even for a small HMM, for example, T = 10 and N = 10.
A possible solution for this problem consists of using dynamic programming. This can be accomplished by the following steps:

PðOjS; λÞ ¼ bs1 o1 bs2 o2 :::bsT oT (10)

PðSjλÞ ¼ π s1 as1 s2 as2 s3 :::asT1 sT (11)

PðOjλÞ ¼ ∑ PðOjS; λÞPðSjλÞ (12)

S

PðO; SjλÞ ¼ PðOjS; λÞPðSjλÞ (13)

That can be condensed as

T1
PðOjλÞ ¼ ∑ π s1 bs1 o1 Π ast stþ1 bstþ1 otþ1 (14)
t¼1
fS1 :::ST g

Equation (14) is used in the forward–backward procedure, as shown in the next section.
The HMM has several unsolved problems that allow several solutions, according to the following procedures:
I Evaluation:
• Problem:
• There is a set of possible HMM and a given number of observations. Which HMM has most probably to generate the given
sequence?
• Solution:
• Compute the probability of the observed sequences, for each HMM;
• Choose the maximum likelihood HMM (the one that corresponds to the maximum likelihood of the observed sequences);
• Use the forward algorithm to reduce the complexity.
II Decoding:
• Problem:
• Given a particular HMM and an observations sequence, which is the most likely sequence of underlying hidden states that
might have generated this sequence of observations?
• Solution:
• Compute the probability of the observed sequences for each possible sequence of underlying hidden states;

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
 A. SIMÕES ET AL.

• Choose the one with the highest probability;

• Use the Viterbi algorithm to reduce complexity.
III Learning:
• Problem:
• Estimate the probabilities of HMM from training data;
• Solution:
• Train with labeled data;

   
Transition probability P qi ; qj ¼ number of transitions from qi to qj =ðtotal number of transitions of qi Þ: (15)

Observations probabilities matrix Pðqi ; V Þ ¼ ðnumber of symbol V occurrences in state qi Þ=ðnumber of all symbol occurrences in state qi Þ:
(16)
• Train with unlabeled data;
• Baum–Welch algorithm basic having in target:
• To maximize the probability of the observations sequence, given the model;
• To estimate new probability from the previous HMM, until [P(current HMM)  P(previous HMM) < e (where “ e ” is a small
number)].
In what concerns labeled and unlabeled data, the next sections deal with this problem.

4.2. Forward–backward algorithm

The prediction of hidden states is an extremely complex problem, which implies the implementation of a simple algorithm to solve it
efficiently.16 A possible and efficient solution consists of using dynamic programming. The first steps are the following:
• Intuition – that is, the probability of the first t observations is the same for all possible t + 1 length state sequences. This can be
defined by the next equation, which is the basis of the forward procedure.

αt ði Þ ¼ PðO1 ; O2 ; O3 ; ::::::; Ot ; St ¼ qi jλÞ (17)

The calculation of αt(i) is achieved by the following steps:

α1 ði Þ ¼ π i bi ðO1 Þ; 1≤i≤N (18)

   N   
αtþ1 ðj Þ ¼ P O1 ; ::::::; Ot ; Otþ1 ; Stþ1 ¼ qj λ ¼ ∑ P O1 ; ::::; Ot ; Otþ1 ; Stþ1 ¼ qj St ¼ qi ; λ :P½St ¼ qi jλ ¼
i¼1
N   
∑ P½O1 ; O2 ::::; Ot jSt ¼ qi ; λ:P½St ¼ qi jλ:P Otþ1 ; Stþ1 ¼ qj St ¼ qi ; λ : ¼
i¼1
(19)
N     
∑ P½O1 ; ::::; Ot ; St ¼ qi jλ:P Otþ1 jStþ1 ¼ qj ; St ¼ qi ; λ :P Stþ1 ¼ qj St ¼ qi ; λ
i¼1
N

αtþ1 ðj Þ ¼ bj ðOtþ1 Þ ∑ αt ði Þaij

i¼1

αtþ1 ðjÞ ¼ ∑ αt ðiÞaij bj ðOtþ1 Þ; 1≤t≤T  1 and 1≤ j≤N (20)

i¼1

The final step for decoding the solution is as follows:

N
PðOjλÞ ¼ ∑ αT ði Þ (21)
i¼1

The next step is the backward procedure, which allows the computation of the probability of observations sequence, given a state
at instant t. This can be carried out according to the following equations:

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
A. SIMÕES ET AL.

βt ði Þ ¼ PðOtþ1 ; Otþ2 ; Otþ3 ; ::::::; OT jSt ¼ qi ; λÞ (22)

Initialization:

βT ðiÞ ¼ 1; 1≤ i ≤N (23)

Induction:
N
βt ði Þ ¼ ∑ aij bj ðOtþ1 Þβtþ1 ðj Þ; t ¼ T  1; T  2; :::1 and 1≤ i ≤N (24)
j¼1

Final step:
N
PðOjλÞ ¼ ∑ π i bi ðO1 Þβ1 ðiÞ (25)
i¼1

4.3. Viterbi algorithm

Completing the forward procedure, it is necessary to decode the most probable sequence of occult states, that is, the one which best
explains the certain observations sequence. This can be carried out by the Viterbi algorithm, synthesized by next equation:

S ¼ arg máx PðS; OjλÞ (26)

S

In Equation (26) “arg máx” is the argument maximum of all previous states.
In evaluating this equation, an auxiliary variable δt(i) is defined, which corresponds to the maximum probability of the recorded
observations sequence, assuming that the final state is inserted in the qi class:

δt ðiÞ ¼ máx P½S1 S2 S3 ::::::St1 ; St ¼ qi ; Ο1 O2 O3 ::::::Ot jλÞ (27)

S1 ;S2 ;S3 ; ::::St1

The algorithm may be compactly stated by the following:

• Initialization:

δ1 ðiÞ ¼ π i bi ðO1 Þ; 1≤i≤N (28)

ψ 1 ði Þ ¼ 0 vvv (29)

• Recursive computation:
 
δt ðj Þ ¼ máx δt1 ðiÞaij bj ðOt Þ; 2≤t≤T e 1≤j≤N (30)
1≤i≤ N

 
ψ t ðj Þ ¼ arg máx δt1 ðiÞaij ; 2≤t≤T e1 ≤ j≤N (31)
1≤i≤N

• Termination:

P ¼ máx ½δT ðiÞ (32)

1≤i≤N

ST  ¼ arg máx ½δT ði Þ (33)

1≤i≤N

The next step consists of computing the most likely states sequence by backtracking:
h i
St  ¼ arg máx δt ði Þaiqtþ1 (34)
1≤i≤N

 
St  ¼ ψ tþ1 Stþ1 ; t ¼ T  2; T  3; :::::; 1 (35)

The result is the generation of the following sequence:

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
 A. SIMÕES ET AL.

S1 ; S2 ; S3 ::::::St1 ; ST (36)

4.4. Baum–Welch algorithm

This method is used to calibrate the model parameters. The aim is to maximize the probability of the observations sequence. This is
the final step to completely define the HMM:
• Given an observations sequence, which is the model that most likely produces that sequence?
Given a model and an observations sequence, the algorithm updates the model parameters, which best match with the
observations.
ξ t(i, j) is defined as the probability of the system to be in state qi at time t and in state qj at time t + 1.

αt ði Þaij bj ðOtþ1 Þβtþ1 ðjÞ

ξ t ði; j Þ ¼ N N
(37)
∑ ∑ αt ðiÞaij bj ðOtþ1 Þβtþ1 ðj Þ
i¼1 j¼1

The next equation gives the probability of the system being in state i at time t.
N
γt ðiÞ ¼ ∑ ξ t ði; jÞ t ¼ 1…:T (38)
j¼1

Now, it is possible to compute new estimates for the model parameters:

α1 ðiÞ:β1 ði Þ
π^i ¼ γ1 ði Þ ¼ (39)
P½Ojλ

T1 T1
∑ ξ t ði; j Þ ∑ αt ði Þaij bj ðOtþ1 Þβtþ1 ðjÞ
^ij ¼
a t¼1
¼ t¼1
(40)
T1 T1
∑ γt ðiÞ ∑ αt ði Þβt ðiÞ
t¼1 t¼1

T
∑ γt ðj Þ
t¼1
∑Tt¼1O ¼k αt ðj Þβt ðjÞ
^j ðk Þ ¼ For Ot ¼V k ¼
b t
(41)
T T
∑ γt ðj Þ ∑ αt ðjÞβt ðj Þ
t¼1 t¼1

4.5. Perplexity measurement

The HMM model performance can be measured in two ways:
1. By classification of accuracy;
2. By perplexity.
Classification of accuracy is the amount of correct predictions of hidden states vector divided by the total amount of hidden states
under prognostication. However, for relatively small data sets, the classification of accuracy is a noisy measure, because each sample
can be assigned for only one class. Therefore, a better measure is the perplexity of the test data set.
The perplexity equation calculates the probability of the system being in state i at time t:
Ns
N1s ∑ LK;i
Perplex qi ¼ e k¼1 (42)

The perplexity measures the confidence of the predictions of the classifier. This classifier is defined as a function of the average of
log-likelihoods, Lk,i, of the Ns data sequences, that finished in state qi, formally denoted by
h  i
Lk;i ¼ log p Stk ¼ qi jok1; :::;T k;i ; λ (43)

where ok1; :::;T k;i stands for the kth sequence of observations of length Tk,i, and qi is the class of hidden states; Ns is the number of
sequences, and λ the model parameters.

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
A. SIMÕES ET AL.

The best possible perplexity is 1, where the correct state is predicted with a probability of 1. On the other hand, a perplexity of 3
corresponds to random guessing with a probability of 1/3 for one or several hidden states.25

5. Validating
The detailed steps of data collection, treatment, and model validation will be too extensive to be exhaustively described in this paper.
The hidden part of the model is evaluated taking as base three state indicators (engine torque, turbo pressure, and oil pressure)
that are measured for three rotation speed of the engine.
The selected four visible impact indicators correspond to the opacity of the gases, gaseous emissions of NOx and CO2, and NOISE
level produced by the engine.
The data was collected during 29 periods of 2 weeks that represent 58 weeks.
As an example, Tables V and VI are the specified data collected in bus number 262. This bus was one of four that integrated the
experimental study.
The collected data were treated according to the algorithms specified in the first author thesis. After treatment, results in the final
data classification are presented in Table VII.
After this, the chronological sequences were used in the construction of HMM, being applied in four buses.
The evolution of state 1–2, 3, 4, and 5 corresponds to a progressive degradation of the system. The last ones, 4 and 5, represent
fault states.

Table V. Hidden variables of the model

Engine Torque Turbo Pressure Oil Pressure
rpm 600 1150 1630 600 1150 1630 600 1150 1630
1st test 10 82 104 1.021 1.087 1.209 2.019 2.947 4.018
2nd test 10 81 102 1.031 1.091 1.212 2.023 2.951 4.033
3rd test 9 76 97 1.015 1.057 1.153 1.974 2.925 4.019
— — — — — — — — — —
29th test 3 67.5 81 1.09 1.09 1.215 2.055 2.999 4.105

Table VI. Visible variables of the model

OPAC NOx CO2 NOISE

rpm 600 1150 1630 600 1150 1630 600 1150 1630 600 1150 1630
1st test 0.45 0.45 0.45 284 280 204 2.35 2.42 2.83 90.6 93.2 95.9
2nd test 0.52 0.52 0.52 280 274 199 2.23 2.43 2.81 90.4 93.8 96.2
3rd test 0.60 0.60 0.60 307 301 238 2.42 2.75 3.07 90.6 94.1 96.5
— — — — — — — — — — — — —
29th test 0.48 0.48 0.48 382 341 255 2.83 2.63 2.93 92.7 95.9 98.1

Table VII. Chronological sequences of HMM values for each bus

1 2 3 4 5 — 25 26 27 28 29

Monitoring time —
Bus 262 State 1 1 2 4 5 — 2 2 2 3 4
Emissions 2 1 4 8 9 — 4 4 3 5 7
Bus 265 State 1 2 3 3 5 — 2 2 2 2 3
Emissions 1 2 5 6 9 — 3 3 6 5 7
Bus 269 State 2 2 1 4 5 — 3 2 5 5 4
Emissions 3 2 1 7 10 — 6 5 10 9 8
Bus 270 State 1 5 5 5 4 — 1 1 2 2 2
Emissions 1 9 10 11 8 — 1 1 2 4 5

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
 A. SIMÕES ET AL.

Table VIII. Relationship between HMM estimated states and measured states
Spectrum of emissions measurement Health states validation
Weeks of validation Weeks of validation

Bus 1 2 3 1 2 3
262 V7 V4 V5 HMM q4 q2 q2
Measurement q4 q2 q2
265 V7 V9 V5 HMM q3 q4 q2
Measurement q3 q5 q2
269 V9 V4 V4 HMM q4 q3 q2
Measurement q5 q3 q2
270 V5 V6 V2 HMM q2 q3 q2
Measurement q3 q4 q1

Table IX. HMM perplexity associated each bus and global

Perplexity for: Bus 262 Bus 265 Bus 269 Bus 270
States 1 and 2 1.4520 1.0000 1.8571 1.0000
State 3 NaN 1.0000 1.2500 1.0000
States 4 and 5 1.0000 1.8333 1.0000 NaN
Global HMM 1.2823 1.2239 1.3241 1.0000
NaN, not a number – state did not happen during validation period.

On the other hand, the evolution of emissions level from classes 1 to 11 means the increasing in environmental impact.
As resulted of this research, it was created the conditions for the application of Hidden Markov Models in predictive maintenance,
with the respective algorithms aforementioned in chapter 4.
The model validation synthesis is presented in following table:
Through the analysis of Table VIII, we can see that did not exist in any of the estimated states by the HMM any deviate higher than
one level of states, which represent a great incentive to the use of the model in future applications with a higher number of training data.
The application of the perplexity calculation model, using MATLAB tool, led to obtain the values showed in Table IX.

6. Conclusions
While enterprises of transports in the USA and Canada are working with high-spare ratio despite low-average fleet age, the SMTUC is
operating with an extremely low-effective spare ratio associated with high-average fleet age. The differential between total active
fleet and the peak-service requirements is partially absorbed by the rebuilding of the more degraded buses. This was the challenging
situation in which the HMM model presented in the paper was designed and validated.
The paper presents a new methodology for the estimation of hidden states of a complex system. It creates the basis of predictive
maintenance strategies with the aim of reducing negative environmental impacts, while simultaneously reduces maintenance and
associated operational costs with vehicles powered by diesel engines, especially in urban areas.
As was researched in PhD thesis’ author, the application of those algorithms shown that environmental predictive maintenance
practices allow to detect when dysfunctional symptoms occur and to follow them up before the fault occurrence. The first signs of
the aging can be identified with an emissions spectrum of high impacts. With this model, it is possible to diagnose the severity of
dysfunctions and, at the same time, to avoid high contamination levels in the air.
The formulation presented in this paper shows that this new paradigm is very promising for state characterization. The system can
give a warning that the deterioration of the bus engine has achieved an inadmissible threshold.
Because HMM is very robust at handling the probabilities of state condition associated to certain spectra of emissions, the
implementation success depends on the classification criteria and monitoring frequency.
The HMM algorithms have demonstrated their adequacy to condition monitoring predictive maintenance of diesel engines.

Acknowledgements
“This research is sponsored by FEDER funds through the program COMPETE – Programa Operacional Factores de Competitividade – and by
national funds through FCT – Fundação para a Ciência e a Tecnologia – under the CEMUC project UID/EMS/00285/2013.”

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017
A. SIMÕES ET AL.

References
1. Honkanen T. Modelling industrial maintenance systems and the effects of automatic condition monitoring. Dissertation for the degree of Doctor of
Science in Technology. Helsinki University of Technology, 2004.
th
2. Simões A, Viegas JM, Farinha T. Ecological predictive maintenance in hospital equipment. 20 Congress of International Federation of Hospital
Engineering (IFHE), Barcelona - Espanha, 2008.
3. Simões A, Farinha T, Fonseca I, Barbosa FM, Marques V. Buses degradation based on exploration conditions. WSEAS International Conferences.
Puerto De La Cruz, Tenerife, Canary Islands, Spain, July 1–3, 2009.
4. Simões A. Manutenção Condicionada às Emissões Poluentes em Autocarros Urbanos - Diagnóstico por Cadeias Escondidas de Markov. Doctoral
Thesis (Ph.D.), UTL / Instituto Superior Técnico: Lisboa, 2011.
5. Djurdjanovic D, Lee J, Ni J. Watchdog Agent - an Infotronics-based Prognostics Approach for Product Performance Degradation Assessment and
Prediction. Elsevier: Amsterdam, 2003.
6. Billinton R, Allan RN. Reliability Evaluation of Engineering Systems: Concepts and Techniques. Pitman Books Limited: London, 1983. doi:10.1016/
0143-8174(83)90020-3.
7. Feng C, Guensler R, Rodgers M. Transit bus load-based modal emission rate model development. U.S. Environmental Protection Agency:
Washington, DC, EPA/600/R-07/106 (NTIS PB2008–108876), 2007.
8. Jimenez-Palacios JL. Understanding and quantifying motor vehicle emissions with vehicle specific power and TILDAS remote sensing. Doctoral
Thesis (Ph.D.), Massachusetts Institute of Technology, 1999.
9. Walker AP. Controlling particulate emissions from diesel vehicles. Plenum Publishing Corporation, Topics in Catalysis Vol 28, Nos. 1–4, 2004. DOI:
10.1023/B:TOCA.0000024455.89457.
10. Brahma A. Methodologies for modelling and feedback control of the nitrogen oxide – BSFC trade-off in high-speed, common-rail, direct-injection
diesel engines. Doctoral Thesis (Ph.D.): The Oyo State University, USA, 2006.
11. UITP. Ticket to the future - 3 stops to sustainable mobility, UITP - International Association of Public Transport: Rue Sainte Marie, 6, B-1080 Brussels
Belgium, Heather Allen, Issue Management, 2003.
12. SAE_J1667, Ed. Society of automotive engineers J1667 recomended practice smoke test procedure, 1996.
13. Wenzel T, Ross M. Emissions from modern passenger cars with malfunctions emissions controls. Society of Automotive Engineers, USA, SAE
Technical Paper 960067, 1996, DOI:10.4271/960067.
14. Drive Clean Program. Benefit analysis and reporting – heavy duty vehicles – years 2000 to 2005. G.W. Taylor consulting, 2007.
15. Al-Najjar B, Alsyouf I. Enhancing a company’s profitability and competitiveness using integrated vibration-based maintenance: a case study.
Department of Terotechnology, School of Industrial Engineering, Växjö University: Lückligs plats 1, 351 95 Växjö, SwedenEuropean Journal of
Operational Research 02/2004. doi:10.1016/S0377-2217(03)00258-3.
16. Rabiner LR. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 1989; 77(2). doi:10.1109/5.18626.
17. Moubray J. Reliability-centered Maintenance - Third Edition, Industrial Press, Inc: New York, 2012, ISBN:0-8311-3445-3.
18. TCRP-SINTHESIS-81. Preventive maintenance intervals for transit buses – a synthesis of transit practice. T. R. B. o. t. N. Academies, Transit
Cooperative Research Program, 2010.
19. COST-319, Ed. Estimation of pollutant emissions from transport. Bruxelas, Comissão Europeia, 1996.
20. COST-346. Energy and fuel consumption from heavy duty vehicles. E. b. S. P. J. a. H. S., 2008.
21. TRL. Transport Research Laboratory - UNITED KINGDOM. T. R. Laboratory, transport research laboratory, 2012.
22. Simões A, Farinha T, Fonseca I, Marques V. Manutenção Condicionada às Emissões Poluentes em Autocarros Urbanos – Uma abordagem ecológica,
APMI: Exponor - Matosinhos, 2007.
23. Oliveira L, Morita M. Introdução aos Modelos Escondidos de Markov (HMM). PPGIA – Programa dePós-Graduação em Informática Aplicada.
Pontifícia Universidade Católica do Paraná – PUC-Pr: Rua Imaculada Conceição, 1155–80215-901 – Curitiba, Pr. Brasil, 2000.
24. NORMA_NP_EN_50126. RAMS – reliability, availability, maintainability and safety, 2000.
25. Simola J, Salojärvi J, Kojo I. Using hidden Markov model to uncover processing states from eye movements in information search tasks. Journal
Cognitive Systems Research 2008; 9(4):237–251. Elsevier: Amsterdam.

Authors' biographies
António Simões holds a PhD in Transports from the Technical University in Lisbon since September 2011. The PhD thesis was about
the theme ”On-Condition Maintenance based on Pollutant Emissions in Urban Areas—Diagnosis by Hidden Markov Models.“ He is
presently an Adjunct Professor at the Mechanical Engineering Department of the Coimbra Institute of Engineering a high school
integrated in Polytechnic Institute of Coimbra. He is author or coauthor of many international scientific papers. His research is focused
in transports area, especially in transports maintenance.
José Manuel Viegas has been Secretary-General of the International Transport Forum at the OECD since August 2012. A Portuguese
national, Mr. Viegas has had a distinguished career in academia and in the private sector before joining the Forum as its chief
executive. A full Professor of Transport at the Technical University of Lisbon, he served as Director of MIT-Portugal’s Transport Systems
focus area and founded TRANSPORTNET, a group of 8 European university research groups in transport systems. As chairman of TIS.pt
consultants, he successfully advised governments and international institutions including the World Bank and the European
Commission on a number of high-profile policy initiatives and transport projects. Mr. Viegas holds a PhD in Civil Engineering from
the Technical University in Lisbon and undertook postgraduate studies in regional studies at the University of Karlsruhe, Germany.
He speaks fluent Portuguese, English, French, Spanish, German, and Italian.
José Torres Farinha is a PhD in Mechanical Engineering. His main research is in the field of maintenance management, including the
field of wind generators on-condition maintenance. He developed a computer maintenance management system. He manages
projects of implementation of on-condition predictive maintenance, and also the introduction of new management policies, like
5S, TPM, LEAN, and related subjects as PDCA cycle, GUT, A3, and so on.
Inácio Fonseca is a PhD in May 2010. The Ph.D. thesis was about the theme ”Maintenance of Renewable Wind Energy Generation through
IP Networks.“ He participated in 2 projects funded by the Portuguese FCT entity. He was a doctoral fellow from October 2007 to May 2010.
He is author or coauthor of many international scientific papers, including 1 book chapter and 5 international transactions papers.

Copyright © 2017 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2017

View publication stats