sensors

Article
An Industrial Digitalization Platform for Condition
Monitoring and Predictive Maintenance of
Pumping Equipment
Michael Short 1, * and John Twiddle 2
1 School of Science, Engineering and Design, Teesside University, Middlesbrough TS1 3BA, UK
2 Scottish & Southern Energy Ltd., Knottingley, West Yorkshire WF11 8SQ, UK
* Correspondence: m.short@tees.ac.uk




Received: 12 June 2019; Accepted: 28 August 2019; Published: 31 August 2019


Abstract: This paper is concerned with the implementation and field-testing of an edge device
for real-time condition monitoring and fault detection for large-scale rotating equipment in the
UK water industry. The edge device implements a local digital twin, processing information from
low-cost transducers mounted on the equipment in real-time. Condition monitoring is achieved
with sliding-mode observers employed as soft sensors to estimate critical internal pump parameters
to help detect equipment wear before damage occurs. The paper describes the implementation
of the edge system on a prototype microcontroller-based embedded platform, which supports the
Modbus protocol; IP/GSM communication gateways provide remote connectivity to the network core,
allowing further detailed analytics for predictive maintenance to take place. The paper first describes
validation testing of the edge device using Hardware-In-The-Loop techniques, followed by trials
on large-scale pumping equipment in the field. The paper concludes that the proposed system
potentially delivers a flexible and low-cost industrial digitalization platform for condition monitoring
and predictive maintenance applications in the water industry.

Keywords: industrial digitalization; industry 4.0; sliding mode observers; soft sensors; edge
computing; condition monitoring; predictive maintenance; field testing; IoT

1. Introduction
With improved connectivity and dramatically increased access to low-cost computational power,
many industries are currently on the verge of a second digital revolution known as ‘Industry 4.00 [1].
Of the many advantages that can potentially be leveraged by Industry 4.0, the promise of increased
integration of the real-time control, monitoring, and operational aspects of equipment in the process
industries is one of the most attractive [1,2]. It has previously been suggested that one of the so-called
grand challenges which can be addressed by digitization is the notion of the ‘sustainable, 100%
available plant’ [1]. The 100% availability notion captures the vision that in the future there will be
only ever be planned maintenance stops; while clearly never achievable in practice, the right balance
between maintenance cost and risk must clearly be aimed for. At the heart of this aspect of industrial
digitalization the topics of real-time Condition Monitoring (CM) and Predictive Maintenance (PM) can
be found [1,2]. As with other Internet-of-Things and Industry 4.0 applications, CM/PM applications
can potentially generate large volumes of high-velocity real-time data; therefore, it is beneficial to run
local processing of this data at the edge of the network (on an ‘edge device’), to reduce the frequency
and real-time requirements of communication to core devices [3]. In addition, since it is essential to
have adequate fault indices that could serve as indicators in order to predict early failures and create
an effective PM schedule [2], in an Industry 4.0 context an appropriate approach would seem to be to
distribute the tasks of CM/indices generation and PM analytics between the network edges and Core.

Sensors 2019, 19, 3781; doi:10.3390/s19173781 www.mdpi.com/journal/sensors

Sensors 2019, 19, 3781 2 of 16

To this end, this paper is concerned with the development, validation and field-testing of a
real-time CM/PM system for deployment upon large-scale pumping equipment in the water industry
in an Industry 4.0 context. The CM and fault detection is achieved with an edge device implementing
sliding-mode observers as soft sensors, which are designed to monitor equipment locally in a water
pumping station. The role of the observers is to detect equipment wear - from mechanical stresses,
or the ingress of coolant or other particulates into coupling bearing support housing - before further
damage occurs to the equipment. It achieves this by monitoring several low-cost transducer signals
from the equipment, and building a digital twin of the equipment being monitored; including the
estimation of two critical internal parameters of the rotating equipment (bearing friction factor and
coolant flow) which are impossible or difficult/expensive to directly measure. The estimated values
(in conjunction with other instrumented measurements) are used as fault diagnosis indices. The edge
device also performs data sampling and filtering and provides outputs for fault detection in the form
of local status indicators and alarms. A Modbus RTU server is implemented on the device, and wider
connectivity to the network core is provided through an additional Serial to Mobile Data (3G/4G)
gateway. The core is able to pull preprocessed and summarized digital data related to local real-time
parameter measurements and estimations at a lower velocity through this gateway, where further
analytics related to PM are applied; this provides an architecture in keeping with the decentralized,
hierarchical structure of the digitalization revolution and IoT [1–3]. The focus of this paper is principally
upon the implementation and testing of the edge device and soft sensors for CM.
Given the relative importance of the ‘100% available plant’ aspirational target, the CM/PM
application in question can be considered to be mission-critical as any ‘false negatives’ indicated by
the system may result in avoidable permanent damage to equipment, and ‘false positives’ may cause
unnecessary system downtime. Both unwanted outcomes may lead to large and unnecessary economic
costs and can potentially interrupt a region’s critical water supply infrastructure. It is therefore of vital
importance to ensure that adequate Verification and Validation (V&V) exercises and prototype field
trials are carried out on any implementation of this edge device. This paper describes such a set of
exercises, carried out on a prototype Commercial Off The Shelf (COTS) microcontroller-based embedded
solution, which is to be deployed locally in a pump house; a potentially harsh industrial environment.
Much previous work has been invested into developing accurate condition monitoring of rotating
equipment in industrial applications; many focus on the use of vibration or acoustic analysis using
real-time data processing techniques such as Wavelet transforms, Machine Learning approaches and
Parameter Optimization (see, e.g., [2,4–8]). However, the specific focus of this paper is on sliding mode
observer techniques, which have also been widely used for condition monitoring and fault diagnosis
in recent years (see e.g., [9–11]). Two of their main advantages are that they exhibit fundamental
robustness against certain kinds of parameter variations and disturbances, while also avoiding the
need for high-speed signal processing techniques required for acoustic or vibration data; a specialized
Sliding Mode Observer (SMO) for fault detection and isolation was developed by Edwards et al. to
exploit these features [10]. Additional key benefits of this observer approach include good accuracy
of operation, combined with comparatively lower-overheads than related techniques [9–11], making
them a good candidate for real-time implementation in an Industry 4.0 context. Therefore, the principal
goal of this paper is to help bridge the theory and practice gap by providing a detailed description
and credible demonstration of an industrial implementation of a simple but effective edge device
implementing this observer-based technique for fault detection of pumping equipment. Both the
observer itself, and the techniques applied to implement the observer in real-time on a small, low-cost
portable embedded system, have a fundamental theoretical grounding. Through laboratory-based
validation using fault injection, combined with online field trials, it is demonstrated that the developed
system seems a good candidate for wider deployment in industry to help achieve the goals of the
second digital revolution.
The remainder of this paper is organized as follows. Section 2 describes the current application
area in more depth, along with the motivation for using observers as soft sensors and their mode
 Sensors 2019, 19, x FOR PEER REVIEW 3 of 16
Sensors 2019, 19, x FOR PEER REVIEW 3 of 16
The remainder of this paper is organized as follows. Section 2 describes the current application
areaThe
Sensorsin more
2019, depth,ofalong
19, 3781
remainder with the
this paper motivationasfor
is organized using observers
follows. as soft sensors
Section 2 describes and their
the current mode of
3 of 16
application
operation.
area in moreSection
depth, 3along
describes themotivation
with the prototype for
hardware and software
using observers as softimplementation
sensors and their of mode
the edgeof
device, along with a discussion of communication network integration. Section
operation. Section 3 describes the prototype hardware and software implementation of the edge 4 discusses off-line
of operation. Section 3 describes the prototype hardware and software implementation of the edge
validation
device, alongtesting
with of the device, of
a discussion while Section 5 is concerned
communication with on-lineSection
network integration. testing 4(field trials).off-line
discusses Section
device, along with a discussion of communication network integration. Section 4 discusses off-line
6 concludes
validation the paper.
testing of the device, while Section 5 is concerned with on-line testing (field trials). Section
validation testing of the device, while Section 5 is concerned with on-line testing (field trials). Section 6
6 concludes the paper.
concludes
2. Slidingthe Modepaper.
Observers as Soft Sensors
2.
2. Sliding
Sliding Mode
Mode Observers
Observers asas Soft
Soft Sensors
Sensors
2.1. Sliding Mode Techniques
2.1.
2.1. Sliding
Sliding Mode
As shownModein Techniques
Techniques
Figure 1, a sliding mode observer is a state observer that employs sliding mode
techniques
As
As shown
shownto give
in accurate
in Figure 1, aestimates
sliding modeof a system’s
observerinternal
is a state state in the that
observer presence of uncertainty
employs sliding modeand
mode
parameter
techniques
techniques to variations.
to give The
give accurate observer
accurate estimates
estimates ofis designed
of aa system’sto maintain
system’s internal
internal statethe sliding
state in
in the motion,
the presence even in the
presence of uncertainty presence
uncertainty and
of faults, and
parameter reconstructs
variations.
variations. The fault
The observersignals
observer is as a function
is designed
designed to of the so-called
to maintain
maintain the
the slidingequivalent
sliding motion,output
motion, even
even in injection
in the signal
the presence
presence
[9–11].
of
of faults,
faults,These
and estimates
andreconstructs offault
reconstructs the unmeasured
signals
fault signals fault
as aasfunction signals
a functionof theofcan thethen
so-called be used as measures
equivalent
so-called outputoutput
equivalent of equipment
injection signal
injection
health,
[9–11]. and
signal [9–11]. employed
These estimates to raise alarms
of the unmeasured
These estimates or to schedule
fault signals
of the unmeasured maintenance
faultcan then be
signals prior
canused to
then critical
as be
measuresfailures
used asofmeasuresoccurring
equipment of
in the equipment
health,
equipment and health,
employed being
andto monitored.
raise alarms
employed to Theor to
raise design procedure
schedule
alarms or tomaintenance
schedule for this model-based
prior estimation
to critical
maintenance prior failures
to critical approach
occurring
failures
is the
in principally
occurringequipment
in thecharacterized by twomonitored.
being monitored.
equipment being components:
The designThe (i)design
the design
procedure of anmodel-based
for this
procedure appropriate sliding surface
estimation
for this model-based where
approach
estimation
the
is system
principally
approach will demonstrate
characterized
is principally the desired
by two components:
characterized dynamics and (ii)
(i) the design(i)
by two components: the selection
of the of
an appropriatean injection
design of an sliding signal that will
surfacesliding
appropriate where
force
the
surface the
system system
willthe
where to reach will
demonstrate
system and themaintain
demonstrate its sliding
desired dynamics motion
and
the desired (ii) [9–11].
the
dynamics selection of the
and (ii) an injection
selectionsignal that will
of an injection
force
signalthe system
that to reach
will force and maintain
the system to reachitsand
sliding motion
maintain its [9–11].
sliding motion [9–11].

Figure 1. Basic operation of a sliding mode observer.

Figure 1. Basic operation of a sliding mode observer.
Figure 1. Basic operation of a sliding mode observer.
2.2. Application
2.2. Applicationin
inthe
theWater
WaterIndustry
Industry
2.2. Application
Muchof
Much in industrial
ofthe
the the Water Industry
industrial plantin
plant inthe
theUK
UKwater
waterindustry
industryconsists
consistsof
of large-scale
large-scalerotating
rotatingmachinery
machinery
suchMuch
such ascan
as canofbebe found
found
the inpumping
in
industrial pumping
plant in stations.
stations.
the UK For Forexample,
waterexample,
industry inconsists
in thefirst
the firstof
stage
stage ofaawater
of
large-scalewater treatment
treatment
rotating works
works
machinery
aa regulated
regulated supply
supply of
of water
water (from
(from aa reservoir)
reservoir) is
is necessary.
necessary. As
As precipitation
precipitation
such as can be found in pumping stations. For example, in the first stage of a water treatment works is
is unpredictable,
unpredictable, water
water
can
acan bedrawn
be drawn
regulated fromof
from
supply aaconstant
constant
water supply
supply—such
(from - such as
a reservoir) as
isaanecessary.
river - that
river—that isprecipitation
Asis close to
close toground
ground levelto
level toregulate
regulate
is unpredictable, the
the
water
reservoir
reservoir
can be drawn level,
level, asshown
as
from shown inFigure
in
a constant Figure 2.2.In
supply In suchas
such
- such anaapplication,
an application,
river - that aaislarge
large amount
closeamount
to ground ofwater
of water may
level may
to haveto
have
regulate to be
be
the
pumped
pumped a a considerable
considerable distance
distance in
in order
order to
to regulate
regulate the
the mains
mains water
water supply.
supply.
reservoir level, as shown in Figure 2. In such an application, a large amount of water may have to be The
The equipment
equipment in
in such
such
stationsisaisrequired
stations
pumped requiredtotohave
considerable havea ahigh
distancehigh
in level
level
order ofofavailability,
toavailability,
regulate the andand is relatively
is relatively
mains expensive
water expensive
supply. The to repair
to equipment
repair if in
damage
if damage such to
to couplings
couplings and and shafts
shafts occurs.
occurs. The The current
current application
application is is concerned
concerned with
with monitoring
monitoring
stations is required to have a high level of availability, and is relatively expensive to repair if damage the the health
health status
status of
toofcouplings
the the main
main shaft shaft
andsupport support
shafts bearing
occurs. bearing
in this
The in
kind
current this
of kind
pumping
applicationof pumping
isequipment,
concerned equipment,
to detect
with to detect
possible
monitoring possible
fault
the fault
conditions
health status
ofconditions
suchtheasmain such
increasedshaftas increased
wear,
supportwater wear,
ingress
bearing water
inorthisingress
coolant
kindflowor
of coolant
failure. flow
pumping failure. to detect possible fault
equipment,
conditions such as increased wear, water ingress or coolant flow failure.

Figure2.2.Overview
Figure Overviewof
ofcurrent
currentapplication
applicationarea.
area.
Figure 2. Overview of current application area.
The shaft support bearings are often monitored using spectral analysis of vibration signals [5].
Increased temperature is also a common symptom of bearing wear or of a lubrication failure [11].
Sensors 2019, 19, x FOR PEER REVIEW 4 of 16

Sensors 2019, 19, 3781 4 of 16

The shaft support bearings are often monitored using spectral analysis of vibration signals [5].
Increased temperature is also a common symptom of bearing wear or of a lubrication failure [11].
several factors
However, several factors such
such as
as ambient
ambient temperature,
temperature, load load or
or instantaneous
instantaneous rotational shaft speed
can also cause natural variations in the measured measured bearing
bearing temperature; a simple limit or threshold threshold
approach to detection of overheating is not generally acceptable, as it needs
to detection of overheating is not generally acceptable, as it needs to accommodate to accommodate a wide a
rangerange
wide of ambient and operating
of ambient and operatingconditions whilstwhilst
conditions minimizing
minimizingthe number
the numberof false alarms.
of false The
alarms.
robustness
The robustnessof the sliding
of the slidingmode
modeapproach
approach cancan
take
takeaccount
accountofofthese
theseenvironmental
environmental and and operational
factors, efficiently
factors, efficientlyeliminating
eliminating them
them fromfrom the detection
the fault fault detection
process,process,
and thusand thus increasing
increasing the
the sensitivity
sensitivity
of of the system
the diagnosis diagnosis systemthe
reducing reducing
number theofnumber of false positives.
false positives. Compared Compared withanalysis,
with spectral spectral
analysis, temperature
temperature monitoringmonitoring
using reliable using reliable buttransducers
but inexpensive inexpensive andtransducers
sliding mode and sliding mode
techniques offers
techniques
many offers
potential many including
benefits, potential benefits, including
lower overall costs, lower overall costs,
lower sampling lower
rates, sampling
and less rates, and
computationally
less computationally
intensive signal processing intensive signal [9–11].
techniques processing techniquesmonitoring
A temperature [9–11]. A approach
temperature alsomonitoring
avoids the
approach also avoids the problem of background noise and vibration from
problem of background noise and vibration from other pumps interfering with the data. Additionally, other pumps interfering
withlow
the thesampling
data. Additionally, the low sampling
rates and computational rates
intensity and seem
would computational
to indicateintensity would
that low-cost seemand
ADC’s to
indicate that low-cost
microcontroller units may ADC’s and microcontroller
be employed to realize theunits may bein
CM system employed to realize
a simple edge thethis
device; CMprovides
system
in amotivation
the simple edge fordevice; this provides
their deployment inthe
themotivation
current work. for their deployment in the current work.

2.3. Technical Details

Previous work by Kitsos et al. [11] has shown that a multiple observer observer approach
approach—with
– with the
observers
observers designed
designed asas virtual
virtual or
or ‘soft’
‘soft’ sensors—to monitor coolant
sensors - to monitor coolant flow
flow and bearing
bearing wear respectively
provides a highly
highly efficient
efficient solution.
solution. In In general
general terms,
terms, the
the observers
observers are designed to detect early
predictors
predictors of ofbearing
bearingoverheating
overheatingdue duetoto either:
either:(i)(i)
loss or or
loss reduction
reductionof coolant flowflow
of coolant to the
to bearing, e.g.,
the bearing,
if theifwater
e.g., sumpsump
the water runs runs
dry ordry(ii)orincreased friction;
(ii) increased e.g., due
friction; e.g.,todue
wear and tear,
to wear andreduction in oil level,
tear, reduction in oil
ingress of particles
level, ingress or water
of particles or into
water bearing itself. Aitself.
into bearing schematic and picture
A schematic of the bearing
and picture in the current
of the bearing in the
application is shownisin
current application Figurein3.Figure 3.
shown

Figure 3. Detail of the shaft coupling.

The observers
observersthemselves
themselvesare are
basedbased
upon upon
two simplified heat transfer
two simplified heatmodels of the
transfer arrangement
models of the
shown in Figure
arrangement 3. Equation
shown in Figure (1) captures the
3. Equation (1) coolant
capturesheat
the transfer dynamics,
coolant heat while
transfer Equation
dynamics, (2)
while
captures
Equationthe
(2) bearing
capturesheat
the transfer dynamics:
bearing heat transfer dynamics:
Q Coolant
. = Q FromBearing
. − Q CoolantOut.
QCoolant = QFromBearing − QCoolantOut
mC CmCCTCoC=Tohc A=c (ThB c−AT
.
c (oT)B−−m
TcoC) c−(Tm.oc −
CcT(iT)o − Ti ) (1)
(1)
. .
T =Tob (T
o
= −bT
1
1 (T) B−−mTbo ) (−Tm−c bT2 ()To − Ti )
B o c 2 o i
. . . .
QBearing = QFriction − QCoolantOut − QAtmosphere
. .
mB CB TB = µF f N − mc Cc (To − Ti ) − hB AB (TB − Ta ) (2)
.
TB = a1 N − a2 (To − Ti ) − a3 (TB − Ta )
Sensors 2019, 19, 3781 5 of 16

where Ti and To are the coolant inlet/outlet temperatures (◦ C) respectively, TB is the measured bearing
temperature (◦ C), Ta is the ambient temperature (◦ C) and scalars ai and bi are appropriate model
.
coefficients employed to simplify the expressions. The coolant mass flow rate is denoted m (kg/s).
The corresponding sets of equations defining the coolant and bearing observers are given below in
Equations (3) and (4) respectively:
.̂
To e. (T̂ − T ) − ν
= b1 (TB − T̂o ) − mb 2 o i o
εo = T̂o − To (3)
νo = Ko εo
.̂
TB a1 N − a2 (To − Ti ) − a3 (T̂B − Ta ) + νB
= e
εB = T̂b − Tb (4)
νB = KB εB
where νB and νO are the ‘injection signals’ for the coolant outlet temperature and bearing observers
respectively, and εi are the model errors. Note that an accented variable denotes the observer model
estimation. The observer is designed such that the state error signals are driven toward zero. When the
.
state error is small, any changes in parameters m and a1 caused by loss or reduction in coolant flow or
because of increased heating due to friction can be approximated using Equations (5) and (6) as follows:
. νo
∆m ≈ (5)
b2 (To − Ti )
νB
∆a1 ≈ (6)
N
where N is the shaft speed (in Hz). The observer has been found to work well under purely simulation
conditions, but to be of practical use in the field an implementation that is capable of functioning
correctly in the relatively harsh environment of a pumping station is required. Remote connectivity to
a control center for data analysis, logging, and predictive maintenance scheduling is also required.
The hardware, software, and communication architectural design of a prototype edge device for this
purpose is described in the next Section.

3. Prototype Edge Device

3.1. Hardware Architecture

Several possibilities were considered for implementing the edge device in hardware, including
the use of Raspberry Pi and Arduino-based devices. However, the microcontroller employed in this
prototype was an Infineon© C167CS housed in a Phycore-167 SBC (Single-Board-Computer) unit.
This unit was mounted on a development board purchased from Phytec©. This configuration was
chosen due to the suitability of the device for deployment in harsh industrial environments (the C167
microcontroller was originally developed for automotive/in-vehicle applications), and the availability
of a reliable/mature development toolchain along with proven static code analysis and execution
time profiling tools. The host CPU features 4 KB of on-chip RAM (IRAM), the CPU and associated
peripherals. The SBC consists of the host CPU, an oscillator/reset/brownout circuit, off-chip 62-KB
FLASH ROM and 2 × 128-KB RAM banks (XRAM), plus a 16-KB non-volatile memory chip (EEPROM)
for storing system configuration information. A 25 MHz CPU clock speed was employed in the design.
Although the C167 has on-chip ADC converters, the resolution was limited to 10-bits. To increase the
resolution to a more acceptable level, an 8-channel 14-bit self-calibrating ADC converter (the AD7856
from Analog Devices Ltd.) was employed. An SPI interface was employed to the host CPU, and 5 V
Zener diodes were employed to prevent the ADC input circuitry being damaged by any excessive
voltages. The hardware employed in the system is shown schematically in Figure 4. Sampling of
input channels occurred at 100 Hz frequency, using a PWM signal generated by the C167 hardware.
 employed in the design. Although the C167 has on-chip ADC converters, the resolution was limited
to 10-bits. To increase the resolution to a more acceptable level, an 8-channel 14-bit self-calibrating
ADC converter (the AD7856 from Analog Devices Ltd.) was employed. An SPI interface was
employed to the host CPU, and 5 V Zener diodes were employed to prevent the ADC input circuitry
Sensors
being2019, 19, 3781 by any excessive voltages. The hardware employed in the system is shown
damaged 6 of 16

schematically in Figure 4. Sampling of input channels occurred at 100 Hz frequency, using a PWM
signal
Very generated
little host CPUby the C167 hardware.
management Veryrequired;
was therefore little host CPUsampling
hence, management wasreduced.
jitter was thereforeThe
required;
main
intervention required by the host CPU was for the extraction of the conversion values once for
hence, sampling jitter was reduced. The main intervention required by the host CPU was the
every
10extraction of the conversion
ms, a jitter-insensitive task.values once every 10 ms, a jitter-insensitive task.

Figure 4. Main hardware elements employed in the system.

Figure 4. Main hardware elements employed in the system.
3.2. Software Architecture
3.2. Software Architecture
Upon power on/reset, the system first performed a software-based self-test, consisting of a ROM
checksumUpon power
test, a RAMon ‘March’
/ reset, test
the followed
system first
by aperformed a software-based
timer/oscillator self-test. The self-test,
latter testconsisting
compares the of a
ROM checksum test, a RAM ‘March’ test followed by a timer/oscillator self-test.
amount of CPU cycles elapsing over a fixed duration of the on-chip hardware timers, and compares The latter test
compares the amount of CPU cycles elapsing over a fixed duration of the on-chip
these values with the time taken to charge an external resistor/capacitor network with fixed time hardware timers,
and compares
constant. If thesethese
testsvalues with the
are passed time takenato
successfully, chargeinitialization
system an external resistor/capacitor
function is called.network
In additionwith
to performing system initialization tasks such as setting up the ADC converter, the nine tunableIn
fixed time constant. If these tests are passed successfully, a system initialization function is called.
addition tothat
parameters performing
define thesystem initialization
observer constants aretasks such asfrom
extracted setting up the ADC
an EEPROM. converter,
Following this,the nine
a task
tunable parameters
scheduler thatapplication
is started. The define the software
observer functionality
constants arewas extracted
brokenfromdownan EEPROM.
into Following
six separate tasks.
this,run-time
The a task scheduler is started. The
software architecture wasapplication
as shown insoftware
Figure 5.functionality was broken down into six
separate tasks.
The first taskThe
wasrun-time
requiredsoftware
to extractarchitecture
the five ADC waschannel
as shown in Figurereadings
conversion 5. from the AD7856
via an SPI link to obtain the input signals. The second task was then required to filter the signals using
recursive first-order low pass filters, and convert these readings into appropriate engineering units.
The third task evaluates the observer models, implementing equations (1–4). Based on the filtered
outputs of the observer, the fourth task drives the local indicator status LED’s. To allow the observer to
converge upon power on/reset, the software is required to suppress any faulty/degrading outputs for
a user-adjustable period, typically no more than 10 s. The fifth task updates the main system state
machine once every 50 ms. The sixth and final task handles the Modbus Slave communications via the
hardware UART (Universal Asynchronous Receiver/Transmitter), which handles buffered character
reception and transmission over the 57,600 bps serial link. Static checking was routinely employed to
help enforce best-practice coding techniques during development of the application software, which
also included application of the MISRA rules and related guidelines [12–14].
Sensors 2019, 19, 3781 7 of 16
Sensors 2019, 19, x FOR PEER REVIEW 7 of 16

Figure
Figure 5. Main
5. Main software
software elements
elements employed
employed in the
in the system.
system.

The firstthe
Although task was required
dynamics of thetomonitored
extract thesystem
five ADC are channel
relatively conversion
slow moving, readings from the AD7856
the efficiency of the
via an is
observer SPIrelated
link to(inobtain
part) the input
to the signals.
selected The second
sampling rate, as task
thewascloserthen therequired
observertoapproximates
filter the signals
a
using recursive first-order low pass filters, and convert these readings
continuous function the better the performance will be [9]. For this reason, as described above the into appropriate engineering
units. sampling
selected The thirdrate taskwasevaluates the observer
100 Hz, with models, implementing
cut-off frequencies for the input digital equations
filters(1–4).
selectedBased
as 10onHz.the
Thefiltered outputs of the observer,
local communications were updatedthe fourth
everytask drives
100 ms. the local
In order indicator
to meet status LED’s.
the real-time To allow
constraints of thethe
observer atonon-preemptive
application, converge upon power
Earliest on / First
Deadline reset, the software
(npEDF) task scheduler is required
was employedto suppress
to controlany
thefaulty/degrading outputs forThis
timing of task executions. a user-adjustable
was because ofperiod, the very typically no moreitthan
low overheads 10 s. (specifically,
requires The fifth task
updates
very the main system
low RAM/ROM and CPU state machine once
overheads, with every
no shared 50 ms. The sixth
resource and final
handling task handles
protocol needed), the Modbus
coupled
Slave communications
with–amongst other things-itsvia the hardware
optimality amongUART the (Universal
non-idling,Asynchronous
non-preemptive Receiver/Transmitter),
schedulers and its
which handles
suitability buffered
for use with both character
periodic and reception
sporadic and
taskstransmission
[15]. In suchover the 57,600
a system, bps serial
however, link. Static
task execution
times are required to be comparatively short to prevent undue blocking occurring. Tasks one toduring
checking was routinely employed to help enforce best-practice coding techniques five
development
were of the application
created as periodic tasks, with software,
the required which also included
periods. As waiting application
for the UART of the MISRA
transmit or rules
receiveand
relatedtoguidelines
register be available,[12–14].
task six (the serial link update task) was created as a sporadic multi-stage
task [12], Although the dynamics
with transmission andofreception
the monitoredbufferingsystem are relatively
employed along with slowthe moving,
16-bytethe efficiency of the
transmit/receive
observer
FIFO on the is related
C167 (in part)
hardware. Theto software
the selectedwas sampling
written inrate, as the ‘C’,
embedded closer andthe observer approximates
compiled/linked with the a
continuous function the better the performance will be [9]. For this
Keil xC16x development toolchain. Timing was verified using the sufficient schedulability condition reason, as described above the
selected
based on CPUsampling rate was
utilization 100 Hz,derived
for npEDF with cut-off frequencies
previously for the input
[16]. Further digital of
descriptions filters selected as 10
the verification
of Hz.
the The local communications
functional were updated
and timing properties of the every
software 100 ms.
for Intheorder to meet
prototype the real-time
edge device have constraints
been
of the application, a non-preemptive
documented in an earlier work [17]. Earliest Deadline First (npEDF) task scheduler was employed
to control the timing of task executions. This was because of the very low overheads it requires
3.3.(specifically,
Communications very Architecture
low RAM/ROM and CPU overheads, with no shared resource handling protocol
needed), coupled with – amongst otherarchitecture
An overview of the communications things - its of optimality
the proposed among the non-idling,
system non-preemptive
is shown schematically in
schedulers and its suitability for use with both periodic and sporadic
Figure 6. As mentioned, the device software and hardware configuration employs a 56,700 bps TIA-232 tasks [15]. In such a system,
however,
Serial link fortask execution times The
communications. are required to be comparatively
device supports short to prevent
the Modbus Supervisory Control undueandblocking
Data
occurring.(SCADA)
Acquisition Tasks one to five were
protocol, created as periodic
and implements a Modbus tasks,
RTUwithslavethein required
software.periods.
When valid As waiting
Modbusfor
the UART
requests transmit
arrive, or receive
a software register the
task handles to berequests
available, and task six (thethe
prepares serial link update
required task) A
responses. was created
cellular
as a sporadic multi-stage task [12], with transmission and reception
Serial/GSM gateway provided (remote) IP connectivity to allow integration into the IoT. The central buffering employed along with
the 16-byte
control stationtransmit/receive
implements a Modbus FIFO onRTU the C167
masterhardware.
implemented The software
in software, was with
written in embedded ‘C’,
periodic/sporadic
and compiled/linked with the Keil xC16x development toolchain. Timing was verified using the
sufficient schedulability condition based on CPU utilization for npEDF derived previously [16].
 An overview of the communications architecture of the proposed system is shown schematically
in Figure 6. As mentioned, the device software and hardware configuration employs a 56,700 bps TIA-
232 Serial link for communications. The device supports the Modbus Supervisory Control and Data
Acquisition (SCADA) protocol, and implements a Modbus RTU slave in software. When valid
Modbus requests arrive, a software task handles the requests and prepares the required responses.
Sensors 2019, 19, 3781 8 of 16
A cellular Serial/GSM gateway provided (remote) IP connectivity to allow integration into the IoT.
The central control station implements a Modbus RTU master implemented in software, with
periodic/sporadic
Modbus transmissions Modbus transmissions and
and retransmissions retransmissions
to slaves handled using to existing
slaves handled usingscheduling
master-slave existing
master-slave
techniques [18].scheduling
A virtualtechniques
COM port [18]. A virtual
(vCOM) COM
driver port (vCOM)
provides a gatewaydriver provides
to carry a gateway
the Modbus to
RTU
carry
frames theover
Modbus
InternetRTU frames
Protocol andover Internet
across Protocol
the Cellular and across
network. In thethe Cellulardesign,
prototype network. In the
the OnCell
prototype
G3150A-LTE design,
Serialthe OnCell from
Gateway Moxa® was
G3150A-LTE Serial Gatewaytofrom
employed Moxacellular
provide ® was employed to provide
IP connectivity in the
cellular
prototype IP Edge
connectivity
device, in
withtheend-to-end
prototype 256-bit
Edge device, withenabled.
encryption end-to-end
This256-bit encryption
provided end-to-endenabled.
secure
This provided
duplex end-to-end securetoduplex
serial communications serial
carry the communications
Modbus to carry
traffic between the the Modbus
network traffic
Core and between
the Edge
the network
device. Coredevelopments,
In future and the Edge flexible
device. scheduling
In future developments, flexible scheduling
of Core-Edge communications andof Core-Edge
off-loading of
communications and off-loading of analytics workload between the Core and
analytics workload between the Core and Edges will be implemented using state-of-the-art techniques, Edges will be
implemented
such as thoseusing state-of-the-art
described in [19]. techniques, such as those described in [19].

Figure
Figure6.6.Overall
Overallcommunications
communicationsarchitecture
architectureofofthe
theproposed
proposedsystem.
system.

4.4.Offline
OfflineValidation
Validation Testing
Testing
Hardware-In-The-Loop
Hardware-In-The-Loop(HIL)(HIL)testing
testingas asaameans
meansforforboth
bothearly
earlyand
andlate-stage
late-stagesystem
systemvalidation
validation
in
inreal-time
real-timeand
andembedded
embedded systems
systems is now
is nowa well-established
a well-established concept [12,20,21].
concept The principle
[12,20,21]. of HIL
The principle of
simulation of an of
HIL simulation embedded system
an embedded is illustrated
system in Figure
is illustrated 7; the7;unit
in Figure under
the unit test test
under (UUT) executes
(UUT) on
executes
representative hardware,
on representative andand
hardware, its outputs
its outputs areare
fedfeddirectly
directlyto to
the
thesimulator.
simulator.The
TheUUTUUToutputs
outputsareare
sampled
sampledandandused
usedasas
input variables
input variablesto atohost dynamic
a host simulation
dynamic simulationmodel, which
model, is evaluated
which in real-
is evaluated in
time. The simulation
real-time. outputs,
The simulation which
outputs, are are
which synthesized
synthesizedrepresentations
representations of of
signals
signalsinternal
internaltotothe
the
dynamic
dynamicmodel,
model,are
arethen
thenfed
fedback
backto to the
the UUT
UUT thereby
thereby closing
closing thethe control
control loop.
loop. Since
Sincethe
theUUT
UUTisis
representative of the final system implementation, many types of specification defect, omissions or
otherwise unexpected interactions that may result in failures or otherwise unacceptable behaviors may
be removed before the system enters operational service. However, the simulation must be designed
and implemented correctly in order for results to be reliable. If this can be achieved, then HIL testing
can be regarded as a representative virtualization technique which allows a potentially large design
space to be explored and tested, with few (if any) consequences should the UUT malfunction or behave
inappropriately at run-time.
otherwise unexpected interactions that may result in failures or otherwise unacceptable behaviors
representative of the final system implementation, many types of specification defect, omissions or
may be removed
otherwise before
unexpected the system
interactions thatenters operational
may result service.
in failures However,
or otherwise the simulation
unacceptable must be
behaviors
designed
may be removed before the system enters operational service. However, the simulation must bethen
and implemented correctly in order for results to be reliable. If this can be achieved,
HIL testingand
designed canimplemented
be regardedcorrectly
as a representative
in order for virtualization technique
results to be reliable. If thiswhich allows
can be a potentially
achieved, then
large
Sensors design
2019, 19, space
3781 to be explored and tested, with few (if any) consequences
HIL testing can be regarded as a representative virtualization technique which allows a potentially should the 9UUT
of 16
malfunction or behave inappropriately at run-time.
large design space to be explored and tested, with few (if any) consequences should the UUT
malfunction or behave inappropriately at run-time.

Figure 7. Overview of Hardware-In-The-Loop (HIL) testing of embedded real-time systems.

Figure 7. Overview of Hardware-In-The-Loop (HIL) testing of embedded real-time systems.
Figure 7. Overview of Hardware-In-The-Loop (HIL) testing of embedded real-time systems.
As mentioned, during development of the observer equations, detailed models of the pumping
As mentioned,
Aswere
mentioned, during development
developmentofofthe theobserver
observerequations,
equations, detailed models of the pumping
process createdduring
for validation purposes. The popular designdetailed models
and analysis of the
package pumping
Simulink©
process
process were created
were created for validation purposes.
forimplementation.
validation purposes. The popular design and analysis package Simulink©
was employed for their In The
order popular
to allowdesign
HILand analysis
testing package
of the Simulink©
edge device to take
was
was employed
employed for
for their
their implementation.InInorder
implementation. ordertotoallow
allow HIL
HIL testing
testing of of the
the edge
edge device
device to to take
take place,
place,
place, the Simulink models of the process were employed along with real-time model simulation and
the
the Simulink
Simulink models
models of the process
the process were
wereemployed
employedalong alongwith with real-time
real-time model
model simulation
simulation and and
interface devices provided by dSpace©. This allowed three scenario’s (baseline, coolant flow fault
interface devices provided by dSpace©. This allowed three scenario’s (baseline,
interface devices provided by dSpace©. This allowed three scenario’s (baseline, coolant flow fault coolant flow fault
and bearing friction fault) to be validated on the embedded implementation. A screenshot of the
and
andbearing
bearingfriction
friction fault) toto be
be validated
validatedon onthetheembedded
embedded implementation.
implementation. A screenshot
A screenshot of HIL
of the the HIL
HIL user interface and testing procedure is as shown in Figure 8. In addition to simulation of the
userinterface
user interface and
and testing procedure
procedureisisasasshown
shownininFigure
Figure 8. 8.
InInaddition
addition to simulation
to simulationof the bearing
of the bearing
bearing unit itself, type k thermocouples along with signal transmitters and signal isolators of the same
unit itself, type k thermocouples along with signal transmitters and signal
unit itself, type k thermocouples along with signal transmitters and signal isolators of the same isolators of the same
characteristics as employed in the field trials (to be described in the following Section) were simulated
characteristicsas
characteristics as employed
employed in inthe
thefield
fieldtrials
trials(to
(tobebedescribed
described ininthethe
following
following Section) were
Section) simulated
were simulated
as a part of the overall model.
asasaapart
partofofthe
the overall
overall model.
model.

Figure 8.
Figure 8. HIL
HILtesting
testingofofthe
theprototype
prototypeedge
edgedevice onon
device a C167 processor.
a C167 processor.
Figure 8. HIL testing of the prototype edge device on a C167 processor.
Figure99shows
Figure shows data
data captured
captured comparing
comparingthe theSimulink-based
Simulink-based observer
observerimplementation
implementation withwith
the the
edge device
edge Figure observer
device9observer
shows data implementation during
captured comparing
implementation the
during the baseline scenario.
the Simulink-based
baseline scenario. Figure
observer 10 shows data
Figureimplementation captured
with the
10 shows data captured
comparing
edge device the
observerSimulink-based
implementation observer
during implementation
the baseline with
scenario. the
Figure
comparing the Simulink-based observer implementation with the embedded observer implementation embedded
10 shows observer
data captured
implementation
comparing the during the coolant flow
Simulink-based fault scenario.
observer In this fault scenario,
implementation withcoolantthe coolant
the embeddedflow rate is
observer
during the coolant flow fault scenario. In this fault scenario, the flow rate is dropped
dropped significantly
implementation 2150 s into the simulation run, and remains as such for a duration of 600 s. As
significantly 2150during thesimulation
s into the coolant flow run,fault
andscenario.
remains asIn such
this fault
for a scenario,
duration of the600
coolant flowberate
s. As can seenis
dropped significantly 2150 s into the simulation run, and remains as such for a duration
in the captured results, although the coolant flow is not a directly instrumented parameter, the observer of 600 s. As
implementation accurately tracks the coolant temperature and estimates the reduced coolant flow
condition. Figure 11 shows data captured comparing the Simulink-based observer implementation
with the embedded observer implementation during the coolant flow fault scenario. In this fault
scenario, the bearing friction factor µ is allowed to drift upwards starting 1000 s into the simulation
run, and increases slowly over time thereafter. As can be seen in the captured results, although the
 reduced
can be seen coolant
in theflow condition.
captured Figure
results, 11 shows
although data captured
the coolant flow is comparing
not a directlythe Simulink-based
instrumented
observer implementation
parameter, with the embedded
the observer implementation observer
accurately tracksimplementation during theand
the coolant temperature coolant flow the
estimates fault
scenario. In this fault scenario, the bearing friction factor μ is allowed
reduced coolant flow condition. Figure 11 shows data captured comparing the Simulink-based to drift upwards starting 1000
s into the simulation run, and increases slowly over time thereafter. As can
observer implementation with the embedded observer implementation during the coolant flow fault be seen in the captured
results, In
scenario. although
this faultthescenario,
bearingthe friction factor
bearing μ is factor
friction a non–measurable
μ is allowed tomodel
drift parameter, the observer
upwards starting 1000
Sensors 2019, 19, 3781 10 of 16
implementation detects
s into the simulation run, and starts to track the gradual increase in the parameter, albeit
increases slowly over time thereafter. As can be seen in the captured with a small
offset. although the bearing friction factor μ is a non–measurable model parameter, the observer
results,
bearing friction factor
implementation detectsµ is a non–measurable
and starts to track themodel parameter,
gradual increase the observer
in the implementation
parameter, albeit with detects
a small
and starts
offset. to track the gradual increase in the parameter, albeit with a small offset.

Figure 9. Comparative analysis of the Simulink-based and edge device-based observers (baseline
case).
Figure 9.9.Comparative
Figure Comparativeanalysis
analysisof of
thethe
Simulink-based andand
Simulink-based edgeedge
device-based observers
device-based (baseline
observers case).
(baseline
case).
35
Temperature oC
C Outlet

30
25 Measured
35
Coolant

Observer Estimate
20
Coolant Outlet
o

30
Temperature

15 Measured
25 0 500 1000 1500 2000 2500 3000 3500
Observer Estimate
20
15 0
Temperature

0 500 1000 1500 2000 2500 3000 3500

error oC

-0.02
0
Temperature

-0.04
(kg/s) Mass error oC

-0.02 0 500 1000 1500 2000 2500 3000 3500

-0.041.5
Flow Rate (kg/s)

01 500 1000 1500 2000 2500 3000 3500

Flow Rate Coolant

1.50.5 Observer Estimate

Coolant Mass

Actual
1 0
0 500 1000Observer1500
Estimate 2000 2500 3000 3500
0.5
Actual Time (s)
0
Figure 10. Comparative
0 analysis
500 of the Simulink-based
1000 1500 and
2000edge device-based
2500 3000 (coolant
3500flow fault case).
Figure 10. Comparative analysis of the Simulink-based
Time (s)
and edge device-based (coolant flow fault
case).
In all tested scenarios, the results obtained indicated that the observer implementations were
Figure
accurate 10. Comparative
enough analysis
for the intended of the Simulink-based
purpose, giving resultsand edgeindistinguishable
almost device-based (coolant
fromflow
the fault
(known)
case).
internal variables in the Simulink model–with less than 5% maximum and 1% average (mean) errors
recorded for the estimated quantities. As such, testing progressed to field trials.
Sensors 2019, 19, 3781 11 of 16
Sensors 2019, 19, x FOR PEER REVIEW 11 of 16

35

Bearing Temperature oC
34 Actual
33 Observer Estimate

32
31
30
1000 1500 2000 2500 3000

0.015
Temperature error oC

0.01

0.005
1000 1500 2000 2500 3000

-3
x 10
5.5
Friction Factor Estimate

Observer Estimate
5 Actual

4.5

4
1000 1500 2000 2500 3000
Time (s)

Figure
Figure 11. Comparative analysis
11. Comparative analysis of
of the
the Simulink-based
Simulink-based and
and edge
edge device-based
device-based observers
observers (bearing
(bearing
friction factor fault case).
friction factor fault case).
5. Online Validation Testing
In all tested scenarios, the results obtained indicated that the observer implementations were
accurate enoughthe
Following forsuccessful off-line
the intended validation
purpose, testing,almost
giving results implementation tests offrom
indistinguishable the edge device
the (known)
were then performed on Sulzer© Water Pumps at Lobwood pumping station, West Yorkshire,
internal variables in the Simulink model – with less than 5% maximum and 1% average (mean) errors UK.
As shownfor
recorded in Figure 12, the quantities.
the estimated pump house Ascontains threeprogressed
such, testing pumps; two toof which
field are active at any point
trials.
in time, with the other being ready as standby in case failure. The active pumps are rotated on a
scheduled basis, allowing
5. Online Validation routine checks and maintenance to be carried out on the third. The use of the
Testing
observer implementation to detect bearing wear and coolant flow failure will clearly provide guidance
Following the successful off-line validation testing, implementation tests of the edge device were
allowing optimization of the maintenance schedule, helping to prevent both active pump failures and
then performed on Sulzer© Water Pumps at Lobwood pumping station, West Yorkshire, UK. As
unnecessary changeover and maintenance. During online testing, the observer implementation was
shown in Figure 12, the pump house contains three pumps; two of which are active at any point in
deployed on pumps two and three as indicated in Figure 12.
time, with the other being ready as standby in case failure. The active pumps are rotated on a
The Input/Output schedule for the observer implementation was as shown in Figure 13.
scheduled basis, allowing routine checks and maintenance to be carried out on the third. The use of
Temperatures were measured via externally mounted, low-cost type-k thermocouples. ALM-46
the observer implementation to detect bearing wear and coolant flow failure will clearly provide
low-cost head-mount transmitters were employed to convert effective signal ranges to current
guidance allowing optimization of the maintenance schedule, helping to prevent both active pump
transmission, and COM-3B signal isolators were employed to convert to current to voltage prior to
failures and unnecessary changeover and maintenance. During online testing, the observer
conversion by the ADCs. The calibration of the combined sensor, transmitter, isolator and ADC for each
implementation was deployed on pumps two and three as indicated in Figure 12.
temperature measurement channel was as follows: [0 ◦ C:100 ◦ C] → [4 mA:20 mA] → [0 V DC:5 V DC]
→ [0000h :0x3FFFh ]. Stated combined accuracies of the transmitter and isolator (due to linearization)
were ±0.1% and ±0.15% of full-scale max, and the combined accuracy of the ADC converter was
±0.0015% max. Although a full meteorological analysis is deferred to future work, the accuracy of
these measurement chains seems adequate to reproduce results of similar accuracy in the prototype
field trials as in the laboratory-based HIL test results, as models of the same thermocouples, signal
transmitter and signal isolator were employed in the laboratory simulations. During testing, a portable
ultrasonic flow meter was used to measure coolant mass flow rates and hence help to calibrate the
observer. Where appropriate, information on pump speed and power was obtained directly from the
electronic switchgear. The observer device was mounted in an IP67 enclosure with input and output
connections. The observer was installed on-site for long-term trials with data recorded to a standard
Sensors 2019, 19, 3781 12 of 16

desktop via the TIA-232 point-to-point connection, as shown in Figure 14. In total, neglecting wiring
costs, component costs for the encapsulated observer implementation were ≈ £200.
Sensors 2019, 19, x FOR PEER REVIEW 12 of 16

Figure 12.
Figure 12. Arrangement
Arrangement of of pumps inside pumping
pumps inside pumping house,
house, showing
showing pump
pump 22 (left),
(left), pump
pump 33 (middle),
(middle),
and pump 1 (right). Pipework
Sensors 2019, 19, x FOR PEERPipework is housed
REVIEW is housed out of view under the metallic walkway. 13 of 16

The Input/Output schedule for the observer implementation was as shown in Figure 13.
Temperatures were measured via externally mounted, low-cost type-k thermocouples. ALM-46 low-
cost head-mount transmitters were employed to convert effective signal ranges to current
transmission, and COM-3B signal isolators were employed to convert to current to voltage prior to
conversion by the ADCs. The calibration of the combined sensor, transmitter, isolator and ADC for
each temperature measurement channel was as follows: [0 °C : 100 °C] → [4 mA : 20 mA] → [0 V DC :
5 V DC] → [0000h : 0x3FFFh]. Stated combined accuracies of the transmitter and isolator (due to
linearization) were ±0.1% and ±0.15% of full-scale max, and the combined accuracy of the ADC
converter was ±0.0015 % max. Although a full meteorological analysis is deferred to future work, the
accuracy of these measurement chains seems adequate to reproduce results of similar accuracy in the
prototype field trials as in the laboratory-based HIL test results, as models of the same thermocouples,
signal transmitter and signal isolator were employed in the laboratory simulations. During testing, a
portable ultrasonic flow meter was used to measure coolant mass flow rates and hence help to
calibrate the observer. Where appropriate, information on pump speed and power was obtained
directly from the electronic switchgear. The observer device was mounted in an IP67 enclosure with
input and output connections. The observer was installed on-site for long-term trials with data
recorded to a standard desktop via the TIA-232 point-to-point connection, as shown in Figure 14. In
total, neglecting wiring costs, component costs for the encapsulated observer implementation were ≈
£200. Figure 13. I/O
Figure13. I/OSchedule
Schedulefor
foronline
onlinetesting
testingof
ofthe
thedevice
deviceimplementation.
implementation.

As shown in Figure 15, the online site trial for Pump 3 (fixed speed) took place over a period
of 7 days in which the CM system ran uninterrupted. Tracking of the observer was highly effective,
with the estimated coolant flow remaining constant except during shut down periods (day 2 and
day 4/5), when the flow rate fell to zero following the bearing temperature cooling (as expected).
The bearing friction factor remained constant at the expected level, again indicating fault free operation,
aside from during the shutdown periods when it fell to zero (as expected).
Sensors 2019, 19, 3781 13 of 16
Figure 13. I/O Schedule for online testing of the device implementation.

Figure 14. Device implementation in IP67 enclosure (bottom right), interface to local PC (bottom left)
instrumentation/communication interfaces
and instrumentation/communication interfaces (top
(top right).
right).

Figure 16 shows the online site trial for Pump 2 (variable speed) which again took place over
a period of 7 days in which the CM system ran uninterrupted. Tracking of the observer was again
just as effective, with the estimated coolant flow remaining piecewise constant, tracking small speed
changes at several points during the trial. During the course of the experiment, pump trips occurred
eight times; during the longer of these trip events (occurring 8000 s into the trial) the estimated flow
rate fell to zero following the bearing temperature cooling (as expected). The bearing friction factor
remained constant, again indicating fault free operation independent from pump load, aside from
during the longer trip event when it fell considerably (as expected). In summary, both the on-line
and off-line simulations and tests demonstrated that the observer-based edge device functioned as
expected, and that parameter estimations were within acceptable limits for the proposed application.
During the course of the field trials, however, it was noted that some simple improvements to fault
detection and annunciation logic will be required for some specific situations when progressing from
trials to continuous operation; most notably, supressing fault detection and annunciation during
shut-down and trip conditions. These, and other improvements, are currently being implemented,
and a full metrological analysis of the modified system is underway. Overall, however, as the
device is a relatively low-cost solution (as mentioned approximately £200, excluding cabling, gateway,
and network access costs) in comparison with the value of the plant, this gives a very promising
indication of its overall suitability.
Sensors 2019, 19, 3781 14 of 16
Sensors 2019, 19, x FOR PEER REVIEW 14 of 16

C oolant O utlet
30 B earing

Temps oC
Inlet
A m bient
20

10
0 1 2 3 4 5 6 7
Power Signal, V

0.5

0
1 2 3 4 5 6

0.1
Observer Errors, oC

0
Coolant Outlet Temperature
Bearing Temperature
-0.1
0 1 2 3 4 5 6 7

1.00
Estimate, l/sec
Coolant Flow

0.75

0.50

0.25

0
0 1 2 3 4 5 6 7

4
Factor Estimate
Bearing Friction

2
x1000

-2
0 1 2 3 4 5 6 7
Time (Days)
Sensors 2019, 19, x FOR PEER REVIEW 15 of 16
Figure 15. Online
Figure15. Online edge
edge device
device implementation
implementation site
sitetrials
trialsresults
resultsdata:
data: Pump
Pump 33 (fixed
(fixedspeed).
speed).
40
Temperatures

As shown in Figure 15, the online site trial for Pump 3 (fixed speed) took place over a period of
7 days in which the20CM system ranCoolant Outlet
(oC)

uninterrupted.
Bearing Tracking of the observer was highly effective, with
Inlet
the estimated coolant flow remaining constant except during shut down periods (day 2 and day 4/5),
Ambient
0
when the flow rate fell 0 to1000 2000
zero following 3000the 4000 bearing 5000 6000
temperature 7000 8000(as 9000
cooling 10000 The bearing
expected).
friction factor remained
4 constant at the expected level, again indicating fault free operation, aside
and Power, V
Pump Speed

from during the shutdown periods Speed when it fell to zero (as expected). Pump Trips
2 Pow er
Figure 16 shows the online site trial for Pump 2 (variable speed) which again took place over a
period of 7 days in0which the CM system ran uninterrupted. Tracking of the observer was again just
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
as effective, with the estimated coolant flow remaining piecewise constant, tracking small speed
changes at several0.1points during the trial. During the course of the experiment, pump trips occurred
Errors (oC)
Observer

eight times; during0 the longer of these trip events (occurring 8000 s into the trial) the estimated flow
rate fell to zero following the bearing temperature
Coolant
Bearing
Outlet cooling (as expected). The bearing friction factor
-0.1
remained constant, 0again indicating
1000 2000 fault
3000 free operation
4000 5000 independent
6000 7000 from pump
8000 9000 load, aside from
10000
during the longer trip2
event when it fell considerably (as expected). In summary, both the on-line and
Estimate, (kg/s )

off-line simulations and tests demonstrated that the observer-based edge device functioned as
Mass Flow

1
expected, and that parameter estimations were within acceptable limits for the proposed application.
During the course 0of the field trials, however, it was noted that some simple improvements to fault
detection and annunciation
0 1000logic will be
2000 3000required 4000 for some6000
5000 specific
7000situations
8000 when
9000 progressing
10000 from
trials to continuous operation; most notably, supressing fault detection and annunciation during
5

shut-down and trip conditions. These, and other improvements, are currently being implemented,
delta a1
x1000

0
and a full metrological analysis of the modified system is underway. Overall, however, as the device
is a relatively low-cost
-5
solution (as mentioned approximately £200, excluding cabling, gateway, and
0 1000 2000 3000
network access costs) in comparison with the4000 5000
valueTime,
of the
(s)
6000
plant, 7000
this 8000
gives 9000
a very 10000
promising indication
of its overall suitability.
Figure 16. Online edge device implementation site trials results data: Pump 2 (variable speed).

6. Conclusions and Further Work

In this paper, developments towards digitization in the water industry have been presented.
Specifically, the development, validation, and field-testing of a real-time edge device as part of a
CM/PM system for deployment upon large-scale pumping equipment in the water industry has been
Sensors 2019, 19, 3781 15 of 16

6. Conclusions and Further Work

In this paper, developments towards digitization in the water industry have been presented.
Specifically, the development, validation, and field-testing of a real-time edge device as part of a
CM/PM system for deployment upon large-scale pumping equipment in the water industry has
been described. Initial field trials with the observer-based edge device have been very promising.
In addition, simulation results and accuracy analysis indicate the system is fit-for-purpose. Future
work will describe longer-term experiences and analyze the fault-detection capability of the proposed
embedded solution. Future work will also describe developments related to integration of multiple
CM edge devices to the network core, and progress the analytics required for optimized operation and
PM of the pumping equipment within a wider Industry 4.0 framework.

Author Contributions: Conceptualization, M.S. and J.T.; methodology, M.S. and J.T.; software, M.S.; validation,
M.S. and J.T.; formal analysis, M.S.; investigation, M.S. and J.T.; data curation, J.T.; writing—original draft
preparation, M.S.; writing—review and editing, M.S.; visualization, M.S. and J.T.; project administration, M.S. and
J.T.; funding acquisition, J.T.
Funding: This research was partly funded by a private industrial research contract with Yorkshire Water PLC.
Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design of the
study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to
publish the results

References
1. Isaksson, A.J.; Harjunkoski, I.; Sand, G. The impact of digitalization on the future of control and operations.
Comput. Chem. Eng. 2018, 114, 122–129. [CrossRef]
2. Dinardo, G.; Fabbiano, L.; Vacca, G. A smart and intuitive machine condition monitoring in the Industry 4.0
scenario. Measurement 2018, 126, 1–12. [CrossRef]
3. Mahdavinejad, M.S.; Rezvan, M.; Barekatain, M.; Adibi, P.; Barnaghi, P.; Sheth, A.P. Machine learning for
Internet of Things data analysis: A survey. Digit. Commun. Netw. 2018, 4, 161–175. [CrossRef]
4. Lei, Y. Intelligent Fault Diagnosis and Remaining Useful Life Prediction of Rotating Machinery;
Butterworth-Heinemann: Oxford, UK, 2016.
5. Tandon, N.; Choudhury, A. A review of vibration and acoustic measurement methods for the detection of
defects in rolling element bearings. Tribol. Int. 1999, 32, 469–480. [CrossRef]
6. Baydar, N.; Ball, A. Detection of gear failures via vibration and acoustic signals using wavelet transform.
Mech. Syst. Signal Process. 2003, 17, 787–804. [CrossRef]
7. Luo, M.; Li, C.; Zhang, X.; Li, R.; An, X. Compound feature selection and parameter optimization of ELM for
fault diagnosis of rolling element bearings. ISA Trans. 2016, 65, 556–566. [CrossRef] [PubMed]
8. Liu, H.; Zhou, J.; Zheng, Y.; Jiang, W.; Zhang, Y. Fault diagnosis of rolling bearings with recurrent neural
network-based autoencoders. ISA Trans. 2018, 77, 167–178. [CrossRef] [PubMed]
9. Edwards, C.; Spurgeon, S.K. Sliding Mode Control: Theory and Applications; Taylor and Francis: Abingdon,
UK, 1998.
10. Edwards, C.; Spurgeon, S.K.; Patton, R.J. Sliding mode observers for fault detection and isolation. Automatica
1999, 36, 541–553. [CrossRef]
11. Kitson, C.; Spurgeon, S.K.; Twiddle, J.A.; Jones, N.B. On Sliding Mode Observer Techniques for Monitoring of
Rotary Machines. In Proceedings of the 19th International Congress and Exhibition on Condition Monitoring
and Diagnostic Engineering Management (COMADEM 2006), Lulea, Sweden, 12–15 June 2006.
12. Short, M. Development Guidelines for Dependable Real-Time Embedded Systems. In Proceedings of the 6th
IEEE/ACS International Conference on Computer Systems and Applications, Doha, Qatar, 31 March–4 April
2008; pp. 1032–1039.
13. The Motor Industry Software Reliability Association (MISRA). Guidelines for the Use of the C Language in
Vehicle Based Software; The Motor Industry Research Association: Nuneaton, Warwickshire, UK, 1994.
14. Holzmann, G.J. The Power of Ten: Rules for Developing Safety Critical Code. IEEE Comput. 2006, 39, 93–95.
Sensors 2019, 19, 3781 16 of 16

15. Short, M. The Case for Non-Preemptive, Deadline-Driven Scheduling in Real-Time Embedded Systems.
In Lecture Notes in Engineering and Computer Science: Proceedings of the World Congress on Engineering 2010
(WCE 2010), London, UK, 30 June–2 July 2010; Newswood Limited: Hong Kong, China; Volume 1, pp. 399–404.
16. Short, M. Improved schedulability analysis of implicit deadline tasks under limited preemption EDF
scheduling. In Proceedings of the 16th IEEE International Conference on Emerging Technologies and Factory
Automation (ETFA 2011), Toulouse, France, 5–9 September 2011.
17. Short, M.; Twiddle, J.; Schwarz, M. Verification and Validation Testing of a Real-Time Fault-Detection System
for Mission-Critical Rotating Equipment. In Proceedings of the 24th International Congress on Condition
Monitoring and Diagnostic Engineering Management (COMADEM 2011), Stavanger, Norway, 30 May–1
June 2011; pp. 1703–1712.
18. Short, M. Eligible earliest deadline first: Server-based scheduling for master-slave industrial wireless
networks. Comput. Electr. Eng. 2017, 64, 305–321. [CrossRef]
19. Kang, J.; Eom, D.-S. Offloading and Transmission Strategies for IoT Edge Devices and Networks. Sensors
2019, 19, 835. [CrossRef] [PubMed]
20. Short, M.; Pont, M.J.; Fang, J. Assessment of performance and dependability in embedded control systems:
Methodology and case study. Control Eng. Pract. 2008, 16, 1293–1307. [CrossRef]
21. Storey, N. Safety Critical Computer Systems; Addison Wesley Publishing: Boston, MA, USA, 1996.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).