2019 IEEE International Conference on Vehicular Electronics and Safety (ICVES)
System Identification using LMS, RLS, EKF and
Neural Network
Gökhan Akgün, Habib ul Hasan Khan, Marawan Hebaish and Diana Göhringer
Technische Universität Dresden, Germany
{goekhan.akguen, habib.khan, marawan.hebaish, diana.goehringer}@tu-dresden.de
Abstract—Model accuracy is the most important step towards
efficient control design. Various system identification techniques
exist which are used to identify model parameters. However,
these techniques have their merits and demerits which need to
be considered before selecting a particular system identification
technique. In this paper, we compared different types of system
identification techniques and used them to identify our DC-
motor use-case. Using the identified system, we designed different
discrete PI controllers in order to investigate the system response.
We concluded EKF provided the best performance in terms of
parameter accuracy and convergence rate.
Index Terms—PID, EKF, LMS, RLS, NARX, System Identifi-
cation Algorithms
I. I NTRODUCTION
Real applications need to be expressed through mathe-
matical equations in simulation tools but all parameters are
not available in datasheets of actuators. This hinders the
modeling of the system. Because of this, an inaccurate or
non-deterministic effect occurs when a designed controller is
connected to a real system. As soon as the user defines a
different set point, an oscillation arises in the closed control
loop of the system. Even though the actuator parameters
exist, an approximation of a complex system is done to ease
the development phase of the controller. This approach is
called white-box modelling and expresses the mathematical
equations through transfer functions. The grey-box or black-
box modelling is deployed when the parameters of a system
are not known. Both techniques provide different approaches
to model the behaviour of the real system. Researchers know
the dynamics of the systems as prior knowledge and use
experimental data to identify its parameters. On the other
hand, black-box modelling is parameterizing a model class
through statistical analysis [1], [2]. In both approaches, system
identification techniques are used to identify the system param-
eters with its dynamics in a transfer function or generating a
statistical model. It identifies the system accordingly to its
input and output signals. In doing so, the technique compares
the output of the system and minimizes the occurring error
to adapt the parameters [3]. This research work compares
different modelling techniques such as Least Mean Square
(LMS), Recursive Least Square (RLS), Extended Kalman
Filter (EKF) and Nonlinear Autoregressive Exogenous Model
(NARX) neural network on a DC motor. Furthermore, PI
controllers are implemented to investigate the response times
of each identified system. The rest of this paper is organized
as follows. In section II, we discuss and compare the related
978-1-7281-3473-4/19/$31.00
Authorized licensed use limited to: Universidad©2019
Nacional IEEE
de Colombia (UNAL). Downloaded on April 30,2021 at 21:50:19 UTC from IEEE Xplore. Restrictions apply.
work. Section III presents the plant description of the DC
motor. In section IV, each system identification technique will
be discussed. The results will be shown in section V. The
estimation accuracy of the identified parameters, the compu-
tational complexity and convergence rate of each algorithms
are discussed in this section. The paper will be concluded with
a comparison of the identification techniques in section VI.
II. R ELATED W ORK
Based on [4], the control parameters of a controller can be
fine-tuned at run-time. The work enabled to change the control
strategy and to find the best suited controller for real appli-
cations. The control parameters needed to be adapted due to
uncertainty or inaccuracy of the considered state-space model
in the simulation environment. Besides the control parameters,
system parameters can also be adapted through identification
techniques. The work can be improved through identifying the
system parameters accurately with identification algorithms.
This would eliminate the occurring inaccuracies and provide
a better tuning. A comparative study [5] is carried out to
identify vehicle’s nonlinear inertial parameters. The authors
compared two different identification methods namely RLS
and EKF. The authors stated that RLS may induce some esti-
mation errors due to its linear operations. Hence, the authors
concluded that EKF provides a better response for nonlinear
systems. The authors in [6] performed system identification
using different adaptive filters like LMS, NLMS and RLS.
The authors highlighted that RLS is more efficient because
of its greater convergence rate but it has more computational
complexity. In [7], the authors used KF and RLS to perform
parameter estimation on a DC-DC converter. The authors
presented that KF performs better than RLS when an abrupt
change in the load is applied in terms of estimation accuracy
and convergence time. The authors in [8] performed a state and
parameter estimation of an induction motor using an EKF. The
estimation results showed that the proposed methodology was
capable of accurate estimation of the stated parameters.
As covered in the last paragraphs, various authors have com-
pared different techniques like LMS, RLS and EKF with one
another. As per our knowledge, no researcher has compared
grey-box techniques with the black-box technique like NARX
which will be the main idea of this work.
 2019 IEEE International Conference on Vehicular Electronics and Safety (ICVES)

Ia If Noise

Ra La Rf

Plant +
+
Lf Vf G(z)
Va ω

d(k)

X(k) + ε(k)
-

Fig. 1. Equivalent circuit of separately excited DC motor y(k)

Model

III. S YSTEM IDENTIFICATION

DC motor is a widely used application in the field of control Adaptive Algorithm
system because of its simplicity and continuous control charac- (LMS,RLS)

teristics. It is a single input single output system (SISO) which

has high performance and ability to be linearly controlled Fig. 2. System identification using adaptive filters
through PID or LQR controllers [9], [10]. The input is the
armature voltage Va and the output is the angular velocity The measured input and output signals are described by the
ω as shown in Fig. 1. whereby the remaining parameters are regression vector X (Equation (4)).
listed in Table I. Equation (1) describes the physical dynamics
of the DC motor as a transfer function where ωn is the natural X T = [y(k − 1), ..., y(k − n), u(k), ..., u(k − m)] (4)
frequency, ζ is the damping ratio and K is the gain parameter In this work, LMS and RLS as adaptive filters are utilized
of the second order system. to identify the coefficients of DC motor and a case study for
Kt Kωn2 grey-box modelling.
G(s) = JR+Lb Rb+Kt Ke
= (1) 1) LMS Algorithm: LMS is considered to be a stochastic
2
s + JL s + JL
s2 + 2ζωn s + ωn2
gradient algorithm which is the most used adaptive filtering
In this research work, a white noise of 60 dB signal to noise algorithm due to its low computation complexity [12]. It
ratio is applied to emulate a real behaviour and the sample updates the filter weights in each iteration to minimize the
time is set to 0.01 seconds in the simulation. error k according to Equation (5) where µ is the step size
and dk is the desired signal from the plant. It controls the
A. Adaptive Filters convergence rate of the filter [13].

Adaptive filters are considered to be suited for time-variant Wk+1 = Wk + µ [dk − (WkT Xk )] Xk (5)
| {z }
systems because their weights attempt to converge to the k
unknown system transfer function coefficients through an The step size should be chosen in a range of 0 ≤ µ ≤ 2/λmax
optimization technique as shown in Fig. 2 [6], [11]. The to have a higher convergence probability where λmax is the
coefficients of a discrete linear transfer function (Equation (2)) maximum eigenvalue of the autocorrelation matrix R [14]. A
are identified by means of minimizing the error. lower step size leads to a better convergence probability of the
b0 + b1 z −1 + b2 z −2 + ... + bm z −m real parameter. But it requires also more time to identify this
G(z) = (2) parameter. The convergence rate can be improved through a
1 + a1 z −1 + a2 z −2 + ... + an z −n
higher step size but the parameter may not converge to the
Each coefficient is summarized in the weight vector W (Equa- real value [14].
tion (3)). 2) RLS Algorithm: RLS is a recursive method of the least
W T = [−a1 , ..., −an , b0 , ..., bm ] (3) squares algorithm which aims to minimize the sum of the
squared error k as shown in Fig. 2. The gain Kk+1 is obtained
by Equation (6) which recursively predicts the weights with
TABLE I the decreasing error k+1 (Equation (7)).
PHYSICAL PARAMETERS OF THE DC MOTOR
Pk Xk+1
Kk+1 = T P X
(6)
Parameters Values λ + Xk+1 k k+1
Moment of inertia of the rotor (J) 0.05 kgm2
T
Viscous friction constant (b) 0.5 Nms Wk+1 = Wk + Kk+1 · [dk+1 − Xk+1 Wk ] (7)
Electromotive force constant(Ke ) 0.836 Vsrad−1 | {z }
Torque constant (Kt ) 0.836 NmA−1 k+1
Electric resistance (R) 1Ω
Electric inductance (L) 0.1 H In Equation (6), Pk is the covariance matrix and λ is the
forgetting factor. It can be chosen in a range of 0 ≤ λ ≤ 1

Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on April 30,2021 at 21:50:19 UTC from IEEE Xplore. Restrictions apply.
 2019 IEEE International Conference on Vehicular Electronics and Safety (ICVES)

u(k)

Q P0 Pk-1 R W1
u(k-1) Z-1
Pk Kk
Covariance Matrix Calculating Kalman Updating Covariance
Prediction Gain Matrix u(k-2) Z-1 W2

x0 Kk u(k-n) Z-1
x̂k x̂k y(k)
State Vector
Updating State Vector
Prediction
y(k-n) Z-1
uk-1 x̂k-1 yk
Delay

y(k-2) Z-1

Fig. 3. EKF algorithm

y(k-1) Z-1 b

[12]. The covariance matrix can be initialized as P0 =I (identity
matrix) since the system parameters are unknown at the first
iteration. Afterwards, the covariance matrix can be updated by
Equation (8) in each iteration.
1 T
Pk+1 = (Pk − Kk+1 Xk+1 Pk )
λ
B. Extended Kalman Filter
Kalman Filter (KF) is originally used to estimate the states
of linear systems. In this research work, KF is designed based
upon the set of equations in [15]. It consists of a prediction
and correction step. In the prediction step, KF estimates the
internal states of the system x̂k based on previous estimated
states x̂k−1 and the previous control signal uk−1 . The states
get updated based on the estimated output signal ŷk , the
real system measurement yk and the calculated Kalman gain
Kk [15]. The gain parameter Kk updates the covariance
matrix Pk−1 within the correction step. The dimension of
the estimated states x̂k needs to be extended for the system
identification. Equation (9) shows the extension of the state
vector through the unknown parameters as ωn the natural
frequency and ζ the damping ratio for a second order system.
Furthermore, the state x1 represents the angular acceleration
and x2 angular velocity of DC motor.
 
x1
 x2 
x̂k = ζ 
 (9)
ωn
The extension leads to a non-linearity in the state space model
whereby EKF has to be used for the identification. EKF is
known for its efficiency in estimating the parameters and
internal states of the system [16]. Equation (10) shows the
obtained non-linearity in the second order system where TS
is the sampling time.
Fig. 4. Block diagram of NARX neural network
EKF identifies the parameters ωn and ζ with the extended state
vector x̂k based on the set of equations in [15]. Each step of
(8) EKF is illustrated in Fig. 3.
C. NARX Neural Network
NARX Neural network is a recurrent multi-layer perceptron
neural network that is used as a black-box identification
method. The main advantage of this identification method to
the previously explained algorithms is that it is not important
to have knowledge about the real system. Moreover, it shows
its effectiveness for linear and nonlinear systems because
it is based on autoregressive models with exogenous inputs
which are known for their efficiency in complex system
identifications [17]. NARX can be described as follows:
y(k) = f (y(k−1), ..., y(k−ny ), u(k−1), ..., u(k−nu )) (11)
NARX is based on a parallel architecture and series parallel
architecture. The series parallel architecture is used during the
training phase while parallel architecture is used during the
testing phase [18]. The output signal depends on the previous
values of the output signal and exogenous input signal [18] as
illustrated in Fig. 4. The high number of the neurons in hidden
layer lead to a high prediction accuracy whereby the training
time and computational complexity may increase accordingly
to it [19].
IV. R ESULTS
The results were performed on a model of a DC motor in
MATLAB/Simulink 2018b. The input and output data were
recorded to identify it with the identification algorithms.
A. Parameters of the Reference System
1) Transfer Function Parameters: RLS and LMS identify
the reference system parameters using Equation (2) which
leads to the resulting parameters in (12).

−2ωn ζx1 − ωn2 x2 K + ωn2 u

   
−1.190276
 x2 
 · TS
 0.332871 
x̂k+1 = x̂k +  (10) Wsys =   (12)
 0   0.058722 
0 0.040711

Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on April 30,2021 at 21:50:19 UTC from IEEE Xplore. Restrictions apply.
 2019 IEEE International Conference on Vehicular Electronics and Safety (ICVES)

2) Model Parameters: EKF identifies the real system pa-

rameters using discrete system model as defined in Equation
(10). The parameters of the reference system for EKF are:
ωn = 48.96725, ζ = 1.12320, K = 0.69731 (13)
B. Identification systems initialization
1) Initialization of LMS: LMS has no prior knowledge
about the system parameters. Its initial weight vector is set
to Wlms,0 = 0 and step size µ = 0.11.
2) Initialization of RLS: Like LMS, the initial weight vector
is set to Wrls,0 = 0, forgetting factor λ = 0.998 and the initial
value of the covariance matix P0 =I.
3) Initialization of EKF: It is initialized by putting the
initial states x0 = 0, the covariance matrix of the measurement
noise R=0.0001, the covariance matrix of the process noise Fig. 5. MSE comparison of RLS, LMS, EKF, NARX NN
Q=0.1I and the covariance matix P0 =0.1I.
4) Initialization of NARX: For training, a series-parallel
architecture is used. The training is done through Lavenberg- have been to use approximate system parameters as initial
Marquardt training algorithm with 2 delays using 100000 conditions in order to achieve a faster convergence. However,
sample training sets and random input signals. The network we choose the above mentioned initial conditions in this
structure is constructed using one hidden layer having 50 research work which provides similar final results and is a
neurons, an input layer with one neuron (voltage) and output worst-case scenario for the convergence of the parameters.
layer with one neuron (angular velocity). In this work, the Fig. 5 shows that EKF gives the most accurate results due
number of the neurons in the hidden layer are chosen to give to its least MSE. It is followed by RLS which has shown
the best performance in regard to the accuracy in an acceptable an accurate result due to the forgetting factor which affects
training time. highly the results. Moreover, it can be noticed that the neural
network gives a relatively higher MSE compared to EKF and
C. System Identification using LMS, RLS, EKF and NARX
RLS. On the other hand, Fig. 5 shows that LMS demonstrates
MATLAB/Simulink is used to perform a series of simu- higher MSE which results in notably less accuracy in the
lations to identify the DC motor (plant in this case) using parameters estimation. Regarding to the convergence rate,
LMS, RLS, EKF, NARX neural network algorithms. During LMS takes more time to converge to true system parameters.
the experimentation phase, random signals are performed as Consequently, for a larger regression vector, LMS becomes
input signals for the identification as proposed in [17]. LMS inefficient because it leads to larger number of computation
gives the following parameter vector. with a very small step size [6]. This makes it unsuitable for
real-time applications. For instance, the convergence time of
 
−1.1899913
 0.3330101  LMS takes ten times more time than RLS followed by NARX
Wlms =   0.0584719 
 (14) with 15 seconds. EKF is found to be the most efficient in
0.0409648 terms of the convergence rate, which took 9.3 seconds. The
convergence time and MSE response of all algorithms are
In a similar way, RLS results the following parameters: shown in Table II.
 
−1.1901804
 0.3329067  E. Analysis of the System Step Response
Wrls = 
 0.0587401 
 (15) In order to check the robustness and the reliability, the step
0.0407731 response of the identified systems are conducted to compare
in regard to the steady state and transient error.
Similarly, EKF gives the parameters as follows: 1) Step Response of LMS, RLS, EKF, NARX: All of the
   
ζ 1.1233239 algorithms are executed using a step input with amplitude of
= (16)
ωn 48.9708504
D. Analysis of the Mean Square Error for the Identified TABLE II
Algorithms I DENTIFIED SYSTEMS COMPARISON

The systems identified using the algorithms exhibit disparity System Convergence time MSE final value (appx)
with the reference system. The Mean Square Error (MSE) RLS 28 seconds 1.2 · 10−6
of the identified systems parameters are shown in Fig. 5. LMS 280 seconds 3.2 · 10−6
EKF 9.3seconds 7.6 · 10−7
Two main factors for the comparison are convergence rate NARX 15 seconds 2.5 · 10−6
and estimation accuracy. Although the best approach would

Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on April 30,2021 at 21:50:19 UTC from IEEE Xplore. Restrictions apply.
 2019 IEEE International Conference on Vehicular Electronics and Safety (ICVES)
Fig. 6. Comparison between system responses
1.5 volts as shown in Fig. 6. These algorithms are compared
based on the angular velocity responses with the reference
system.
2) Error in Step Response of LMS, RLS, EKF, NARX: Fig.
7 shows that the identified systems mimic the real system
in a very accurate manner in terms of the transient, steady
state error and the response time. It is proven that EKF gives
the most accurate response highlighting that the transient and
steady state errors are negligible. This is due to the fact
that EKF gives the least MSE compared to all remaining
algorithms. This behaviour is followed by RLS. It results in
an overshoot due to the error in parameter estimation. In case
of LMS, the steady-state response demonstrates also a large
relative error from the real system response. NARX shows a
high transient response error and relatively accurate steady-
state response because the neural network is trained based on
a random training data. Since NARX is not trained for the
step input, this results in a high transient error. After a certain
time, NARX comes to a settling point having low steady state
error.
Fig. 7. Error between system responses
Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on April 30,2021 at 21:50:19 UTC from IEEE Xplore. Restrictions apply.
TABLE III
PI C ONTROLLER G AINS
System Kp Ki
Reference System 0.0071750 1.4350145
LMS 0.0071960 1.4392110
RLS 0.0071758 1.4351679
EKF 0.0071750 1.4350147
NARX 0.0071639 1.4327966
F. Analysis of the PI Controller Step Response
Different PI controller are connected to each identified
system in order to investigate the robustness and the reliability
of the system in a closed loop simulation. The gain parameters
are shown in Table III. The main reason for choosing a PI
controller is that the system is found to be over-damped. It
helps also to optimize the response time of the system with-
out overshoot. All closed-loop responses achieve the angular
velocity of 1 rad/second as shown in Fig. 8. The time response
shows an accurate overlapping with the reference system. Even
a self-caused disturbance is removed by the closed loop control
of the identified systems. Fig. 9 illustrates that EKF gives
the highest convergence rate to the desired angular velocity.
Moreover, EKF has a very low overshoot with a negligible
steady-state error. EKF generates very small response error
which depicts that identified parameters are almost identical
to the true system parameters. EKF is followed by RLS which
responses with a relatively low overshoot and a low steady-
state response error. LMS have shown a larger transient error
because of its higher MSE. NARX has higher transient error
and a low convergence rate to the settling point. This is due
to the fact that the system is trained with random samples
to achieve generality and hence lacks the needed accuracy
for the step response. When we have performed a self-caused
disturbance, the controller of EKF, RLS and LMS were able to
reduce the error in the closed loop. NARX achieved relatively
high error during the disturbance period.
Fig. 8. Comparison of controlled systems responses
 2019 IEEE International Conference on Vehicular Electronics and Safety (ICVES)
TABLE IV
C OMPARISON B ETWEEN SYSTEM IDENTIFICATION TECHNIQUES
Parameters LMS
Parameters Convergence Rate Low
System Identification Accuracy Intermediate
Computation Complexity Low
Testing Transient Error Intermediate
Testing Steady-State Error High
Fig. 9. Comparison of the error in the system responses
V. C ONCLUSIONS
The results for the four identification algorithms are sum-
marized in Table IV. All techniques except LMS have a high
convergence rate as it is discussed in section IV. The MSE
comparison shows that LMS and NARX have an intermediate
accuracy compared to RLS and EKF. The highest accuracy
can be achieved with EKF but at the cost of the increased
computational complexity due to its matrix operations. RLS
gives a high system identification accuracy with intermediate
complexity. A better performance regarding the transient and
steady state error can be achieved through EKF followed
by RLS and NARX. LMS has witnessed relatively high
steady state and transient errors. It is found out that EKF
proved to have fastest convergence rate and the most accurate
algorithm followed by NARX and RLS. LMS has the slowest
convergence rate in order to identify the system parameters.
R EFERENCES
[1] J. Hauth. Grey-box modelling for nonlinear systems. In Dissertation in
the Technical University of Kaiserslautern, pages 1 – 383, 2008.
[2] H. J. Tulleken. Grey-box modelling and identification using physical
knowledge and bayesian techniques. Automatica, 29(2):285–308, 1993.
[3] L. Ljung. System identification. Wiley Encyclopedia of Electrical and
Electronics Engineering, pages 1–19, 1999.
[4] G. Akgün, H. u. H. Khan, M. A. Elshimy, and D. Göhringer. Dynamic
tunable and reconfigurable hardware controller with ekf-based state re-
construction through fpga-in the loop. In 2018 International Conference
on ReConFigurable Computing and FPGAs (ReConFig), pages 1–8, Dec
2018.
Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on April 30,2021 at 21:50:19 UTC from IEEE Xplore. Restrictions apply.
RLS EKF NARX NN
High High High
High High Intermediate
Intermediate High High
Low Low High
Low Low Low
[5] R. Zarringhalam, A. Rezaeian, W. Melek, A. Khajepour, S. Chen, and
N. Moshchuk. A comparative study on identification of vehicle inertial
parameters. In 2012 American Control Conference (ACC), pages 3599–
3604, June 2012.
[6] S. A. Ghauri and M. F. Sohail. System identification using lms, nlms and
rls. In 2013 IEEE Student Conference on Research and Developement,
pages 65–69. IEEE, 2013.
[7] M Ahmeid, M Armstrong, S Gadoue, and P Missailidis. Parameter
estimation of a dc-dc converter using a kalman filter approach. 2014.
[8] A. Mühürcü S. Aksoy and H. Kizmaz. State and parameter estimation
in induction motor using the extended kalman filtering algorithm. In
2010 Modern Electric Power Systems, pages 1–5. IEEE, 2010.
[9] P. M. Meshram and R. G. Kanojiya. Tuning of pid controller using
ziegler-nichols method for speed control of dc motor. In IEEE-
International Conference On Advances In Engineering, Science And
Management (ICAESM -2012), pages 117–122, March 2012.
[10] G. Huang and S. Lee. Pc-based pid speed control in dc motor. In 2008
International Conference on Audio, Language and Image Processing,
pages 400–407, July 2008.
[11] T. Jabbar S. Arshad, S. Qamar and A. Malik. Parameter estimation
of a dc motor using ordinary least squares and recursive least squares
algorithms. In Proceedings of the 8th International Conference on
Frontiers of Information Technology, FIT ’10, pages 31:1–31:5, New
York, NY, USA, 2010. ACM.
[12] S S. Haykin. Adaptive Filter Theory, volume 4. 01 2002.
[13] M. Bellanger. Digital processing of signals: Theory and practice. New
York, Wiley-Interscience, 1984, 395 p. Translation., 01 1984.
[14] D. Bismor. Lms algorithm step size adjustment for fast convergence.
Archives of Acoustics, 37(1), 2013.
[15] A. Mills. Design and implementation of an fpga-based piecewise affine
kalman filter for cyber-physical systems. In Graduate Theses and
Dissertation, pages 1 – 98, 2016.
[16] B. Terzic and M. Jadric. Design and implementation of the extended
kalman filter for the speed and rotor position estimation of brushless dc
motor. IEEE Transactions on Industrial Electronics, 48(6):1065–1073,
Dec 2001.
[17] H. Liu and X. Song. Nonlinear system identification based on narx
network. In 2015 10th Asian Control Conference (ASCC), pages 1–6,
May 2015.
[18] R. Ahmed H. Camblong Z. Boussaada, O. Curea and M. B. Najiba. A
nonlinear autoregressive exogenous (narx) neural network model for the
prediction of the daily direct solar radiation. Energies, 11:620, 03 2018.
[19] Yazeed A Al-Sbou and Khaled M Alawasa. Nonlinear autoregressive
recurrent neural network model for solar radiation prediction. Inter-
national Journal of Applied Engineering Research, 12(14):4518–4527,
2017.