Master's degree in Automatic Control and Robotics

Chapter 4:
Optimal Parameter Estimation
 Master's degree in Automatic Control and Robotics

Conceptual Idea

• Optimal parameter estimation utilizes optimization to generate the parameter estimation

considering the inputs and measurements
• As in the case of optimal control, the practical implementation of parameter estimation is
done using a computer. For this reason, parameter estimation is also typically implemented
in discrete-time.

parameter
estimation
 Master's degree in Automatic Control and Robotics

Problem Formulation (1)

• The problem of parameter estimation is transformed in an optimization problem:

(1) The parameter estimation goals are transformed into an objective function named J that is
function of the parameters in a time horizon N
– The objective function contains estimation error due to the sensor noise.
(2) The model and the physical limitations of the parameters are the constraints of the
optimization problem

• Optimal parameter estimation can be applied to multivariable linear and non-linear systems.
• The theory of optimal parameter estimation can developed in continuous time or in discrete-
time.
• As in the case of optimal control, we will focus in discrete-time formulation because is the
one more used in practice and because fits the optimization theory presented in the first part
of the course:

x  fc ( x(t ),u(t )) 
Ts
 x(k  1)  f ( x(k),u(k)
 Master's degree in Automatic Control and Robotics

Problem Formulation (2)

• As in the case of optimal control, we will focus in discrete-time formulation because is the
one more used in practice and because fits the optimization theory presented in the first part
of the course.

• There are many methods for discretizing a numerical system (as e.g., Euler, Runge-Kutta, etc.)

• Let’s consider the simplest one based on the Euler method

x(k  1)  x(k)
x(t ) 
Ts

• Then, the non-linear model of the system in continuous-time can be expressed in discrete-
time as follows
x  fc ( x(t ),u(t )) 
Ts
 x(k  1)  f ( x(k),u(k)
 Master's degree in Automatic Control and Robotics

Problem Formulation (3)

• The problem of parameter estimation is transformed in an optimization problem:

(1) The estimation goals are transformed into an objective function named J that is function of
the estimated states in a time horizon N
(2) The model and the physical limitations of the parameters are the constraints of the
optimization problem

N
min
ˆ
 e T

k n
(k)e(k)

suject to :
e(k)  y(k)  f ( y(k  1),u(k  1), ˆ) k  n, ,N
ˆ   ,  
 
 Master's degree in Automatic Control and Robotics

Problem Solution

• To solve the optimization problem associated to the optimal parameter estimation problem,
there are two procedures

(1) Analytically using the theory learned in the first part of the course.
(2) Numerically using numerical solvers as the ones available in Matlab.

• For linear systems, both solutions are possible when neglecting the physical constraints
affecting parameters.

• For non-linear systems, the analytical solutions is almost impossible, so only the numerical
one is possible.
 Master's degree in Automatic Control and Robotics

Problem Formulation: The Linear Case (Least Squares)

• When the system to be estimated can be formulated or approximated using a linear model,
the system can be represented in the standard linear form that after discretising can be
expressed in the regressor form:

x  Ac x(t )  Bcu(t ) 
Ts
 y(k)  c T (k)

• Then, the previous optimization problem can reformulated in the following way

N
min
ˆ

 e T

k n
(k)e(k)

suject to :
e(k)  y(k)  c T (k)ˆ k  n, ,N
ˆ   ,  
 
 Master's degree in Automatic Control and Robotics

Numerical Solution

• The numerical solution of the optimal parameter estimation problem problem can be
obtained using an optimization language as Yalmip

x = sdpvar((nx,1,N+1),(1,1,N+1));
v = sdpvar((ny,1,N),(1,1,N));

constraints = [];
for k = 2:N
objective = objective + v’{k}*(R-^1)*v{k};
constraints = [constraints, y(k) == C*x{k} +v{k}];
end

options = sdpsettings(‘solver', ‘quadprog');

optimize(constraints,objective,options);
 Master's degree in Automatic Control and Robotics

Analytical Solution

• The analytical solution of the optimal parameter estimation problem can be done using
several approaches:

1. Lagrange (or substitution) method

2. Dynamic programming

• To obtain the analytical solution, physical constraints are neglected and the objective
function is expressed in vector/matrix form as follows

N
min
ˆ 
( N )
e T
(k)e(k)
k n

suject to :
e(k)  y(k)  c T (k)ˆ(k) k  n, ,N
 Master's degree in Automatic Control and Robotics

Analytical Solution: Substitution Method

• To solve the previous optimization problem a new objective function is created as follows

N
J   (( y(k)  c T (k)(k))T ( y(k)  c T (k)(k))  (Y(N)  C(N) (N))T (Y(N)  C(N)(N))
k n

• To obtain the analytical solution: J  0

• This implies the following partial derivatives:

J
0
ˆ
(k)
 Master's degree in Automatic Control and Robotics

Analytical Solution: Non-Recursive Expression

Optimal Estimation

The optimal parameter estimation obtained from the analytical solution is

ˆ(k)  [C T (k)C(k)] 1 C T (k)Y(k)

Covariance Estimation
The covariance estimation is given by:
1
P(k)  C T (k)C(k) 
 Master's degree in Automatic Control and Robotics

Analytical Solution: Recursive Expression

Optimal Estimation

The optimal estimation obtained from the analytical solution is a state observer:
ˆ(k  1)  ˆ(k)  K(k  1)( y(k  1)  c T (k  1)
ˆ (k))
where
P(k)c(k  1)
K(k  1) 
1  cT (k  1)P(k)c(k  1)

Covariance estimation
P(k)c(k  1)c T (k  1)P(k)
P(k  1)  P(k) 
1  cT (k  1)P(k)c(k  1)
1
where P(k)  C T (k)C(k) 
 Master's degree in Automatic Control and Robotics

Analytical Solution of Parameter Estimation: Example (1)

• Calculate the optimal parameter estimation with the analytical solution for the following
system:
y(k)  au(k)  bu(k  1)

considering that the following input/output data has been collected from the system:

k u y
0 1.0 0.9
1 0.8 2.5
2 0.6 2.4
3 0.4 1.3
4 0.2 1.2
5 0 0.8
6 0.2 0
7 0.4 0.9
8 0.6 1.4
 Master's degree in Automatic Control and Robotics

Analytical Solution of Parameter Estimation: Example (2)

• First, we will apply the non-recursive method using data until k=7

ˆ(7 )  [C T (7 )C(7 )] 1 C T (7 )Y(7 ) ˆ(7 )  0.3214 

 2.4464 
 

Y(7 )  C(7 )ˆ(7 )

 2.5  0.8 . 
10
 2.4  06 0.8 
   .
 13
.  0.4 . 
06
    a 
.   0.2
 12 0.4   
b
0.8   0 0.2   
   
 0  0.2 0
0.9  0.4 0.2 
  

1  7.1429 5.3571
• The covariance matrix is given by: P(7 )  C T (k)C(k)    
 5.3571 4.4642 
 Master's degree in Automatic Control and Robotics

Analytical Solution of Parameter Estimation: Example (3)

• First, we will apply the non-recursive method using data until k=8 using the result at time k=7

ˆ( 8 )  ˆ(7 )  K( 8 )( y( 8 )  c T ( 8 )ˆ(7 )) ˆ( 8 )   06071

. 
 2.2560 
 
cT ( 8 )  06
. 0.4 

P(7 )c T ( 8 )  12500
. 
K( 8 )    
1  cT ( 8 )P(7 )c( 8 )  0.8333 

• The covariance matrix is given by:

P(7 )c( 8 )c T ( 8 )P(7 )  4.4643 35714

. 
P( 8 )  P(7 )   
1  cT ( 8 )P(7 )c( 8 )  35714
. 3.2738 
 Master's degree in Automatic Control and Robotics

Kalman Filter for Parameter Estimation

• The Kalman filter can be used for parameter estimation, considering that x̂(k)  ˆ(k)

A  I ,B  0 ,C  c T

ˆx(k  1)  Ax(k)
ˆ  Bu(k)  L(k)( y(k)  Cx(k))
ˆ ˆ(k  1)  ˆ(k)  K(k  1)( y(k  1)  c T (k  1)
ˆ (k))

A  I ,B  0 ,C  c T ,R  1
P(k)c(k  1)
1 K(k  1) 
L(k)  AP(k)C [R  CP(k)C ]
T T
1  cT (k  1)P(k)c(k  1)

A  I ,B  0 ,C  c T ,Q  0
P(k)c(k  1)cT (k  1)P(k)
P(k  1)  Q  [ A  L(k)C ]P(k)A T
P(k  1)  P(k) 
1  cT (k  1)P(k)c(k  1)