See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/325257623

Lectures Notes on Optimal Control

Presentation · May 2018

DOI: 10.13140/RG.2.2.14238.48961

CITATIONS READS

0 5,147

1 author:

Toufik Souanef
Cranfield University
22 PUBLICATIONS 44 CITATIONS

SEE PROFILE

All content following this page was uploaded by Toufik Souanef on 22 May 2018.

The user has requested enhancement of the downloaded file.

 Optimal Control
Lecture Notes

SOUANEF Toufik

University of Science and Technology Houari Boumediene - Algiers

2018

1 / 14
Outline

1 Introduction

2 Unconstrained Optimal Control

3 Optimal Control with Constraints on the Input

4 Minimum Time Optimal Control

5 Optimal Linear Quadratic Regulator (LQR)

6 Optimal Linear Quadratic Gaussian (LQG) Control

7 Optimal LQR et LQG Control of Discrete Systems

2 / 14
Definition of Optimal Control

Given the Dynamic nonlinear Multi-Input Multi-Output (MIMO) system

ẋ(t) = f (x(t), u(t))

In this lecture the system is assumed to be controllable

The objective is to elaborate the admissible control law u ∗ (t) which transfers the
system from the initial state x(t0 ) to the final state x(tf ) under the following
performance criterion
Z tf

J = Φ(x(tf ), tf ) + L(x, u, t) dt
t0

3 / 14
Performance Criterion

Minimum time
Z tf
J= dt
t0

Minimum fuel Z tf
J= ku(t)k1 dt
t0

Minimum energy
1
Z tf
J= u > (t)u(t)dt
2 t0

Minimum tracking error

1
Z tf
J= e> (t)e(t)dt
2 t0

with e(t) = xr (t) − x(t) where xr (t) is the reference trajectory

The Linear quadratic(LQ) criterion

1 > 1
Z tf  
J= x (tf )Fx(tf ) + x > (t)Qx(t) + u > (t)Ru(t) dt
2 2 t0

4 / 14
Unconstrained Optimal Control
Given the Dynamic nonlinear Multi-Input Multi-Output (MIMO) system
ẋ(t) = f (x(t), u(t))
with the following performance criterion:
Z tf

J = Φ(x(tf ), tf ) + L(x, u, t) dt
t0
The Hamiltonian is given as follows
H(x, u, λ, ) = L(x, u, t) + λ> f (x, u)
The necessary conditions of optimality are defined by

∂H(x, u, λ)
Optimal control: u ∗ (t) → =0
∂u
∂H(x, u, λ)
Optimal state: x ∗ (t) → = ẋ ∗ (t)
∂λ
∂H(x, u, λ)
Optimal co-state: λ∗ (t) → = −λ̇∗ (t)
∂x
The boundary conditions are given by
h ∂Φ i
Unspecified final time: + H(x ∗ , u ∗ , λ∗ ) = 0
∂t tf
h ∂Φ i
Unspecified final state: −λ =0
∂x tf

5 / 14
Optimal Control with Constraints on the Input

Given the Dynamic nonlinear Multi-Input Multi-Output (MIMO) system

ẋ(t) = f (x(t), u(t))

with the following performance criterion:

Z tf

J = Φ(x(tf ), tf ) + L(x, u, t) dt
t0

and the constrained control law u(t) ∈ M − , M +

 

The Hamiltonian is given as follows

H(x, u, λ, ) = L(x, u, t) + λ> f (x, u)
The necessary conditions of optimality are defined by

Optimal control: u ∗ (t) → H(x ∗ , u ∗ , λ∗ ) ≤ H(x ∗ , u, λ∗ ), ∀u(t) ∈ M − , M +

 

∂H(x, u, λ)
Optimal state: x ∗ (t) → = ẋ ∗ (t)
∂λ
∂H(x, u, λ)
Optimal co-state: λ∗ (t) → = −λ̇∗ (t)
∂x
The boundary conditions are the same that for the unconstrained case.

6 / 14
Minimum Time Optimal Control of Linear Systems
Given the linear single-input single-output (SISO) system
ẋ(t) = Ax(t) + bu(t)
with the constraint |u(t)| ≤ M
The objective is to elaborate the optimal control law u ∗ (t) to transfer the system from
the initial state x(t0 ) to the final state x(tf ) = 0 in minimum time.
The system eigenvalues should not be of positive real part
The performance criterion is given by
Z t
f
J= dt
t0
Applying the Pontryaguine minimum, the optimal control law is given by
u ∗ (t) = −M sign b> λ∗ (t)

with
>
λ∗ (t) = e(−A t)
λ0
Remark: In the MIMO case:
ẋ = Ax(t) + Bu(t)
The input matrix B can be written as follows
B = [b1 |b2 ...|bm ]
hence
ui∗ (t) = −M sign bi> λ∗ (t) ;


i = 1...m

7 / 14
Optimal Linear Quadratic Regulator (LQR)

Given the linear Multi-Input Multi-Output (MIMO) system

ẋ(t) = Ax(t) + Bu(t)

with the Linear Quadratic criterion (LQ)

1 > 1
Z tf  
J= x (tf )F x(tf ) + x > (t)Qx(t) + u > (t)Ru(t) dt
2 2 t0

It is assumed that there are no constraints on the control input

Using the necessary conditions of optimality, the open-loop control law is given by

u ∗ (t) = −R −1 B > λ∗ (t)

Assuming that λ∗ (t) = P(t)x ∗ (t), the closed-loop control law is given by

u ∗ (t) = −K (t)x ∗ (t),

with: K (t) = R −1 B > P(t)

P(t) is the solution of the Riccatti equation:

Ṗ(t) = P(t)A + A> P(t) + Q − P(t)BR −1 B > P(t)

with P(tf ) = λ(tf )

8 / 14
Infinite Horizon Linear Quadratic Regulator (LQR)

Given the linear Multi-Input Multi-Output (MIMO) system

ẋ(t) = Ax(t) + Bu(t)

The Linear Quadratic (LQ) criterion for infinite horizon is defined by

1 ∞ >
Z 
J= x (t)Qx(t) + u > (t)Ru(t) dt
2 t0

The closed-loop control law is written as follows

u ∗ (t) = −Kx ∗ (t),

with: K = R −1 B > P

P, constant in this case, if the solution of the Riccatti equation:

PA + A> P + Q − PBR −1 B > P = 0

Stability: The eigenvalues of the closed-loop system are of negative real part
Robustness: Infinite gain margin and a phase margin of 600
Matlab
[K , P] = lqr (A, B, Q, R)

9 / 14
Optimal Linear Quadratic Gaussian (LQG) Control with Infinite
Horizon
Given the linear Multi-Input Multi-Output (MIMO) system assumed to be controllable
and observable
ẋ(t) = Ax(t) + Bu(t) + Gv (t)
y (t) = Cx(t) + Hw(t)

where v (t) and w(t) are uncorrelated white noises, with a null and variance (intensity)
V and W , respectively.
The objective is to compute the optimal control law u ∗ (t) to satisfy the linear quadratic
conditions
1 ∞ >
Z 
J= x (t)Qx(t) + u > (t)Ru(t) dt
2 t0
the design of the control law is done in two steps
1- Estimation of the state vector using Kalman filter

˙
x̂(t) = Ax̂(t) + Bu(t) + L(y (t) − C x̂(t))

with:

avec: L = Pe CW −1

P is the solution of the Riccatti equation:

Pe A> + APe + V − Pe C > W −1 CPe = 0

10 / 14
Optimal Linear Quadratic Gaussian (LQG) Control with Infinite
Horizon(contd)

2- Design of the Optimal Control Law

The closed-loop control law is given by

u ∗ (t) = −K x̂ ∗ (t),
avec: K = R −1 B > P

where P is the solution of theRiccatti equation

PA + A> P + Q − PBR −1 B > P = 0

Matlab
sys = ss(A, [B G], C, 0)
[kest, L, P] = kalman(sys, V , W )
[K , P] = lqr (A, B, Q, R)
We can also use the commands lqe and lqg

11 / 14
Optimal LQR Control of Discrete Systems

Given the dynamic discrete Multi-Input Multi-Output (MIMO) system

x(k + 1) = Ax(k ) + Bu(k )

The objective is to design the control law u ∗ (k ) which satisfies the Linear Quadratic
discrete criterion
∞
1 X > 
J= x (k )Qx(k ) + u > (k )Ru(k )
2
k =0

The optimal closed-loop control law is given by

u ∗ (k ) = −Kx ∗ (k ),
−1
avec: K = R −1 B > (A> ) (P − Q)

P is the solution of the Riccatti equation:

P = A> P −1 + BR −1 P A

Matlab
Methode 1 Methode 2
Given the continuous system ẋ(t) = Ac x(t) + Bc u(t) Given the continuous
sysD = c2d(sysC, Ts ), Ts is the sample time  system ẋ(t) = Ac x(t) +Bc u(t)
with J = 21 t∞ x > (t)Qx(t) + u > (t)Ru(t) dt
R
[K , P] = dlqr (A, B, Q, R) pour 0
P∞  > >

[K , P] = lqrd(Ac , Bc , Q, R, Ts )
J = 12 k =0 x (k )Qx(k ) + u (k )Ru(k )

12 / 14
Optimal LQG Control of Discrete Systems
Given the dynamic discrete Multi-Input Multi-Output (MIMO) system controllable and
observable
ẋ(k + 1) = Ax(k ) + Bu(k ) + Gv (k )
y (k ) = Cx(k ) + Hw(k )
where v (k ) and w(k ) are uncorrelated white noises, with a null and variance (intensity)
V and W , respectively.
The objective is to design the optimal control lawu ∗ (k ) to satisfy the LQ criterion
∞
1 X > 
J= x (k )Qx(k ) + u > (k )Ru(k )
2
k =0

The design of the control law is made in two steps:

1- Estimation of the state vector using Kalman filter
˙ + 1) = Ax̂(k ) + Bu(k ) + L(y (k ) − C x̂(k ))
x̂(k
2- Design of the optimal control laws
The closed-loop control law is given by
u ∗ (k ) = −K x̂ ∗ (k ),
Matlab
Methode 1 Methode 2
Given the continuous system Given the continuous system
ẋ(t) = Ac x(t) + Bc u(t) + Gv (t); y (t) = Cx(t) + Hw(t) ẋ(t) = Ac x(t)
 + Bc u(t) + Gv (t); y (t) =Cx(t) + Hw(t) avec
sysD = c2d(sysC, Ts ), Ts is the sampling time J = 1 t∞ x > (t)Qx(t) + u > (t)Ru(t) dt
R
2 0
[kest, L, P] = kalman(sys, V , W )
[kest, L, P] = kalmd(sys, V , W , Ts )
[K , P] = dlqr (A, B, Q, R) with
P∞  > [K , P] = lqrd(Ac , Bc , Q, R, Ts )
x (k )Qx(k ) + u > (k )Ru(k )

J = 1
2 k =0
13 / 14
References

1- Desineni Subbaram Naidu, Optimal Control Systems, CRC PRESS, 2003,

2- Radhakant Padhi, Optimal Control Guidance and Estimation, Lecture Notes,

View publication stats

14 / 14