Lehrstuhl für Systemtheorie und OvGU

Regelungstechnik Magdeburg
Prof. Dr.-Ing. Rolf Findeisen WS 2015/16
Optimal Control
http://ifatwww.et.uni-magdeburg.de/syst/

Solution to Mock Exam Optimal Control

WS 2018/19

The mock exam consists of 10 pages (7 Tasks, 38 Points).

Surname, Name:

Student Number:

Study Direction:
Task 1 (5 points)
An optimal control problem for the system

ẋ = f (x, u), x(0) = x0 ∈ Rn (1.a)

is given by
Z tf
min x(τ )T Qx(τ ) + u(τ )T Ru(τ ) dτ + x(tf )T F x(tf ), (1.b)
u(·) 0

where tf is fixed, u ∈ Rp , x ∈ Rn and Q, F are positive semi definite and R is positive definite.

The solution of the optimal control problem shall be calculated using Pontryagin’s Minimum
Principle (PMP). Solve the optimal control problem via the PMP following the steps outlined
below:

a) Define the Hamiltonian of the problem. (1 point)

b) Write the equations for the state x, the co-state λ and the input u defining the two-point
boundary value problem (TPBVP). The answer should also provide information about
the boundary conditions of the TPBVP. (3 points)

c) Assuming that the input u? (t), t ∈ [0, tf ] solves the TPBVP. Is it guaranteed that u?
minimizes the objective function (1.b)? Why or why not? (1 point)

Solution
a) Hamiltonian: H(x, u, λ) = xT Qx + uT Ru + λT f (x, u), with λ ∈ Rn .

b) Since the Hamiltonian is time invariant and the time tf is fixed, we have the special case:
H(x∗ , u∗ , λ) = H ∗ = constant.
The TPBVP is given by
∗
ẋ∗ = ∂H ∗ ∗ ∗
∂λ = f (x , u ) with x (t0 ) = x0
∗
∗ ∂f (x∗ ,u∗ )
λ̇ = − ∂H
∂x = −(2Qx + λ ∂x ) with λ̇(tf ) = x∗ (tf )T F x∗ (tf )
∂f (x∗ ,u∗ )
and u∗ is given by ∂H
∂u |u = 2Ru + λ
∗
∗
∂u =0

c) No. PMP gives in general only necessary conditions. For special cases like linear systems
with quadratic cost, sufficiency can be also established. In this case, however, we have no
information about the system f (x, u), and thus no claims about sufficiency can be made.
Task 2 (5 points)
Consider the linear time-invariant discrete-time system x(tk+1 ) = Ax(tk ) + Bu(tk ), where
x ∈ Rn , u ∈ Rp . We want to implement a linear quadratic regulator (LQR) with an infinite
∞
(x(tk )T Qx(tk ) + u(tk )T Ru(tk )). Assume that the solution to the
P
horizon, using the cost
k=0
discrete algebraic Riccati equation is given by P .

a) The LQR feedback law is given by u(tk ) = Kx(tk ), with K = −(R + B T P B)−1 B T P A.
Which conditions should the matrices A, B, Q, and R satisfy for this feedback law to be
stabilizing? (2 points)

b) Suppose that we want to stabilize the system around the origin while satisfying input
constraints u(tk ) ∈ U ⊂ Rp . The set U contains the origin in its interior. Does the LQR
guarantee that the input constraints are always satisfied? (1 point)

c) Now suppose that we additionally want to satisfy the state constraints x(tk ) ∈ X ⊂ Rn .
The set X contains the origin in its interior. Does the LQR guarantee that the state
constraints are always satisfied? (1 point)

d) Which type of controller could stabilize the system for all x(tk ) ∈ X while satisfying
u(tk ) ∈ U? (1 point)

Solution
1
a) The pair (A, B) must be stabilizable, the pair (A, Q 2 ) must be detectable, with Q positive
semi-definite and R positive definite.

b) No. An LQR cannot guarantee that the input constraints are satisfied.

c) No. An LQR cannot guarantee that the state constraints are satisfied.

d) One possibility is model predictive control with stability guarantees.

Task 3 (8 points)
a) Which of the following static optimization problem does not admit at least one so-
lution?

min(−4) min (u21 + u22 )

1) u1 2) u1 ,u2
s.t. u21 > −4, s.t. u1 ≥ 0, u2 ≥ 0,
n
 
max − u2i
P
ui i=1
max(−u21 + u2 )
3) 4) u1 ,u2
s.t. u2i≥ 1, s.t. u2 < 0,
∀i ∈ [1, n],
where all optimization variables are scalar. Justify your answer. (2 points)

b) Consider the following optimization problem

min(−z 3 )
z≤α

where α, z ∈ R.

1) For which α is the optimization problem convex? Justify your answer. (2 points)

2) State the Karush-Kuhn-Tucker (KKT) conditions. For which α are they necessary?
Justify your answer. (2 points)

3) Find a solution of the problem in terms of α. (2 points)

Solution
a) 1) admit any solution u ∈ R.
2) has u1 = 0, u2 = 0 as solution (the constraints are active).
3) has all possible combinations of ui = ±1, for i = 1, . . . , n as solutions.
4) is a concave maximization problem very similar to Problem 2). With u2 ≤ 0 the
solution is u1 = 0, u2 = 0 (u2 is active). However, the feasible set is open (u2 < 0)
and thus the minimum cannot be achieved. This problem does not admit a solution.

b) 1) The cost function is not convex in its domain R. Thus, this problem is not convex
for any α.
2) The Lagrange function for this problem is given by L(z, µ) = −z 3 + µ(z − α).
The KKT conditions are

∇z L(z, µ) = −3z 2 + µ = 0,
∇µ L(z, µ) = z − α ≤ 0,
µ(z − α) = 0,
µ ≥ 0.

The KKT conditions are necessary for all α ∈ R.

3) From the KKT conditions we have 3z 2 = µ and µ(z − α) = 0. We have then
3z 2 (z − a) = 0. The solution to this equation are z = 0 and z = α. Although z = 0
satisfies the KKT conditions, it is not a meaningful solution to the optimization
problem. Therefore, the solution to the optimization problem is given by z = α.
Task 4 (6 points)
Consider the following discrete multi-stage optimal control problem:
N −1
(ui )2
X
min (1 − xN )2 + (4.a)
{u0 ,u1 ,...,uN −1 }
i=0

subject to the discrete dynamics:

xk+1 = xk + uk , x0 = x0 , (4.b)

and where xk and uk are scalars. Note that the initial state x0 and the process time N is fixed.

a) Solve the problem for N = 2 and x0 = 0 using the KKT conditions. (2 points)

b) Define the dynamic programming recursion for the problem. State the initialization of
the recursion. (2 points)

c) Solve the problem for N = 2 and x0 = 0 using the dynamic programming recursion.
(2 points)
Solution
a) The Lagrange function is given by

L(z, µ) = (1 − x2 )2 + u20 + u21 + µ0 (x0 − x0 ) + µ1 (x0 + u0 − x1 ) + µ2 (x1 + u1 − x2 )

where z = (x0 , x1 , x2 , u0 , u1 )T is the optimization variable, and µ = (µ0 , µ1 , µ2 )T is

the vector of Lagrange multipliers.
The KKT conditions are given by the following equalities:
   
−µ0 + µ1 0

 −µ1 + µ2  
  0 

∇z L =  −2(1 − x2 ) + µ2 = 0
   

   
 2u0 + µ1   0 
2u1 + µ2 0
   
x0 − x0 0
∇µ L =  x0 + u0 − x1  =  0 
   
x1 + u1 − x2 0

By solving this system of equations the solution z ∗ is obtained, which is given by x∗0 = 0,
x∗1 = 31 , x∗2 = 23 u∗0 = 13 , u∗1 = 31 .

b) The DP recursion and its initialization are given by:

n o
Jj∗ (xj ) = min u2j + Jj+1
∗
(xj + uj ) ,
uj
∗
JN (xN ) = (1 − xN )2 .

c) For N = 2 and x0 = 0:

J2∗ (x2 ) = (1 − x2 )2
n o
J1∗ (x1 ) = min u21 + (1 − (x1 + u1 ))2
u1

let Jˆ1 (x1 , u1 ) = u21 + (1 − (x1 + u1 ))2 , then u∗1 = arg min Jˆ1 (x1 , u1 ). From first-order
u1
optimality conditions we have that

∂ Jˆ1 (x1 , u1 )
|u1 =u∗1 = 0
∂u1
1 − x1 (1 − x1 )2
u∗1 = −→ Jˆ1 (x1 , u∗1 ) = J1∗ (x1 ) =
2 2
1
 
J0∗ (x0 ) = min u20 + (1 − (x0 + u0 ))2
u0 2
(1 − x0 )
u∗0 = .
3
Since x∗0 = x0 = 0, by iterating forward we get u∗0 = 31 , x∗1 = 31 , u∗1 = 13 , and x∗2 = 23 .
Task 5 (4 points)

a) Describe the main idea of direct methods for the numerical solution of optimal control
problems. (2 points)

b) Name at least two methods for the approximation of the system dynamics using full
discretization methods. (1 point)

c) Which are the differences with respect to the size of the optimization problems that result
from single direct shooting, multiple direct shooting and full discretization? (1 point)

Solution
a) Direct methods are based on the discretization of the inputs and / or states to solve
numerically optimal control problems. The main idea is to divide the time horizon into
different time intervals in which the control input is parametrized, so that a static opti-
mization problem with a finite number of decision variables is obtained.

b) Euler discretization and orthogonal collocation on finite elements.

c) The size of the optimization problem with single direct shooting is smaller than the one
with multiple direct shooting, which is smaller than the one obtained with full discretiza-
tion.
Task 6 (4 points)

a) Enumerate at least three major advantages of model predictive control with respect to
traditional linear controllers. (2 points)

b) Enumerate at least two important drawbacks of model predictive control with respect to
traditional linear controllers. (2 points)

Solution
a) • Explicit considerations of constraints
• Handling of MIMO systems
• Handling of nonlinear systems
• Incorporation of available predictions

b) • Quality of the model is critical

• Computationally expensive
Task 7 (6 points)

a) In a model predictive control (MPC) context, calculating the exact solution of an optimal
control problem at each sampling time does not imply stability of the closed-loop system.
Explain why does this occur. (2 points)

b) One possibility to enforce stability of the closed-loop is to use the zero terminal con-
straint MPC strategy. The cost at time t is defined as J = tt+T F (x(τ ), u(τ ))dτ
R

where F (x(t), u(t)) is a positive definite function, T is the prediction horizon and ẋ =
f (x(t), u(t)) represents the system dynamics. Formulate the optimal control problem that
has to be solved at each sampling time for a zero terminal constraint MPC. (1 point)

c) The optimal solution calculated at time t (until t + T ) is denoted by u? (t) and the
corresponding optimal cost is
Z t+T
?
J (t) = F (x? (τ ), u? (τ ))dτ.
t

Obtain the expression of an upper bound of the optimal cost of the optimal control
problem at the following sampling time (J ? (t + 1)) if a zero terminal constraint MPC is
applied. Note that, as usual, we consider minimization problems. (3 points)

Solution
a) Because for a finite horizon, the predicted trajectories at each sampling time are not equal
to the closed-loop trajectories.

b)
Z t+T
min J= F (x(τ ), u(τ ))dτ
u(t) t
s.t.: ẋ = f (x(t), u(t))
x(t + T ) = 0

c) An upper bound of the optimal cost at time t + 1 can be obtained using as control
trajectory the optimal trajectory obtained at time t using a zero control input for the
last step. The feasible solution would be: û = [u?t+1:T , 0] and then the upper bound is
R t+T
J ? (t + 1) ≤ J ub (t + 1) = t+1 F (x(τ ), û(τ ))dτ .