OPTIMAL CONTROL

7
7.1 EXERCISE 1

Solve the following optimal control problem

 1
max (x − u2 )dt x = u x(0) = 2.
0

7.1.1 Solution with the first variation

The Lagrangian is
 1
L(x, u, λ, μ) = [(x − u2 ) − λ(x − u)]dt − μ(x(0) − 2).
0

Now perform the first variation


d 
δL(x, u, λ, μ) =  L(x + αδx , u + αδu , λ + αδλ , μ + αδμ )
dα 
α=0

Making the substitution yields

 1
d
δL = lim [x + αδx − (u + αδu )2 − (λ + αδλ )(x + αδx − u − αδu )]dt
α→0 dα 0

−(μ + αδμ )(x(0) + αδx(0) − 2)

After differentiation one has

 1
δL = lim [δx − 2(u + αδu )δu − δλ (x + αδx − u − αδu )
α→0 0

−(λ + αδλ )(δx − δu )]dt − δμ(x(0) + αδx(0) − 2) − (μ + αδμ )δx(0)

And collecting terms

 1
= [δx − 2uδu − δλ (x − u) − λ(δx − δu )]dt
0

−δμ (x(0) − 2) − μδx(0)

97
98 OPTIMAL CONTROL

One can derive directly this last passage skipping the above calculations if performs the
differentiation of each variable. Now there is the problem to express the variation δx in
terms of the other variations. To see that consider the term involving δx , that is λδx . If
one differentiates λδx with respect to t has
d d
[λδx ] = λ δx + λδx =⇒ λδx = [λδx ] − λ δx .
dt dt
Using this relation in the above expression gives
 1
d
δL = [δx − 2uδu − δλ (x − u) − (λδx ) + λ δx + λδu ]dt
0 dt
−δμ (x(0) − 2) − μδx(0) .

Now collecting the variations in order to use the du Bois-Reymond theorem leads to
 1
δL = [δx (1 + λ ) + δu (−2u + λ) + δλ (u − x )]dt
0

−λ(1)δx(1) + (λ(0) − μ)δx(0) − δμ(x(0) − 2)

This leads to this system of ordinary differential equations,

1 + λ = 0 −λ(1) = 0

−2u + λ = 0 λ(0) − μ = 0

u − x = 0 x(0) − 2 = 0

From the first differential equation one has λ = −t + c, then using the initial condition
λ(1) = 0 yields λ(1) = −1 + c = 0 =⇒ c = 1. So the multiplier is λ(t) = −t + 1.
From the multiplier one can resolve the optimal control u(t), in facts from the second
differential equation −2u + λ = 0 =⇒ u(t) = λ(t) 2
= −t+1
2
.
Finally one can reconstruct the state variable x(t). From the first differential equation
x = −t+1
2
=⇒ x(t) = − 14 t2 + 12 t + c. Now from the initial condition on x(t) one has
x(0) = c = 2.
In general, to prove the fact that this control maximizes the integral is a difficult task,
therefore it is better to rely on special theorems. 

7.2 EXERCISE 2

Solve the following optimal control problem (OCP).

 2
min J = min u(t)2 − 2x(t) dt with x (t) = 1 − 2u(t) x(0) = 1, x(2) = 0.
0
 7.2 E X E R C I S E 2 99

7.2.1 Solution with variations calculus

To solve the problem, one can use calculus of variation in order to find a maximum of
the functional J, so
 2
min J = max −J = max −u(t)2 + 2x(t) dt.
0

The Lagrangian function is given by

 2
L(x, u, λ, μ1 , μ2 ) = −u2 + 2x − λ(x − 1 + 2u) dt − μ1 (x(0) − 1) − μ2 (x(2) − 0)
0

Now performing the first variation of L yields

 2
δL = −2uδu + 2δx − λ(δx + 2δu ) − δλ (x − 1 + 2u) dt
0

−μ1 δx(0) − δμ1 (x(0) − 1) − μ2 δx(2) − δμ2 (x(2) − 0).

To simplify the variation δx it is enough to derive λδx ,

(λδx ) = λ δx + λδx =⇒ λδx = (λδx ) − λ δx ,

so the previous expression becomes

 2
δL = −2uδu + 2δx + λ δx − 2λδu − δλ (x − 1 + 2u) dt
0

−λ(2)δx(2) + λ(0)δx(0) − μ1 δx(0) − δμ1 (x(0) − 1) − μ2 δx(2) − δμ2 (x(2) − 0).

Collecting the expression of each variation gives the associated boundary value problem.

δu : −2u − 2λ δ x : 2 + λ δλ : x − 1 + 2u

δx(0) : λ(0) − μ1 δx(2) : −λ(2) − μ2

δμ1 : x(0) − 1 δμ2 : x(2)

From the variation δx one can solve the multiplier λ, in facts

λ = −2 =⇒ λ(t) = −2t + c c ∈ R.

From variation δu one can solve the optimal control u,

−2u − 2λ = −2u − 2(−2t + c) = 0 =⇒ u = 2t − c.

From the differential equation given by the variation of the multiplier λ

x = 1 − 2u =⇒ x = 1 − 2t + c =⇒ x(t) = −t2 + (c + 1)t + d d ∈ R.

100 OPTIMAL CONTROL

Now the constants c, d can be evaluated using the boundary condition x(0) = 1 and
x(2) = 0.

x(0) = d = 1 =⇒ d = 1
1
x(2) = −4 + 2c + 2 + 1 = 0 =⇒ c = .
2
In conclusion the optimal control is u(t) = 2t − 12 and the associated trajectory or status
is x(t) = −t2 + 32 t + 1. The extremal value of the functional is therefore
 2  2  2  
2 1 2 3
max −u(t) + 2x(t) dt = − 2t − + 2 −t + t + 1 dt
0 0 2 2
 2
= −6t2 + 5t + 7/4 dt
0
2
5 2 7  3
= −2t + t + t
2 4 0
5
=− .
2
If one tries another function that satisfies the required constraints, for example x(t) =
(t − 2)(t − 12 ) and the associated control u(t) = 2t − 52 , that the functional −J gives
− 23
6
≈ −3.833 < −2.5.
It can be showed, for example using the Hamiltonian function or with the second
variation, that it is a maximum point1 , so for the original problem it will be a minimum.


7.3 EXERCISE 3

Solve the following OCP,

 1
min −x2 − u2 dt x = 2x + u x(0) = 1.
0

7.3.1 Solution with the Hamiltonian

The OCP has a quadratic control, so we can apply the Hamiltonian to solve it,

H = −x2 − u2 + λ(2x + u).

The equation for the control is ∂H/∂u = −2u + λ = 0 which implies u = λ/2. The
adjoint equation is λ = 2(x − λ), thus we have to solve a system of two differential
equations. The boundary conditions for this system are x(0) = 1 and λ(1) = 0. To

2
1∂ H
= −2 < 0
∂u2
 7.4 E X E R C I S E 4 101

solve the system there are various techniques as the Laplace transform or differential
equations methods, we choose the latter. Differentiating the adjoint equation yields,

λ = 2(x − λ ) = 2(2x + u − 2x + 2λ) = 2(2λ + u) = 4λ + 2u = 4λ + λ = 5λ.

This is a second order differential equation which has boundary conditions given
√ √
by λ(1) = 0 and λ (1) = 5λ(1) = 0. Its general solution is λ(t) = c1 e 5t + c2 e− 5t ,
substituting the boundary conditions gives c1 = c2 = 0 therefore λ(t) = 0. This implies
u(t) = 0 and the differential equation for the state x becomes x = 2x. The last equation
solves in x(t) = ke2t with x(0) = k = 1, that is x(t) = e2t .
Let us check if the candidate solution u(t) = 0 is a minimum, looking at the ∂ 2 H/∂u2 =
−2 < 0, shows that u(t) = 0 can not be a minimum, so this problem does not have
solution.

7.4 EXERCISE 4

Solve the following OCP,

 1/3
min u2 + 2x2 dt x = 2u + x x(0) = 1.
0

7.4.1 Solution with the Hamiltonian

The OCP has a quadratic control, so we can apply the Hamiltonian to solve it,

H = u2 + 2x2 + λ(2u + x).

The equation for the control is ∂H/∂u = 2u + 2λ = 0 which implies u = −λ. The adjoint
equation is λ = −4x − λ, thus we have to solve a system of two differential equations.
The boundary conditions for this system are x(0) = 1 and λ(1/3) = 0. Differentiating
the adjoint equation yields,

λ = −4x − λ = −4(2u + x) − λ = −8u − 4x + 4x + λ = −8u + λ = 9λ.

This is a second order differential equation which has boundary conditions given by
λ(1/3) = 0 and λ (1/3) = 9λ(1/3) = 0. Its general solution is λ(t) = c1 + c2 e−5t ,
substituting the boundary conditions gives c1 = c2 = 0 therefore λ(t) = 0. This implies
u(t) = 0 and the differential equation for the state x becomes x = 2x. The last equation
solves in x(t) = ke2t with x(0) = k = 1, that is x(t) = e2t .
Let us check if the candidate solution u(t) = 0 is a minimum, looking at the ∂ 2 H/∂u2 =
−2 < 0, shows that u(t) = 0 can not be a minimum, so this problem does not have
solution.
102 OPTIMAL CONTROL

7.5 EXERCISE 5

Solve the following Optimal Control Problem (OCP):

 11
min −x dt x = u x(0) = 0, x(11) = 1, −1 ≤ u ≤ 1.
0

7.5.1 Solution with the Hamiltonian

⎧
⎨t 0≤t≤6
x(t) = , λ(t) = t − 6, u(t) = 1.
⎩−t + 12 6 < t ≤ 12

The Hamiltonian of the problem is

H = −x + λu.

The adjoint equation is λ = −∂H/∂x = 1, which implies that λ(t) = t + c for an

unknown constant c. From the Pontryagin maximum principle u = arg min λu = −signλ.
Hence
⎧
⎪
⎪
⎪
⎨1 t+c<0
u= ? t+c=0
⎪
⎪
⎪
⎩−1 t + c > 0.

The problem can have one or zero switching points, depending on the value of c. If
c ≤ −11 then λ(t) < 0 on the whole [0, 11] and there are not switching points, thus
x(t) = t + k for a constant k to be determined from the boundary conditions. It is clear
that a single line x(t) = t + k can not match x(0) = 0 and x(11) = 1, therefore this case
does not occur. The same conclusion holds for the case c ≥ 0 because x(t) = −t + k
can not meet the boundary conditions. If there is a switching point, it occurs when
λ(t ) = t + c = 0, that is if t = −c. This is possible if and only if c ∈ (−11, 0), so in the
first segment λ < 0, u = 1 and x(t) = t + k for t = [0, −c]. Substituting the left boundary
condition x(t) = t. In the second segment for t ∈ [−c, 11] we have u = −1 because
λ > 0, thus x(t) = −t + k and the right boundary condition yields k = 12. From the fact
that x(t) is a continuous function, the two segments have to join for t = t = −c, i.e.
−c = c + 12 which gives c = −6.

7.6 EXERCISE 6

Solve the following Optimal Control Problem (OCP):

 4
min u2 + 2u − x dt x = x + u x(0) = 1, u ≥ −3.
0
 7.7 E X E R C I S E 7 103

7.6.1 Solution with the Hamiltonian

 4 
e 1 t e4 −t 3 −3 + e4 e−t
x(t) = − e − e + , λ(t) = 1 − e4 e−t , u(t) = .
4 2 4 2 2
The Hamiltonian of the problem is

H = u2 + 2u − x + λ(x + u)

The adjoint equation is λ = −∂H/∂x = 1 − λ with λ(4) = 0, which implies that

λ(t) = 1 + c2 e−t for an unknown constant c. Imposing the boundary condition gives
c = −e4 , thus λ(t) = 1 − e4 e−t . The control is quadratic in the Hamiltonian and it can
be solved by ∂H/∂u = 2u + 2 + λ = 0, that implies

−2 − λ −2 − 1 + e4 e−t −3 + e4 e−t
u= = = > −3.
2 2 2
We have to solve now the differential equation for the state x, x = x + u. We do it
applying the Laplace transform.

e4 1 3 e4 1 3
sX − x(0) = X + · − ⇐⇒ (s − 1)X = 1 + · − .
2 s + 1 2s 2 s + 1 2s
Dividing for the LHS and performing partial fraction decomposition we have

1 e4 1 3 1
X = + · 2 − ·
s−1 2 s − 1 2 s(s − 1)
 
1 e4 1 3 1 1
= + · 2 − −
s−1 2 s −1 2 s−1 s
1 1 e4 1 3 1
=− · + · 2 + · .
2 s−1 2 s −1 2 s
The solution is obtained by the inverse Laplace transform and is

1 e4 3
x(t) = − et + sinh t +
2 2 2
 4 
e 1 t e4 −t 3
= − e − e + .
4 2 4 2

7.7 EXERCISE 7

Solve the following Optimal Control Problem (OCP):

 2
min x(2) + x − u dt x  = 1 + u2 x(0) = 1.
0
104 OPTIMAL CONTROL

7.7.1 Solution with the Hamiltonian

1 11 1
x(t) = t + + , λ(t) = −t + 3, u(t) = .
4(3 − t) 12 2(3 − t)
The Hamiltonian of the problem is

H = x − u + λ(1 + u2 )

The adjoint equation is λ = −∂H/∂x = −1 with λ(2) = 1, which implies that λ(t) =
−t + c for an unknown constant c. From the final condition we have that c = 3 and
λ(t) = −t + 3. The control is quadratic in the Hamiltonian and it can be solved by
∂H/∂u = −1 + 2λu = 0, that implies
1 1
u(t) = = .
2λ(t) 2(3 − t)

The differential equation for the state is x = 1 + u2 which yields to

1 1 1
x (t) = 1 + =⇒ x(t) = t + · +k
4(3 − t) 2 4 3−t

for a constant k that can be retrieved by the initial condition x(0) = 1, i.e. x(0) =
1
0 + 12 + k = 1 =⇒ k = 1112
.

7.8 EXERCISE 8

Solve the following Optimal Control Problem (OCP):

 2
min 3x + 2u dt x = e−u + t3 x(1) = e−2 , 2 ≤ u ≤ 3.
1

7.8.1 Solution with the Hamiltonian

1 1
x(t) = t4 + e−2 t − , λ(t) = −3t + 6, u(t) = 2.
4 4
The Hamiltonian of the problem is

H = 3x + 2u + λ(e−u + t3 ).

The adjoint equation is λ = −∂H/∂x = −3 with λ(2) = 0, which implies that λ(t) =
−3t + c for an unknown constant c. From the final condition we have that c = 6 and
λ(t) = −3t + 6. The control is not linear in the Hamiltonian and it can be solved by
∂H/∂u = 2 − λe−u = 0, that implies
 
2 −u −3t + 6
=e ⇐⇒ −u = ln 2 − ln λ =⇒ u(t) = ln(−3t + 6) − ln 2 = ln ,
λ 2
 7.9 E X E R C I S E 9 105

but this control violates the constraint 2 ≤ u ≤ 3, in facts it results u < 2, therefore
u(t) = 2. The differential equation for the state is x = e−2 + t3 which yields to
1 1 1
x(t) = t4 + e−2 t + k =⇒ x(1) = e−2 = + e−2 + k =⇒ k = − .
4 4 4

7.9 EXERCISE 9

Solve the following Optimal Control Problem (OCP):

 2
et
min −2xet dt x = + x x(1) = 0, 1 ≤ u ≤ 2.
1 u

7.9.1 Solution with the Hamiltonian

(t − 1)et
x(t) = , λ(t) = et − e4 e−t , u(t) = 2.
2
The Hamiltonian of the problem is
 
t et
H = −2xe + λ +x .
u(t)

The adjoint equation is λ = −∂H/∂x = 2et − λ with λ(2) = 0, which implies that
λ(t) = c1 et + c2 e−t for an unknown constant c1 , c2 . From the condition λ(2) = 0 we have
that c1 = 1 and c2 = −e4 (in facts λ + λ = 2c1 et = 2et ), hence λ(t) = et − e4 e−t . The
control can be solved by Pontryagin theorem and gives

λet e2t − e4
= ≤0
u u
for t ∈ [1, 2] and u > 0, therefore it violates the constraint and thus u = 2. The
differential equation for the state yields to

(t − 1)et
x(t) = .
2