Bounded Controls

A maximization problem

Z t1
max f (t, x (t ) , u (t )) dt (1)
t0

subject to x 0 (t ) = g (t, x (t ) , u (t )) , x (t0 ) = x0 , (2)

a u b. (3)
where f and g are assumed to be continuously di¤erentiable functions of
three arguments.

() May 23, 2023 1 / 17

Let u (t ) and x (t ) are the optimal control and state, respectively.
Hence we have

Z t1
∆J = J J = f (t, x, u ) + λg (t, x, u ) + x λ0
t0
f (t, x , u ) λg (t, x , u ) x λ0 dt λ (t1 ) δx (t1 )
Z t1
= fx + λgx + λ0 (x x )
t0
+ (fu + λgu ) (u u )] dt λ (t1 ) δx (t1 ) (4)

() May 23, 2023 2 / 17

Let λ obey the liner di¤erential equation

λ 0 (t ) = [fx (t, x , u ) + λ (t ) gx (t, x , u )] , λ (t1 ) = 0. (5)

Equations (4) and (5) imply

Z t1
[fu (t, x , u ) + λgu (t, x , u )] δudt 0. (6)
t0

It is obvious that δu 0, whenever u = a; δu 0, whenever u = b;

δu is unrestricted whenever a < u < b.
Thus, u = a only if fu + λgu 0; a < u < b only if fu + λgu = 0;
u = b only if fu + λgu 0.

() May 23, 2023 3 / 17

 Let us construct the Hamiltonian

H (t, x (t ) , u (t ) , λ (t )) = f (t, x (t ) , u (t )) + λg (t, x (t ) , u (t )) . (7)

Equation (6) implies that

∂H
= 0 results in (6) when a < u < b. (8)
∂u
∂H
= λ0 results in (5). (9)
∂x
∂H
= x 0 results in (2). (10)
∂λ

() May 23, 2023 4 / 17

 A maximization problem:

max H = f + λg (11)
subject to a u b. (12)

Construct the Lagrangian

L = f + λg + w1 (b u ) + w2 ( u a) . (13)
The …rst-order conditions are:

∂L
= fu + λgu w1 + w2 = 0, (14)
∂u

w1 0, w1 (b u ) = 0, (15)

w2 0, w2 (u a) = 0. (16)
() May 23, 2023 5 / 17
 Example 1:

Z T
min c1 u 2 + c2 x dt (17)
0

subject to x 0 (t ) = u (t ) , x (0) = 0, x (T ) = B, u (t ) 0. (18)

Example 2:

Z T
rt rT
max e [R M (u (t )) h (t )] [1 F (t )] dt + e S (T ) [1 F (T )]
0
(19)
subject to F 0 (t ) = [1 u (t )] h (t ) [1 F (t )] , 0 u 1. (20)

() May 23, 2023 6 / 17

Further Control Constraint

A maximization problem:

Z t1
max f (t, x, u ) dt (21)
t0

xi0 = gi (t, x, u ) , i = 1, ,n (22)

h (t, u ) 0, (23)

x (t0 ) = x0 . (24)

() May 23, 2023 7 / 17

Let u (t ) and x (t ) are the optimal control and state vector,
respectively.
Therefore, we have

Z t1
∆J = J J = f (t, x, u ) + λg (t, x, u ) + x λ0
t0
f (t, x , u ) λg (t, x , u ) x λ0 dt
λ (t1 ) [x (t1 ) x (t1 )]
Z t1
= fx + λgx + λ0 (x x )
t0
+ (fu + λgu ) (u u )] dt λ (t1 ) δx1 (25)

() May 23, 2023 8 / 17

Let λ obey the liner di¤erential equation

λ 0 (t ) = [fx (t, x , u ) + λ (t ) gx (t, x , u )] , λ (t1 ) = 0. (26)

Equations (25) and (26) imply

!
m n
∑ ∂f /∂uj + ∑ λi ∂gi /∂uj δuj 0. (27)
j =1 i =1

() May 23, 2023 9 / 17

If h (t, u1 , , um ) > 0, then

n
∂f /∂uj + ∑ λi ∂gi /∂uj = 0. (28)
i =1

If h (t, u1 , , um ) = 0, then we require

m
dh = ∑ (∂h/∂uj ) δuj 0. (29)
j =1

() May 23, 2023 10 / 17

Apply Farkas’Lemma, we get

n
∂f /∂uj + ∑ λi ∂gi /∂uj + w ∂h/∂uj = 0, j = 1, ,m (30)
i =1

Combining (27) and (29), we have

n
∂f /∂uj + ∑ λi ∂gi /∂uj + w ∂h/∂uj = 0, j = 1, ,m (31)
i =1

w 0, wh = 0. (32)

() May 23, 2023 11 / 17

Let us construct the Lagrangian

L = f (t, x, u ) + λg (t, x, u ) + wh (t, x, u ) (33)

This Lagrangian implies that

∂L
= 0 results in (30). (34)
∂u
∂L
= λi0 results in (26). (35)
∂xi
∂L
= xi0 results in (22). (36)
∂λi

() May 23, 2023 12 / 17

A maximization problem:

Z t1
max f (t, x, u ) dt (37)
t0

xi0 = gi (t, x, u ) , i = 1, ,n (38)

hj (t, x, u ) 0, j = 1, ,p (39)

x (t0 ) = x0 . (40)

() May 23, 2023 13 / 17

 De…ne

Z t1
J= f + λg λx 0 + wh dt, (41)
t0

where w 0, wh = 0.
Therefore we have

Z t1
δJ = fx + λgx + λ0 + whx δx
t0
+ (fu + λgu + whu ) δu ] dt λ (t1 ) δx1 (42)

() May 23, 2023 14 / 17

Let λ obey the liner di¤erential equation

λ 0 (t ) = [fx + λ (t ) gx + whx ] , λ (t1 ) = 0. (43)

Hence we …nd

fu + λgu + whu = 0. (44)

() May 23, 2023 15 / 17

Let us construct the Lagrangian

n p
L = f (t, x, u ) + ∑ λi gi (t, x, u ) + ∑ wj hj (t, x, u ) (45)
i =1 j =1

This Lagrangian implies that

∂L
= 0 results in (44). (46)
∂u
∂L
= λi0 results in (43). (47)
∂xi
∂L
= xi0 results in (38). (48)
∂λi

() May 23, 2023 16 / 17

Assignments
Kamien and Schwartz (Part II)

Exercise 10.4, 11.3, 11.4.

() May 23, 2023 17 / 17