Optimal Control

Lecture 9

Optimal Control for Continuous Time Dynamical Systems: Partially fixed terminal state,
free terminal time cases

John T. Wen

February 12, 2002

JTW-OC9 RPI ECSE 6440 Optimal Control.

Outline

 time invariant case

 partially fixed terminal state problem

 free terminal time problem (including minimum time)

 a static optimization perspective

 examples

Ref: Bryson & Ho, Sec. 2.7-2.8

Last Time

 unicycle examples: simplify the TPBVP as much as possible (reducible to an alge-

braic nonlinear least square problem).

February 12, 2002Copyrighted by John T. Wen Page 1

JTW-OC9 RPI ECSE 6440 Optimal Control.

Time Invariant Case

Along the optimal trajectory,

∂H ∂H ∂H ∂H
Ḣ = + u̇ + λ̇ + ẋ
∂t ∂u ∂λ ∂x
∂H ∂H
= + u̇ + f T λ̇ λ̇T f
∂t ∂u
∂H
= :
∂t
If f and L do not depend on t explicitly, then ∂H
∂t = 0, or H is a constant along the optimal
trajectory. This is used in the minimum drag nose shape problem.

February 12, 2002Copyrighted by John T. Wen Page 2

JTW-OC9 RPI ECSE 6440 Optimal Control.

Fixed or Partially Fixed Terminal State

Consider the minimization problem

Z tf
J = φ(x(t f ); t f ) + L(x(t ); u(t ); t ) dt
t0

subject to the dynamical constraint

ẋ(t ) = f (x(t ); u(t ); t ); x(t0 ) = x0 :

Suppose, in addition, a function of the final state is specified (e.g., certain component).
For example, for a point-to-point motion control problem, final velocity may be specified
to be zero. Let the constraint be ψ(x(t f ); t f ) = 0 where ψ 2 R q . We can include the
constraint through an additional Lagrange multiplier ν:
Z tf  
J = φ(x(t f ); t f ) + ν ψ(x(t f ); t f ) +
T
L(x(t ); u(t ); t ) + λ (t ) ( f (x(t ); u(t ); t ) ẋ(t )) dt :
T
t0

February 12, 2002Copyrighted by John T. Wen Page 3

JTW-OC9 RPI ECSE 6440 Optimal Control.

Fixed or Partially Fixed Terminal State (cont.)

The first order condition yields the same Euler-Lagrange Equations as before, except with
respect to x(t f ):
 T " T #
∂φ ∂ψ
λ(t f ) = + ν :
∂x(t f ) ∂x(t f )
The TPBVP now involves 2n ODE’s and q algebraic equations ψ(x(t f ); t f ) = 0 (and q
additional variables in ν).
Special case: ψi (x(t f )) = xi (t f ) and φ = φ(xq+1 ; : : : ; xn ). Then
8
< ν i = 1; : : : ; q
i
λi (t f ) = ∂φ
: i = q + 1; : : : ; n:
∂xi (t f )

February 12, 2002Copyrighted by John T. Wen Page 4

JTW-OC9 RPI ECSE 6440 Optimal Control.

Free terminal time problem

When t f is free, we need to include δt f in the first optimality condition for the addition
equation (called Transversality Condition):
 
∂φ 
Free terminal state case: + H  =0
∂t f tf
Fixed terminal state case: H jt f =0 (H = 0 in the time invariant case.)
 
∂φ 
T ∂ψ
Partially fixed terminal state: +ν + H  = 0:
∂t f ∂t tf

February 12, 2002Copyrighted by John T. Wen Page 5

JTW-OC9 RPI ECSE 6440 Optimal Control.

Minimum time control

Consider the minimum time control problem with

Z tf
J= dt ; ẋ = f (x; u); x(0) = x0 ; x(t f ) = x f :
0

Hamiltonian: H = 1 + λT f (x; u).

 T
∂f
Costate: λ̇ = ∂x λ
 T
∂f
Control: ∂u λ=0
Transversality: H (t f ) = 0
Time invariance: H (t ) = constant = 0.

February 12, 2002Copyrighted by John T. Wen Page 6

JTW-OC9 RPI ECSE 6440 Optimal Control.

Examples

1. Zermelo’s problem (Example 1 in Sec.2.7 in B& H)

2. Brachistochrone problem (Problems 2 and 6 in Sec 2.7 in B& H)

February 12, 2002Copyrighted by John T. Wen Page 7

JTW-OC9 RPI ECSE 6440 Optimal Control.

Another Approach to Free Terminal Time Problem

Let t f be a new state variable, i.e., x n+1 = t f . Then

ẋn+1 = 0:

Define τ = t t
t f = xn+1 and xτ (τ) = x(xn+1 τ). State dynamics in terms of τ:
2 3 2 3 2 3 2 3
dxτ
xn+1 f (xτ (τ); u(xn+1τ); xn+1 τ) xτ (0) x0
4 dτ 5 = 4 5; 4 5=4 5:
dxn+1
dτ 0 xn+1 (0) xn+1 (1)

The optimal control problem is now a fixed terminal time problem:

Z1
J = φ(xτ (1); xn+1 ) + xn+1 L(xτ (τ); u(xn+1 τ); xn+1 τ) dτ:
0

Note that if the original system is linear, the transformed system is bilinear.

February 12, 2002Copyrighted by John T. Wen Page 8

JTW-OC9 RPI ECSE 6440 Optimal Control.

Static Optimization Perspective

We have seen the optimal constrol problem can be converted to an equivalent TPBVP
which can in turn be posed as a nonlinear least square problem involving n variables (e.g.,
final or inital costates).
Alternatively, we can take a static perspective (as in HW#2). For a given input u =

u(t ) : 0  t  t f . Let the solution of ẋ = f (x; u; t ), x(t 0 ) = x0 , be x(t ) = Φ(t; u; x0 ; t0 ).
Then, for given t0 and x0 ,
Z tf
J = φ(x(t f ); t f ) + L(x(t ); u(t ); t ) dt
0

can be written as a function of u:

J = J1 (u)
which is a static functional optimization problem. To solve this problem, some finite
dimensional approximation of u is needed.

February 12, 2002Copyrighted by John T. Wen Page 9

JTW-OC9 RPI ECSE 6440 Optimal Control.

Static Optimization (cont.)

Typically, u is written as a linear combination of basis functions:

`
u(t ) = ∑ λi ψi (t ) = ΨT (t )λ:
i=1

The problem then reduces to a finite dimensional static optimization problem:

J = J1 (ΨT λ) = J2 (λ):

Typical choices of ψi : Fourier basis, Legurre polynomials, wavelet functions, Walsh func-
tions, radial basis functions (RBF), etc.

February 12, 2002Copyrighted by John T. Wen Page 10