1 Optimal control

1.1 Problem definition

Given the system x& (t ) = f (x (t ),u (t ), t ) an admissible control u has to be found so that
x follows an admissible trajectory and minimizes the cost function
tf
J = h(x (t f ), t f ) + ∫ g (x (t ), u (t ), t )dt
t0

1.2 Discrete optimal control

Given the system x k +1 = f (x k ,u k ,) , an admissible control u has to be found so that x

follows an admissible trajectory and minimizes the cost function
N −1
J = h (x N ) + ∑ L ( x k , u k ) .
i =1

2 Dynamic programming
2.1 Principle of dynamic programming

Though dynamic programming, the optimal control is fund reverse, starting from the
final state and back to the initial state, while de optimal control is recorded.
Principle of optimality: A control strategy, which is optimal in the interval [0,N], is
also optimal in the interval [k,N], with 0 ≤ k ≤ N . Presuming that J *k + 1 (x k + 1 ) is the
optimal cost from the state k+1 to the final state, only L(x k ,u k ) + J *k +1 (x k +1 ) has to be
optimized. In this case uk can be calculated from
*
uk
( *
) uk
( *
)
J k (xk ) = min L(xk ,uk ) + J k +1 (xk +1 ) = min L(xk ,uk ) + J k +1 (f (xk ,uk ))
This relation can be used for computing the optimal control, starting with
( x N ) = h (x N ) .
J *N
The result of the above equation is an optimal control found by trying all the
admissible control values for all the admissible states.
2.2 The algorithm (discrete dynamic programming)

1. Define the discrete system

2. Determine a final number of admissible control values u min ≤ u ≤ u max .
3. Determine a final number of admissible state values x min ≤ x ≤ x max .
4. Compute J *N (x N ) = h(x N ) for all possible values of x.
5. For every k=N-1, N-2, … 0
For every admissible state (xj)
For every admissible control value (ui) calculate
a. The next state ( x k +1 = f (x j , u i ,) ).
b. The cost of getting to the next state ( L(x j , u i ) )
c. The cost of getting to the final state ( L(x j , u i ) + J *k +1 (f (x j , u i )) )
The optimal control in step k for the state xj is for which
L(x j ,u i ) + J *k +1 (f (x j ,u i )) = min .
6. Given the initial state ( x (0 ) = x 0 ), and the optimal control values for every state
in every step, reconstruct the state trajectory x 0 , x1 , x 2 ,...x N and the control
trajectory u 0 ,u1 , u 2 ,...u N .