Professional Documents
Culture Documents
A Child'S Guide To Optimal Control Theory
A Child'S Guide To Optimal Control Theory
A Child'S Guide To Optimal Control Theory
Martin Boileau
1. Introduction
This is a simple guide to optimal control theory. In what follows, I borrow freely from
Kamien and Shwartz (1981) and King (1986).
F (x(t); u(t); t) dt
(P )
subject to
x_ i = Qi (x(t); u(t); t);
xi (0) = xi0 ;
xi (T ) 0;
u(t) 2 ;
i = 1; : : : ; n;
i = 1; : : : ; n;
i = 1; : : : ; n;
2 IRm :
(1)
(2)
(3)
(4)
where,
x(t) is a n-vector of state variables (xi (t)). These describes the state of the system at any
point in time. Also, note that dxi =dt = x_ i .
u(t) is a m-vector of control variables. These are the choice variables in the optimization
problem.
F () is a twice continuously dierentiable objective function. Throughout, we assume that
this function is time additive.
Qi () are twice continuously dierentiable transition functions for each state variable.
1
n
X
i Qi (x; u; t):
i=1
i = 1; : : : ; n:
n
X
i=1
i Qi (x; u; t):
In the above, we call i (t) co-state variables. They are analogous to Lagrange multipliers.
The necessary conditions for a maximum are:
@H()
= 0; k = 1; : : : ; m;
@uk
@H()
= _ i (t); i = 1; : : : ; n;
@xi
i (T ) 0; i (T )xi (T ) = 0; i = 1; : : : ; n:
The sucient conditions for a maximum are that the Hamiltonian function be concave in
x and u.
Finally, in some cases, our optimization problem will also include other constraints:
max
F (x(t); u(t); t) dt
(P 1)
subject to
x_ i = Qi (x(t); u(t); t);
Gj (x(t); u(t); t) 0;
xi (0) = xi0 ;
xi (T ) 0;
u(t) 2 ;
i = 1; : : : ; n;
j = 1; : : : ; q;
i = 1; : : : ; n;
i = 1; : : : ; n;
2 IRm :
q
X
j Gj (x; u; t)
j=1
= F (x; u; t) +
n
X
i Qi (x; u; t) +
i=1
q
X
j Gj (x; u; t);
j=1
j (t)Q(x(t); u(t); t) = 0;
j = 1; : : : ; q:
4. An Economic Example
Consider the following problem:
max
et u (c(t)) dt
subject to
a(t)
_
= ra(t) c(t);
a(0) = a0 ;
a(T ) aT :
c(t)
_
u0 (c(t))
= (r )
;
c(t)
c(t)u00 (c(t))
such that
gc (t) = (c(t))(r );
where (c(t)) = u0 (c(t))=[c(t)u00 (c(t))] > 0. Thus, consumption grows whenever the
market discount rate is larger than the subjective discount rate. In that case, the consumer
is less impatient than the market.
4
et f (x(t); u(t)) dt
(P 2)
x_ i = Qi (x(t); u(t));
Gj (x(t); u(t)) 0;
xi (0) = xi0 ;
i = 1; : : : ; n;
(1)
j = 1; : : : ; q;
i = 1; : : : ; n;
(2)
H (x; u; ) = e
f (x; u) +
n
X
i Qi (x; u):
i=1
The Lagrangian is
L(x; u; ; ) = et f (x; u) +
n
X
i Qi (x; u) +
i=1
j (t) 0;
T !1
j (t)Q(x(t); u(t)) = 0;
5
j Gj (x; u):
j=1
q
X
i = 1; : : : ; n:
j = 1; : : : ; q:
n
X
i Qi (x; u):
i=1
The transformation from present value to current value aects the rst-order conditions. To see this, let us restate our problem P 2 using the current value Hamiltonian. The
problem is
Z 1
max
et f (x(t); u(t)) dt
(P 2)
0
subject to
x_ i = Qi (x(t); u(t));
Gj (x; u) 0;
xi (0) = xi0 ;
i = 1; : : : ; n;
(1)
j = 1; : : : ; q;
i = 1; : : : ; n;
(2)
n
X
i Qi (x; u):
i=1
The Lagrangian is
L(x; u; ; ) = f (x; u) +
n
X
i Qi (x; u) +
i=1
q
X
j=1
j Gj (x; u):
j (t) 0;
T !1
j (t)Q(x(t); u(t)) = 0;
i = 1; : : : ; n:
j = 1; : : : ; q:
et u (c(t)) dt
subject to
a(t)
_
= ra(t) c(t);
a(0) = a0 :
Abstracting from the time subscript, the current value Hamiltonian is
H(a; c; ) = u(c) + [ra c] :
Its rst-order conditions are
@H
= 0 =) u0 (c) = 0;
@c
h
i
h
i
@H
= _
=) r = _ ;
@a
lim et (T ) 0;
lim et (T )a(T ) = 0:
T !1
T !1
such that
gc (t) =
c(t)
_
u0 (c(t))
= (r )
c(t)
c(t)u00 (c(t))
or
gc (t) = (c(t))(r );
where (c(t)) = u0 (c(t))=[c(t)u00 (c(t))] > 0. Thus, consumption grows whenever the market discount rate is larger than the subjective discount factor. In that case, the consumer
is less impatient than the market.
8. Summary
To summarize, here is the simple cookbook. Assume that you face the following problem:
max
et f (x(t); u(t)) dt
subject to
x_ i = Qi (x(t); u(t));
xi (0) = xi0 ;
i = 1; : : : ; n;
i = 1; : : : ; n:
H (x; u; ) = e
f(x; u) +
n
X
i Qi (x; u):
i=1
T !1
i = 1; : : : ; n:
n
X
i Qi (x; u):
i=1
T !1
i = 1; : : : ; n:
References
Kamien, Morton I. and Nancy L. Schwartz, 1981, Dynamic Optimization: The Calculus
of Variations and Optimal Control in Economics and Management, New York: North
Holland.
King, Ian P., 1986, A Cranker's Guide to Optimal Control Theory, mimeo Queen's University.
10