Professional Documents
Culture Documents
ECE 307 - Techniques For Engineering Decisions: George Gross
ECE 307 - Techniques For Engineering Decisions: George Gross
ECE 307 - Techniques For Engineering Decisions: George Gross
Engineering Decisions
Dynamic Programming
George Gross
Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
DYNAMIC PROGRAMMING
making
1
STAGES AND STATES
decision
dn
variable
sn stage n
state
(input)
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
2
RETURN FUNCTION
A decision dn at the stage n transforms the state
sn at the current stage n into the state sn+1 at
the stage n + 1
The state sn and the decision dn are associated
with the value of the objective; the effect is
measured by the return function denoted by
rn ( s n , d n )
The optimal decision at stage n is the decision
d *n that optimizes the return function for state sn
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
RETURN FUNCTION
dn decision
variable
sn stage n
state return
rn ( d n, sn)
(input) function
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
3
ROAD TRIP EXAMPLE
A poor student is traveling from NY to LA
To minimize costs, the student plans to sleep
at friends’ houses each night in cities along the
trip
Based on past experience he can reach
Columbus, Nashville or Louisville after 1
day
Kansas City, Omaha or Dallas after 2 days
San Antonio or Denver after 3 days
LA after 4 days
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
680 610
2 Columbus 5 K. City 8 Denver
580
550 515 1030
790
900 6 Omaha
NY 1 3 Nashville 760 10 LA
700 790
770 510 1050 1390
940
660 790
830
4 Louisville 7 Dallas 9 S. Antonio
270
4
ROAD TRIP
⎧ ⎫
⎪ ⎪
stage 3 : f 3 (5) = min ⎨(610 + 1030),(790 + 1390)⎬ = 1640
⎪⎩ 1640 2180 ⎪⎭
⎧ ⎫
⎪ ⎪
f 3 (6) = min ⎨(540 + 1030),(940 + 1390)⎬ = 1570
⎪⎩ 1570 2330 ⎪⎭
⎧ ⎫
⎪ ⎪
f 3 (7) = min ⎨(790 + 1030),(270 + 1390)⎬ = 1660
⎪⎩ 1820 1660 ⎪⎭
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
5
ROAD TRIP EXAMPLE CALCULATIONS
stage 2 :
⎧⎪ ⎫⎪
f 2 (2) = min ⎨(680 + 1640) , (790 + 1570) , (1050 + 1660) ⎬ = 2320
⎩⎪ 2320 2360 2710 ⎭⎪
⎧⎪ ⎫⎪
f 2 (3) = min ⎨(580 + 1640) , (760 + 1570) , (660 + 1660)⎬ = 2220
⎩⎪ 2220 2330 2320 ⎭⎪
⎧⎪ ⎫⎪
f 2 (4) = min ⎨(510 + 1640) , (700 + 1570) , (830 + 1660)⎬ = 2150
⎩⎪ 2150 2270 2490 ⎭⎪
stage 1 :
⎧⎪ ⎫⎪
f1 (1) = min ⎨(1550 + 2320) , (900 + 2220) , (770 + 2150) ⎬ = 2870
⎩⎪ *2870* 3120 292 ⎭⎪
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
6
WORKING BACKWARDS: PICK UP
MATCHES GAME
There are 30 matches on a table and 2 players
match
win?
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
possible situations:
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
7
WORKING BACKWARDS: PICK UP
MATCHES GAME
1 ⇒ 4 left P1 removes 3
P2’s move is to pick 2 ⇒ 3 left P1 removes 2
3 ⇒ 2 left P1 removes 1
We can reason similarly for the cases of 9, 13,
17, 21, 25, and 29 matches
Therefore, P1 wins if P1 picks 30 – 29 = 1 match
in the first move
In this manner, we can assure a win for any
arbitrary number of matches in the game
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
8
OIL TRANSPORT TECHNOLOGY
intermediate
region
oil
substations
storage final
destinations
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
9
OIL TRANSPORT TECHNOLOGY
Possible approaches to solving such a problem
include:
enumeration: exhaustive evaluation of all
possible paths; too costly since there are
more than 100 possible paths
myopic decision rule: at each node, pick
as the next mode the one reachable by the
cheapest path (in case of ties the pick is
arbitrary) ; for example,
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
0 3 11 15 19 26 29 31 36
oil I-E II-E III-D IV-E V-D VI-D VII-C B
storage
but such a path is not unique and not
guaranteed to be optimal
serial dynamic programming (DP) : we
need to construct the problem solution by
defining the stages, states and decisions
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
10
DP SOLUTION
We define a stage to represent each pumping
region and so each stage corresponds to the
set of vertical nodes I, II, . . . , VII
We use backwards recursion: start from a final
destination and work backwards to the oil stage
We define a state to denote a particular
pumping station ↔ sk
A decision refers to the selection of the branch
from each state sk , so there are at most three
choices for a decision dk :
L ↔ left F ↔ forward R ↔ right
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
11
DP SOLUTION:
STAGE 1 REGION VII
optimal decision return
d1
s1 d *1 f *1 ( s1 )
R L F
A 7 R 7
B 6 3 F 3
C 7 5 6 L 5
D 6 5 3 F 3
E 7 8 5 F 5
F 4 2 6 L 2
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION
STAGE 2 REGION VI
optimal decision
cumulative cost in proceeding
from s2 to a final destination
d2
s2 d 2* f 2* ( s2 )
R L F
A 10 12 R 10
B 9 12 7 F 7
C 5 6 7 R 5
D 8 7 6 F 6
E 7 6 11 L 6
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
12
OIL TRANSPORT : STAGE 2
⎛ ⎞
f ( s2 ) = min r2 ( s2 , d 2 ) + f 1 ( s1 ) ⎟
* ⎜ *
d2 ⎜
⎟
2
⎝ ⎠
a function of only s1
⇓
for given d 2 , s1 is set
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION:
STAGE 3 REGION V
{
f 3* ( s3 ) = min r3 ( s3 , d 3 ) + f 2* ( s2 )
d3
}
d3
s3 d *3 f *3 ( s3 )
R L F
A 14 16 R 14
B 14 17 15 R 14
C 10 5 6 R 10
D 9 12 9 R, F 9
E 12 15 L 12
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
13
DP SOLUTION:
STAGE 4 REGION IV
{
f 4* ( s4 ) = min r4 ( s4 , d 4 ) + f 3* ( s3 )
d4
}
d4
s4 d 4* f 4* ( s4 )
R L F
B 17 18 23 R 17
C 15 22 16 R 15
D 18 17 16 F 16
E 16 21 L 16
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION:
STAGE 5 REGION III
{
f 5* ( s5 ) = min r5 ( s5 , d 5 ) + f 4* ( s4 )
d5
}
d5
s5 d *5 f *5 ( s5 )
R L F
A 19 R 19
B 18 18 R,F 18
C 24 23 17 F 17
D 20 19 25 L 19
E 21 17 F 17
F 20 L 20
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
14
DP SOLUTION:
STAGE 6 REGION II
{
f 6* ( s6 ) = min r6 ( s6 , d 6 ) + f 5* ( s5 )
d6
}
d6
s6 d *6 f 6* ( s6 )
R L F
A 25 R 19
B 21 25 18 R,F 18
C 28 21 23 F 17
D 27 26 29 L 19
E 26 23 22 F 17
F 18 23 L 20
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION:
STAGE 7 REGION I
{
f 7* ( s7 ) = min r7 ( s7 , d 7 ) + f 6* ( s6 )
d7
}
d7
s7 d *7 f 7* ( s7 )
R L F
A 27 32 R 27
B 26 33 26 R,F 26
C 34 25 27 L 25
D 25 27 33 R 25
E 27 35 30 R 27
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
15
THE OPTIMAL TRAJECTORY
f 8* ( s8 ) = min{27 + 6, 26 + 4, 25 + 7, 25 + 8, 27 + 3 }
= 30
oil
storage
I II III VI V VI VII
B A A B C D D
16
OIL TRANSPORT PROBLEM
SOLUTION
We obtain the diagram shown on the next slide
by retracing the steps at each stage
The solution
provides all the optimal solutions
is based on logically breaking up the problem
into stages with computation in each stage
being a function of the number of states in the
stage
provides all the suboptimal paths
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
3 3
6 3
9 6
oil substations
storage
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
17
OIL TRANSPORT PROBLEM
SOLUTION
different than D
2 7 final
VI - D VII - C destination
D
and so the sub optimal cost solution costs are 33
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
18
FACILITIES SELECTION
and considers:
3 location alternatives
building type
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
FACILITIES SELECTION
size
19
FACILITIES SELECTION
building size
B1 B2 B3 B4 none
site
R1 C1 R2 C2 R3 C3 R4 C4 R0 C0
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
20
DP SOLUTION APPROACH
parameters
DP SOLUTION APPROACH
stage ↔ site
amount of funds available
state ↔
for construction
decision ↔ building type
profits attained, i.e., the
return function ↔
revenues
impact of a decision on the
transition function ↔
availablity of resources
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
21
DP SOLUTION APPROACH
As we move from stage n backwards to stage n-1
after a decision dn is taken the funds available
for construction are
sn−1 = sn − cn ← costs of decision d n
The recursion relation is
{
f n* ( sn ) = max f n ( sn , d n ) + f n−*1 ( sn−1 )
dn
} n = 1,2, 3
with
s n − 1 = sn − c n
and
f n ( sn , d n ) = rn ( sn , d n ) = Rn← profits of decision d n
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION APPROACH
We use backwards DP and start with site I, a
purely arbitrary choice, as stage 1, where this
stage 1 represents the last decision in the 3 -
stage sequence
22
DP SOLUTION : STAGE 1 SITE I
f1 ( s1 ) = max { r1 ( s1 , d1 )}
0 ≤ d1 ≤ 4 R1
s1 0 1 2 3 4 d 1* f 1* ( s1 )
21 ≥ s1 ≥ 5 0 .5 .65 .80 1.4 4 1.4
4 ≥ s1 ≥ 3 0 .5 .65 .80 3 .80
2 0 .5 .65 2 .65
1 0 .5 1 .50
0 0 0 0 0 0 0 0
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
f 2* ( s2 ) = max { r2 ( s2 , d 2 ) + f 1* ( s1 )}
0 ≤ d2 ≤ 4 R2
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
23
DP SOLUTION : STAGE 2 SITE II
s2 0 d2 1 2 3 4 dd **22 ff**22(S
( s 22))
21 ≥ s ≥ 13 1.40
2 2.02 2.18 2.36 3.20 4 3.20
12 1.40 2.02 2.18 2.36 2.60 4 2.60
11 1.40 2.02 2.18 2.36 2.60 4 2.60
10 1.40 2.02 2.18 1.76 2.45 4 2.45
9 1.40 2.02 1.58 1.61 2.30 4 2.30
8 1.40 2.02 1.58 1.61 1.80 1 2.02
7 1.40 2.02 1.43 1.61 1 2.02
6 1.40 1.42 1.28 1.46 3 1.46
5 1.40 1.42 0.78 0.96 1 1.42
4 0.80 1.27 1 1.27
3 0.80 1.12 1 1.12
2 0.65 0.62 0 0.65
1 0.50 0 0.50
0 0.00 0 0.00
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
SAMPLE CALCULATIONS
Consider the case s2 = 10 and d2 = 0 :
C2 = 0 and R2 = 0
then, s1 = 10 and d 1* = 4 so that
f 1* ( s1 ) = 1.4
consequently, f2(s2) = 1.4
Consider the case s2 = 10 and d2 = 4 :
C2 = 8 and R2 = 1.8
then, s1 = 2 and d 1* = 2 so that
f 1* ( s1 ) = .65
consequently, f2(s2) = 2.45
We can, then, show that
f 2* ( s2 ) = 2.45
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
24
DP SOLUTION : STAGE 3 SITE III
At stage 3 , the first actual decision is made and
so exactly 21 million is available and s3 = 21
We compute the elements in the table using
f 3* ( s3 ) = max{ r3 ( s3 , d 3 ) + f *2 ( s2 )}
d3 N
where R3
s2 = s 3 − C 3
d3
d 3* f *3 ( s3 )
s3 0 1 2 3 4
21 3.2 3.91 4.4 4.1 4.45 4 4.45
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
OPTIMAL SOLUTION
Optimal profits are 4.45 million
The optimal decision is obtained by retracing
steps from stage 3 to stage 1:
d *3 = 4 ↔ construct B2 at site III
s2 = s3 – C3 = 21 – 11 = 10
d *2 = 4 ↔ construct B4 at site II
s1 = s2 – C2 = 10 – 8 = 2
d 1* = 2 ↔ construct B2 at site 1
C1 = 5 and C1+C2+C3 = 21
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
25
SENSITIVITY CASE
We next consider the case where the maximum
investment available is 15 million
By inspection, the results in stages 1 and 2
remain unchanged; however, we must recom-
pute stage 3 results with the 15 million limit
d3
s3 d 3* f *3 ( s3 )
0 1 2 3 4
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
SENSITIVITY CASE
The optimal solution for this sensitivity case
obtains maximum profits of 3.31 million and the
decision is
d *3 = 1 ↔ construct B1 at site III
s 2 = s3 − C 3 = 15 − 4 = 11
d *2 = 4 ↔ construct B4 at site II
s1 = s2 − C 2 = 11 − 8 = 3
d 1* = 3 ↔ construct B3 at site I
C1 = 3 and C1 + C 2 + C 3 = 15
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
26
OPTIMAL CUTTING STOCK PROBLEM
A paper company gets an order for:
DP SOLUTION APPROACH
A stage is an order and since there are 4 orders
we construct a 4 – stage DP
d4 d3 d2 d1
r4 r2 r3
r1
A state in stage n is the remaining ft of paper
left for the order being processed at stage n and
all the remaining stages
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
27
DP SOLUTION APPROACH
A decision in stage n is the amount of rolls to
produce in stage n :
⎡F ⎤ F
d n = ⎢ 0 ⎥ , the largest integer in 0
⎣ Ln ⎦ Ln
where
Ln = length of order n ( ft )
DP SOLUTION APPROACH
The return function at stage n is the additional
revenues gained from producing dn rolls
The transition function measures amount of
paper remaining at stage n
sn − 1 = sn − d n Ln n = 2,3,4
s0 = s1 − d1 L1
⎡s ⎤
Clearly, d1 = ⎢ 1 ⎥
⎣ L1 ⎦
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
28
DP SOLUTION APPROACH
The recursion relation is
f n* ( sn ) = max {r ( s , d
n n n ) + f n−*1 ( sn−1 ) }
⎡ ⎤
0 ≤ dn ≤ ⎢ sn ⎥
L
⎣ n⎦
n = 1, … , 4
where
sn−1 = sn − d n Ln
and
f 0* ( s0 ) = 0
f n ( sn , d n ) = rnd n + f n−*1 ( sn − d n Ln )
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION APPROACH
stage n 1 2 3 4
length of
2.5 4 3 2
order ( ft )
29
DP SOLUTION : STAGE 1
f 1* ( s1 ) = max
0 ≤ d1 ≤ 5
{r1 ( s1 , d1 )} = max {3.10 d1 }
0 ≤ d1 ≤ 5
⎡ 13 ⎤
d1 ≤ ⎢ = 5
⎣ 2.5 ⎥⎦
s1
f *1 ( s 1 )
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP : SOLUTION STAGE 2
f *2 ( s2 ) = max
0 ≤ d2 ≤ 3
{5.25 d 2 + f 1* ( s2 − 4 d 2 ) }
⎡ 13 ⎤
d2 ≤ ⎢ ⎥ = 3
⎣4⎦
s2
f *2 ( s 2 )
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
30
DP SOLUTION : STAGE 3
f *3 ( s3 ) = max
0 ≤ d3 ≤ 4
{4.40 d 3 + f *2 ( s3 − 3 d 3 ) }
⎡ 13 ⎤
d3 ≤ ⎢ ⎥ = 4
⎣3⎦
s2
f *3 ( s 3 )
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION : STAGE 4
f *4 ( s4 ) = max
0 ≤ d4 ≤ 6
{ 2.5 d 4 + f *3 ( s4 − 2 d 4 ) }
⎡ 13 ⎤
d4 ≤ ⎢ ⎥ = 6
⎣2⎦
*
d4 0 1 2 3 4 5 6 d 4* f 4 ( s4 )
s4 = 13 18.45 17.5 18.2 17.15 16.2 16.9 15 0 18.45
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
31
DP OPTIMAL SOLUTION
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
SENSITIVITY CASE
32
SENSITIVITY CASE : STAGE 4′
⎡ 11 ⎤
d 4′ ≤ ⎢ ⎥ = 5
⎣2⎦
d 4′ 0 1 2 3 4 5 d 4*′ f 4*′ ( s4 )
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
$15.7
d 4*′ = 1, ↔ 1 roll of 2 ft
d 3*′ = 3 ↔ 3 rolls of 3 ft
d 2′ = d 1′ = 0 ↔ no rolls of 4 ft
* *
↔ no rolls of 2.5 ft
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
33
ANOTHER SENSITIVITY CASE
We consider the case with the initial 13 ft, but in
addition we get the constraint that at least 1 roll
of 2 ft must be produced:
d4 ≥ 1
and since s2 = s1 = 0
d 2*′′ = 0 ↔ 0 rolls of 4 ft
d 1*′′ = 0 ↔ 0 rolls of 2.5 ft
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
34
INVENTORY CONTROL PROBLEM
demand 40 60 30 40 30 20
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
lot size 10 20 30 40 50
discount
4 5 10 20 25
%
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
35
INVENTORY CONTROL PROBLEM
There are ordering costs: each order incurs
fixed costs of $ 2 and $ 8 for shipping, handling
and insurance
The storage limitations of the retailer require
that no more than 40 units be in inventory at the
end of the month and the storage charges are
0.2 $/unit; inventory at time 0 and inventory at the
e.o.p. 6 are 0
Assumption: demand occurs at a constant rate
throughout the month
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION APPROACH
backward process
stage
6 5 4 3 2 1
n
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
36
DP SOLUTION APPROACH
d6 d5 d4 d3 d2 d1
r6 r5 r4 r3 r2 r1
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION APPROACH
The state variable at stage n is defined as the
amount entering inventory given that there are
n months remaining – the present month n
plus months n – 1 , n – 2 , . . . , 1
The decision variable at stage n is the amount of
units ordered to satisfy demands for the n
months
The transition function is defined by
sn-1 = sn + dn – Dn n = 1, 2, . . . , 6
s0 = 0 s6 = 0 demand in month n
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
37
DP SOLUTION APPROACH
The return function at stage n is given by
rn ( d n , sn ) = φ ( d n ) + hn ( sn + d n − Dn )
0.2( sn + dn − Dn )
with storage costs
φ ( dn ) = 10
+ [1 − ρ ( d n ) ] d n
fixed discount
costs factor
d n = 10,20,30,40,50
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION APPROACH
dn 0 10 20 30 40 50
{
f n* ( sn ) = min φ (d n ) + hn ⎡⎣ sn + d n − Dn ⎤⎦ + f n−*1 ( sn−1 )
dn
}
n = 1,...,6
s0 = 0 and so f 0* ( s0 ) = 0
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
38
DP SOLUTION : STAGE 1
s0 = 0 ⎫
⎬ ⇒ s1 = 20, 10 or 0 ⇒ d *1 = 0 , 10 or 20
D1 = 20 ⎭
s1 20 10 0
d *1 0 10 20
φ ( d *1 ) 0 48 86
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION : STAGE 2
D2 = 30 and s1 = s2 + d 2 − 30
{
f *2 ( s2 ) = min φ (d 2 ) + 0.2 ⎡⎣ s2 + d 2 − 30⎤⎦ + φ (d *1 )
d2
}
d2
d *2 f 2* ( s2 )
s2 0 10 20 30 40 50
0 204 188 164 50 164
10 172 168 142 40 142
20 134 136 122 122 30 122
30 86 98 90 0 86
40 50 52 0 50
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
39
DP SOLUTION : STAGE 3
D3 = 40 and s2 = s3 + d 3 − 40
*
3
d3
{
f ( s3 ) = min φ (d 3 ) + 0.2 ⎡⎣ s3 + d 3 − 40⎤⎦ + f *2 ( s2 ) }
d3
d *3 f 3* ( s3 )
s3 0 10 20 30 40 50
0 302 304 40 302
10 282 282 286 30, 40 282
20 250 262 264 252 20 250
30 212 230 244 230 218 10 218
40 164 192 212 210 196 0 164
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION : STAGE 4
D4 = 30 and s3 = s4 + d 4 − 30
{
f *4 ( s4 ) = min φ (d 4 ) + 0.2 ⎡⎣ s4 + d 4 − 30⎤⎦ + f *3 ( s3 )
d4
}
d4
d 4* f 4* ( s4 )
s4 0 10 20 30 40 50
0 420 422 414 50 414
10 388 402 392 384 50 384
20 350 370 372 362 332 50 332
30 302 332 340 342 210 0 302
40 284 302 310 290 0 284
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
40
DP SOLUTION : STAGE 5
D5 = 20 and s4 = s5 + d 5 − 20
{
f 5* ( s5 ) = min φ (d 5 ) + 0.2 ⎡⎣ s5 + d 5 − 20⎤⎦ + f *5 ( s5 )
d5
}
d5
d 5* f *5 ( s5 )
s5 0 10 20 30 40 50
0 500 504 474 468 50 468
10 462 472 454 446 452 40 446
20 414 434 422 426 430 0 414
30 286 384 394 410 10 384
40 336 356 378 0 336
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION : STAGE 6
n=6 D6 = 40 and s6 = 0
s5 = s6 + d 6 − 40 = d 6 − 40
{
f *6 ( s6 ) = min φ (d 6 ) + 0.2 ⎡⎣ s6 + d 6 − 40⎤⎦ + f *5 ( s5 )
d
}
6
d6 0 10 20 30 40 50 d *6 f 6* ( s6 )
d *6 = 40 ⇒ d *5 = 50 ⇒ d *4 = 0 ⇒ d *3 = 40 ⇒ d *2 = 50 ⇒ d 1* = 0
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
41
OPTIMAL SOLUTION
d *6 = 40 which corresponds to s5 = 0 and costs 606
d 1* = 0 with costs 0
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
OPTIMAL SOLUTION
d *6 = 40 d *5 = 50 d *4 = 0 d *3 = 40 d *2 = 50 d *1 = 0
s6 = 0 s5 = 0 s4= 30 s3= 0 s2= 0 s1 = 20 s0=0
stage stage stage stage stage stage
6 5 4 3 2 1
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
42
OPTIMAL SOLUTION
optimal trajectory is
s0 = 0 s1 = 20 s2 = 0 s3 = 0 s4 = 30 s5 = 0
10 k$ invested in year 1
43
MUTUAL FUND INVESTMENT
STRATEGIES
We operate under the protocol that
once invested, the money cannot be
withdrawn until the end of the 5-year horizon
all short term gains may be reinvested in
either of the two funds or withdrawn in which
case the withdrawn funds earns no more
further interest
The objective is to maximize the total returns at
the end of 5 years
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
44
MUTUAL FUND INVESTMENT
STRATEGIES
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION APPROACH
d5 d4 d3 d2 d1
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
45
DP SOLUTION METHOD
For stage n , the state sn is the amount of capital
available for investment in the year 6 – n
DP SOLUTION METHOD
The use of backward recursion considers year 5
first and each of the previous years sequentially
Basic considerations:
for each year 6 – n
dn is invested in fund A returns d n i A ( SDT )
( sn − d n )is invested in fund B returns ( sn − d n )i B ( SDT )
for the year 6 – n + 1
sn−1 = d n i A + ( sn − d n ) iB + 1000 n = 2, 3,4,5
s5 = 10,000
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
46
THE OBJECTIVE
The objective is to maximize the total returns
5
max R = ∑r
n =1
n
DP SOLUTION : STAGE 1
For stage 1
r1
s1
d1
where
r1 = (1 + I A )d1 + (1 + I B )( s1 − d1 ) + i Ad1 + i B( s1 − d1 )
= ( I A + i A − I B − iB ) d1 + (1 + I B + i B ) s1
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
47
DP SOLUTION : STAGE 1
r1 = earnings in stage 1 (returns realized at
the end of 5 years)
⎧ d1 ( I A + i A − I B − i B ) + ⎫
f 1* ( s1 ) = max { r1 } = max ⎨ ⎬
d1 d1 ⎩ s1 (1 + I B + i B ) ⎭
⎧ d ( 0.04 + 0.025 − 0.03 − 0.04) + ⎫
= max ⎨ 1 ⎬
0 ≤ d1 ≤ s1 ⎩ s1 (1 + 0.03 + 0.04) ⎭
= max {d1 ( − 0.005) + s1 (1.07)} maximum
optimal d1
return in
decision stage 1
d *1 = 0 f 1* ( s1 ) = 1.07 s1
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION : STAGE 2
r2 = returns realized at the end of 5 years due to
the decision in stage 2
= d 2 (1 + I A )2 + ( s2 − d 2 )(1 + I B )2
= d 2 ⎡⎣(1 + I A )2 − (1 + I B )2 ⎤⎦ + s2 (1 + I B )2
s1 = s2 iB + d 2 ( i A − iB ) + 1,000
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
48
DP SOLUTION : STAGE 2
We select d 2* to maximize
f 2* ( s2 ) = max
d2
{r 2 + f 1* ( s1 ) }
= max
d2
{d 2 (1.042 − 1.032 ) + s2 (1.03)2 + f 1* ( s1 ) }
⎧ d 2 ( .0207) + 1.0609 s2 + ⎫
= max ⎨ ⎬
0 ≤ d2 ≤ s2 ⎩1.07[ .04 s2 + d 2 ( − .015) + 1,000]⎭
= max
d2
{d 2 (.0046) + 1.1037 s2 + 1070}
d 2* = s2 f 2* ( s2 ) = 1.108 s2 + 1070
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION : STAGE 3
r3 = returns realized at the end of 5 years due to
the decision d3
= d 3 (1 + I A )3 + ( s3 − d 3 )(1 + I B )3
= d 3 ⎡⎣(1 + I A )3 − (1 + I B )3 ⎤⎦ + s3 (1 + I B )3
s2 = s3 iB + d 3 ( i A − iB ) + 1,000
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
49
DP SOLUTION : STAGE 3
We select d 3* to maximize
f 3* ( s3 ) = max
d3
{r 3 + f 2* ( s2 ) }
⎧ d 3 (1.043 − 1.033 ) + s3 (1.03)3 + ⎫
= max ⎨ ⎬
d3
⎩1.108 s2 + 1,070 ⎭
= max {2,178 + 1.1481s3 + .0018d 3 }
0 ≤ d3 ≤ s3
d 3* = s3 f 3* ( s3 ) = 1.15 s3 + 2,178
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION : STAGE 4
= d 4 (1 + I A )4 + ( s4 − d 4 )(1 + I B )4
= d 4 ⎡⎣(1 + I A )4 − (1 + I B )4 ⎤⎦ + s4 (1 + I B )4
s3 = s4 i B + d 4 ( i A − iB ) + 1,000
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
50
DP SOLUTION : STAGE 4
We select d 4* to maximize
f 4* ( s4 ) = max
d4
{r + f
4
*
3 ( s3 )}
= max
d4
{d 4 (1.044 − 1.034 ) + s4 (1.03)4 + 1.15 s3 + 2,178 }
= max
0 ≤ d4 ≤ s4
{3328 + 1.1772 s4 + .0156d 4}
d 4* = s4 f 4* ( s4 ) = 1.193 s4 + 3,328
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
DP SOLUTION : STAGE 5
r5 = returns realized at the end of 5 years due to
the decision d5
= d 5 (1 + I A )5 + ( s5 − d 5 )(1 + I B )5
s4 = s5 i B + d 5 ( i A − iB ) + 1,000
= 10,000 iB + d 5 ( i A − iB ) + 1,000
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
51
DP SOLUTION : STAGE 5
We select d 5* to maximize
⎧ ⎫
⎪ ⎪
f 5 ( s5 ) = max ⎨10,000(1.03) + d 5 (1.04 − 1.03 ) + f 4 ( s4 ) ⎬
* 5 5 5 *
0 ≤ d5 ≤ s4 ⎪
⎪⎭
⎩ 11,593 0.0574
⎡⎣1,000 + 600 + d 5 ( −.04)⎤⎦ 1.193 + 3,328
⎧ (.0574 − 0.048) ⎫
= max ⎨16,830 + d 5 ⎬
0 ≤ d5 ≤ s5 ⎩ 0.097 ⎭
= 16,830 + 0.097(10,000)
d 5* = 10,000 f 5* ( s5 ) = 16,927
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
OPTIMAL SOLUTION
optimal return at end of 5 years is 16,927 using
the following strategy
beginning investment in
of year fund A fund B
1 10,000 0
2 STD returns + 1,000 0
3 STD returns + 1,000 0
4 STD returns + 1,000 0
5 0 STD returns + 1,000
ECE 307 © 2006 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.
52