Deterministic dynamic Programming

Hadi Sutanto Saragi S.T M.Eng

Deterministic Dynamic Programming
Dynamic Programming

Dynamic Programming was introduced by Richard Ernest

bellman (1953). Dynamic programming has long been applied to
numerous areas in mathematics, science, engineering,
business, medicine, information systems, biomathematics,
artificial intelligence, among others. Applications of dynamic
programming have increased as recent advances have been
made in areas such as neural networks, data mining, soft
computing, and other areas of computational intelligence. The
value of dynamic programming is to obtain their optimal
solutions.
Dynamic Programming is used when the problem can be solve
in stages/sub problems. Hadi saragi
 Shortest path method
1. Transportation Problem

5
4 7 7
8
9
6 9 9 9
9
10
6 11
1 3 6 10
8
6
6 8 10
7 5
6
2 5 8
8

Hadi saragi
Sortest path method

 Let us apply the backward recursion approach

 We assume that in order to reach the destination (city 10) the person
should start from city 8 or 9
 We now find the best (or least cost way) to reach city 10, if we were at city
8 or 9.
 Since the decision as ‘where to go next’ that depends on the stage of
decision making, we have to make a decision when we have one more
stage to go.

Hadi saragi
 Sortest path method by Knapsack Algorithm

5
4 7 7
8
9
6 9 9 9
9
10
6 11
1 3 6
8 10
6
6 8 10
7 5
6
2 5 8
8

Stage 5 Stage 4 Stage 3 Stage 2 Stage 1

Backward Recursion Approach Hadi saragi
Stage I f1 ( s1 , xn t )  C1 xn t
From To Cost

8 10 10
ƒ1
9 10 9

Stage II f 2 ( s2 , xn t )  C2 xn t  f1 ( s1 , xn t )
From To Cost

5 10 (8) 18

5 10(9) 15*

6 10(8) 18*
ƒ2
6 10(9) 19

7 10(8) 19

7 10(9) 16*
Hadi saragi
Stage III f 3 ( s3 , xn t )  C3 xn t  f 2 ( s2 , xn t )

From To Cost

2 10 (5) 23

2 10(6) 24

2 10(7) 21*

3 10(5) 21*
Ƒ3 3 10(6) 29

3 10(7) 25

4 10(5) 24

4 10(6) 24

4 10(7) 21*

Hadi saragi
Stage IV f 4 ( s4 , xn t )  C4 xn t  f 3 ( s3 , xn t )
From To Cost

1 10 (2) 28
Ƒ4 1 10(3) 27

1 10(4) 27

According to calculation, There is two optimal solution.

First optimal solution 1 – 3 – 5 – 9 – 10

Second optimal solution 1 – 4 – 7 – 9 – 10

Hadi saragi
2. Reliability Problem

 Let us consider an equipment that functions using four components

connected in series.
 Each component has a certain reliability and the reliability of the
equipment is the product of the individuals component reliabilities.
 In order to increase the reliability of the equipment, we can have some
additional components in standby and reliabilities and cost for standby
units for the components is linear value.

Hadi saragi
 exp
There are four component in a production equipment.

No Component A Component B Component C Component D

R C R C R C R C
1 0.6 100 0.7 80 0.7 75 0.8 60
2 0.7 130 0.75 100 0.8 80 0.85 70
3 0.8 150 0.8 120 0.9 85 0.88 75
4 0.9 170 0.85 140 0.95 90 0.9 80

A B C D

Assume we have budget $400 : find the optimal condition to maximize the reliability

Hadi saragi
Stage I f1 ( s1 , x4 )  Rx4

S4 R X4 X4

60 – 69 0.8 1

70 - 74 0.85 2
ƒ1
75- 79 0.88 3

80 - 145 0.90 4

Hadi saragi
 Stage II f 2 ( s2 , x3 )  Rx3 ( f1 ( s1 , x4 ))

S3,4 X4 = 1 X4 = 2 X4 = 3 X4 = 4 F2 X3
(s2,x3)
135 – 139 0.7 x 0.8 - - - 0.56 1*

140 – 144 0.8 x 0.8 - - - 0.64 2*

145 – 149 0.9 x 0.8 0.7 x 0.85 - - 0.72 3

150 – 154 0.95 x 0.8 0.8 x 0.85 0.7 x 0.88 - 0.76 4

ƒ2
155 – 159 0.95 x 0.8 0.9 x 0.85 0.8 X 0.88 - 0.765 3*

160 – 164 0.95 x 0.8 0.95 x 0.85 0.9 x 0.88 0.7 x 0.9 0.8075 4

165 – 169 0.95 x 0.8 0.95 x 0.85 0.95 x 0.88 0.8 x 0.9 0.836 4

170 – 174 0.95 x 0.8 0.95 x 0.85 0.95 x 0.88 0.9 x 0.9 0.836 4

175 – 220 0.95 x 0.8 0.95 x 0.85 0.95 x 0.88 0.95 x 0.9 0.855 4*
Hadi saragi
 S3,4 X4 = 1 X4 = 2 X4 = 3 X4 = 4 F2 X3
(s2,x3)
135 – 139 0.7 x 0.8 - - - 0.56 1*

ƒ2 140 – 144 0.8 x 0.8 - - - 0.64 2*

155 – 159 0.95 x 0.8 0.9 x 0.85 0.8 X 0.88 - 0.765 3*

175 – 220 0.95 x 0.8 0.95 x 0.85 0.95 x 0.88 0.95 x 0.9 0.855 4*
 Stage III f 3 ( s3 , x2 )  Rx2 ( f 2 ( s2 , x3 ))

S2,3,4 X3 = 1 X3 = 2 X3 = 3 X3 = 4 F3 (s3,x2) X2

215 –219 0.7 x 0.56 - - - 0.392 1

220 – 224 - 0.7 x 0.64 - - 0.448 2

235 – 274 - - 0.7 x 0.765 - 0.5355 1*

240 – 300 0.75 x 0.64 0.48 2

ƒ3
255 – 259 0.8 x 0.56 0.75 x 0.765 0.57375 2

260 – 274 0.8 x 0.64 0.512 3*

275 – 294 0.85 x 0.56 0.8 x 0.765 0.75 x 0.855 0.64125 2*

295 – 300 0.85 x 0.765 0.65025 4*

Hadi saragi
 Stage III f 3 ( s3 , x2 )  Rx2 ( f 2 ( s2 , x3 ))

S2,3,4 X3 = 1 X3 = 2 X3 = 3 X3 = 4 F3 (s3,x2) X2

235 – 274 - - 0.7 x 0.765 - 0.5355 1*

260 – 274 0.8 x 0.64 0.512 3*

ƒ3
275 – 294 0.85 x 0.56 0.8 x 0.765 0.75 x 0.855 0.64125 2*

295 – 300 0.85 x 0.765 0.65025 4*

 Stage IV f 4 ( s4 , x1 )  Rx1 ( f 3 ( s3 , x2 ))

S2,3,4 X2 = 1 X2 = 2 X2 = 3 X2 = 4 F4 (s4,x1) X1

335 – 359 0.6 x 0.535 - - - 0.321 1

360 – 364 - - 0.6 x 0.512 - 0.307 1

365 – 374 0.7 x 0.535 - - - 0.3745 2

Ƒ3 375 - 384 - 0,6 x 0.64125 - 0.38475 1

385 – 389 0.8 x 0.535 - - 0.428 3*

390 – 394 - - 0.7 x 0.512 - 0.3584 2

395 – 400 - - - 0.6 x 0.6505 0.3903 1

Hadi saragi
 3. Inventory Problem

Gyan Company forecast the demand for product during each of the next five months
will be
As follows :

Month 1. 3 Unit
Month 2 : 2 Unit
Month 3 : 2 Unit
Month 4 : 4 Unit
Month 5 : 5 Unit

During a month in which any units are produced, a setup cost of 5$ is incurred. The cost
of produce one unit is 2$ for each month. At the end of each month, a holding cost of
1 $ per unit On hand is incurred. Minimum and Maximum production capacity is 1 and
5 Unit each month. The size of the Company warehouse restricts the ending inventory
for each month to 4 units at most. The Company wants to determine a production
schedule that will meet all demands on time and Will minimize the sum of production
and holding costs during the five months. Assume that 0 unit Are on hand at the
beginning of the first month. Hadi saragi
 1.1 2.1 3.1 4.1 5.1

1.2 2.2 3.2 4.2 5.2

1.3 2.3 3.3 4.3 5.3 6.1

1.4 2.4 3.4 4.4 5.4

1.5 2.5 3.5 4.5 5.5

Stage 5 Stage 4 Stage 3 Stage 2 Stage 1

Hadi saragi
Stage I f1 ( s1 , in t )  C1 xn t

i x C1 xn-t = 5 + 2 (x) Total

Cost
ƒ1 0 5 15 15
-1 4 13 13

Hadi saragi
Stage II f 2 ( s2 , in t )  C2 xn t  f1 ( s1 , in t )

i x C1 xn-t = 1*(i) + 5 + 2 (x) Total

Cost
0 5 15 30
ƒ2 0 4 13 28*

1 5 16 29*

Hadi saragi
Stage III f 3 ( s3 , in t )  C3 xn t  f 2 ( s2 , in t )

i x C1 xn-t = 1*(i) + 5 + 2 (x) Total

Cost
0 2 15 30
1 3 13 28

2 4 16 29

3 5
ƒ3

Hadi saragi