Dynamic Programming

K S Deepika
Department of MBA
Syllabus

Unit 5: Dynamic Programming

Introduction to Dynamic programming, Belman’s principle

of optimality, Some practical examples; Equivalence of LP
and DP solutions (demonstrating using examples), Handling
stochasticity in Dynamic programming and Linear
programming
Dynamic Programming

Dynamic programming is both a mathematical

optimization method and a computer programming
method.

Developed by Richard Bellman 

Dynamic Programming is a technique in computer

programming that helps to efficiently solve a class of
problems that have overlapping sub problems and optimal
substructure property.
Properties of Dynamic Programming

Also called recursive optimization

Optimal substructure: Coin change problem

Overlapping sub problem: Fibonacci series

Dynamic Programming- Types

Deterministic Programming
- Shortest path
- Knapsack
- equipment replacement
- capital budgeting problem.

Probabilistic/ Stochastic Programming

- Inventory problem
Dynamic Programming- Applications

Shortest distance problems

Investment problems

Knapsack problem

Replacement problems

Cargo handling problems

LPP, …
Dynamic Programming

Where is Dynamic Programming Used?

Dynamic programming is used where we have problems,

which can be divided into similar sub-problems, so that
their results can be re-used. Mostly, these algorithms are
used for optimization. Before solving the in-hand sub-
problem, dynamic algorithm will try to examine the results
of the previously solved sub-problems.

No standard mathematical formulation is used.

Dynamic Programming

What Is the main idea behind Dynamic Programming?

To break the problems into sub-problems and save the

result for the future so that we will not have to compute
that same problem again.

Further optimization of sub-problems which optimizes the

overall solution is known as optimal substructure property.
Dynamic Programming
Dynamic Programming Languages

• JavaScript

• Python

• Ruby

• PHP

• Lua

• Perl
How does it work?

Memorization: Top down approach

Tabulation: Bottom up approach

Bell Man's principle of Optimality

The principle of optimality is the basic principle of dynamic

programming, which was developed by Richard Bellman:

An optimal path has the property that whatever the initial

conditions and control variables (choices) over some initial
period, the control (or decision variables) chosen over the
remaining period must be optimal for the remaining
problem, with the state resulting from the early decisions
taken to be the initial condition.
Bell Man's principle of Optimality
Dynamic Programming Terms
Stage: The DPP can be decomposed or divided into a
sequence of smaller sub problems called stages of the
original problem.

At each stage, there are number of decision alternatives, and

the best out of them is called stage decision.

State: The condition of the decision process at a stage

The variables which specify the condition of the decision

process at a particular stage are called stage variables.
Dynamic Programming Terms

A multi stage decision system, in which each decision and

state variable can take only finite number of values, can be
represented graphically by a decision tree.

Circles: nodes
Lines: decisions
Dynamic Programming Example 1

Employment smoothening problem

A firm has divided its marketing area into three zones. The
amount of sales depends on the number of salesmen in
each zone. The firm has been collecting the data regarding
sales and salesmen in each area over a number of past
years. The information is summarised in the table. For the
next year, the firm has only 9 salesmen and the problem is
to allocate the salesmen to three different zones so that
the total sales are maximum.
Dynamic Programming Example 1
Profit in thousand
rupees
No. of
salesmen Zone 1 Zone 2 Zone 3
0 30 35 40
1 45 45 50
2 60 50 60
3 70 65 70
4 80 70 80
5 90 80 95
6 100 90 100
7 105 100 110
8 100 100 110
9 90 100 110
Dynamic Programming Example 1

Let x1, x2 and x3 be the number of sales men allocated to

zone1, zone2 and zone 3 respectively.

f1(x1), f2(x2), f3(x3) be the profit for the respective zones.

Sub problems

Stage 1: Zone 1
Stage 2: Zone 1 + Zone 2
Stage 3: Zone 1+ Zone 2 + Zone 3
Dynamic Programming Example 1
Dynamic Programming Example 1
Dynamic Programming Example 1
Dynamic Programming Example 1
Dynamic Programming Example 2

A medicine manufacturing concern has nine medical

representatives working in 3 sales area. The profitability of
each representative in three sales area is as follows.

Determine the optimum allocation of medical

representatives in order to maximize the profits.
Dynamic Programming Example 2
Profit in thousand
rupees
No. of
representatives Area 1 Area 2 Area 3
0 32 37 44
1 47 47 56
2 62 54 62
3 72 66 72
4 81 74 84
5 92 84 97
6 100 95 104
7 107 100 112
8 102 102 112
9 92 102 112
Dynamic Programming Example 3
Resource allocation problem
The owner of chain of four grocery stores has purchased six crates of fresh
strawberries. The table gives the estimated profits at each store when it is allocated
various number of boxes.

Estimated profits
No. of boxes Store 1 Store 2 Store 3 Store 4
0 0 0 0 0
1 4 2 6 2
2 6 4 8 3
3 7 6 8 4
4 7 8 8 4
5 7 9 8 4
6 7 10 8 4
Equivalence of LP and DP solutions

Formulation of LP
Handling stochasticity in DP and LP

Unlike in a deterministic problem, where a decision’s

outcome is only determined by the decision itself and all
the parameters are known, in stochastic programming
there is uncertainty and the decision results in a
distribution of transformations.
THANK YOU

K S Deepika
Department of MBA
deepikaks@pes.edu