Bold Stroke

Advanced Algorithms CS 539/441

OR
In Search Of Efficient General Solutions

Joe Hoffert
Joseph.W.Hoffert@Boeing.com

January 13, 2003

Bold Stroke

Outline
General Techniques For Polynomial Time Solutions
Greedy Algorithms
Dynamic Programming
Linear Programming

Problems w/ unlikely Poly-time Solutions (NP Complete)

Next best solutions (e.g., Approximation Algorithms)
Lower Bound Techniques
On-Line/Dynamic Algorithms

January 13, 2003

Bold Stroke

Greedy Algorithms
Locally optimal choice leads to globally optimal solution.
Will not often yield optimal solution.
Framework
Prove Greedy Choice Property Holds:
first step g made by greedy algorithm is part of some optimal
solution (i.e., using g does no worse than an arbitrary optimal
solution)

Prove Optimal Substructure Property Holds

We need subproblem P of P left after g is chosen
P is optimally solved in S. That is, the solution to P contained within
S is optimal for P.

Examples: Earliest Deadline First (EDF) scheduling,

Huffman coding
January 13, 2003

Bold Stroke

Greedy Algorithm Example

Maximize # of jobs that meet deadlines
EDF scheduling: sort jobs by deadline O(n log n)
Job

Job

Job

Job

Job

Job Job
1

Schedule job
with shortest
deadline first

Job Job
1

must prove this choice

is at least as good as
any other

January 13, 2003

Job

Job

Job

Job

Job

Job

Subproblem P is to schedule remaining jobs 2 - 5

must prove this

subproblem is
independent of first
choice made

Bold Stroke

Dynamic Programming
Locally optimal choice doesnt lead to a globally optimal
solution but there is still optimal substructure
Framework
Try all possible first choices:
One of these must be the correct first choice

Prove Optimal Substructure Property Holds

We need subproblem P of P left after first choice
P is optimally solved in S. That is, the solution S to P contained
within S is optimal for P.

Use bottom up approach to efficiently compute optimal cost for all

subproblems (a.k.a. overlapping subproblems)

Examples: assembly line scheduling, longest common

subsequence
January 13, 2003

Bold Stroke

Dynamic Programming Example

Assembly line scheduling: find minimal cost through stations
assembly line 1

a1,1

e1

a1,2
t1,1

chassis
enters

a1,3

a1,n-1

t1,2

t2,1

a1,n
x1

t1,n-1

t2,2

completed
auto exits

t2,n-1

e2
assembly line 2

x2
a2,1

a2,2

a2,3

assembly line 1

chassis
enters

4
1

a2,n-1

a2,n

completed
auto exits

4
assembly line 2

January 13, 2003

2
8

Bold Stroke

Linear Programming
Problem defined by set of linear equations (equalities or
inequalities)
Poly-time but high exponent on time complexity (e.g., n8 to n10)
Proves poly-time solution, more efficient solutions might be found

Example: 2 dimensional constraints, minimum cost flow

x

Maximize x1 + x2 given constraints below

x2

x2

capacity = 1
cost = 3

y
Minimize cost for flow of 4 units from s to t
x
s
x2 >= 0

January 13, 2003

x1

x1

1 of 1
cost = 3

Bold Stroke

NP-Complete Problems
NP complete problems are:
Complexity class Non-deterministic Polynomial (NP), i.e., solution
to NP problem can be verified in polynomial time
NP-hard, i.e., reducible from all other NP problems in poly-time

Unlikely to have poly-time solutions, not (yet) proven

Example: MP-Scheduling problem
Job
1

Job
2

Job
3

Job Job
n-1 n

Is there a schedule in
which all jobs are processed
by specified time?
January 13, 2003

Processor Processor
1
2

Job
Job
3
2
Processor Processor
1
2

Processor
m

Job
Job Job
n
n-1 1
Processor Processor
m-1
m

Bold Stroke

Next Best Solutions

What to do when problem shown to be NP-complete:
Is problem a special case? For instance, vertex-cover is NPcomplete but if graph is a tree there exists a poly time algorithm for
this case
Input size may be small enough not to be a problem
Use a heuristic and hope it gets something good enough
Use approximation algorithms (can be proven to be good enough
or not good enough)
LP relaxation (IP is NP-complete, LP may give a good enough
answer)
0-1 Knapsack relaxation to fractional knapsack
Non-preempted schedule relaxed to have preemption

January 13, 2003

Bold Stroke

Approximation Algorithms
Poly-time algorithms with provable bounds relative to
optimal solution
If approximation bound is some constant n then the algorithm is
said to be an n-approximation

Example: 2-approximation 0-1 Knapsack problem

Item 1

Item 2

Item 3

Item 4
Sort items by value per
unit quantity (i.e., total
value/amount) and start
filling up the knapsack

Item 3
Item 1
Item 5
Item 2

January 13, 2003

Item 4

Item 5

Take the larger of the first item

that wont completely fit and the
sum of all the previous items in
the knapsack. This must be at
least of optimal solution which
is a 2 approximation.

Bold Stroke

Lower Bound Techniques

Determine minimum number of operations needed for any
solution
Use decision-tree lower bound algorithm when applicable
All possibilities are enumerated as leaves in a decision tree
Traverse the depth of the tree
Example: lower bound for median among two sorted arrays

Adversary strategy
Devise a strategy that makes it hard for any algorithm to find a
solution (i.e., maximize the number of steps needed for any solution)
Example: merging two sorted arrays

Provides guidance for how good an algorithm can be

January 13, 2003

Bold Stroke

Adversary Strategy Example

Example: merging two sorted arrays
How many comparisons must be made before any algorithm has
the arrays sorted?
a1 a2 a3 a4

b1 b2 b3 b4

an-1 an

Adversary strategy
a1 b1 a2 b2 a3 b3
If < 2n 1 comparisons are made
more than one possible answer
left (i.e., adversary can switch the
elements that have not been
compared)
ai-1 bi-1 ai

January 13, 2003

i-1

bi-1 ai

bi ai+1 bi+1

a 4 b4

bn-1 bn

an-1 bn-1 an bn

bi ai+1 bi+1

i-1

bi-1 bi

ai ai+1 bi+1

Bold Stroke

On-Line/Dynamic Algorithms
Algorithms that dont have knowledge of future
requests/input
Compare to off-line algorithms that have complete knowledge of
input
Provable bound compared to off-line solution

Examples: Ski rental vs. purchase, scheduling jobs with

deadlines using EDF,
B = cost to buy
R = cost to rent
Can have 2-competitive
algorithm, i.e., provable
2B bound on cost
January 13, 2003

Bold Stroke

Conclusion
Techniques exist for:
Provable poly-time algorithms
Determining unlikely poly-time algorithms
Next best solutions with provable bounds

When to use what technique

Mostly developed through practice and intuition

What does it matter?

Provable solutions

January 13, 2003