Professional Documents
Culture Documents
Lecture 01
Lecture 01
Outline
Definitions
Algorithms
Problems
Course Objectives
Administrative stuff
Analysis of Algorithms
What is an Algorithm?
Algorithms are the ideas behind computer programs.
An algorithm is the thing that stays the same whether the
program is in C++ running on a Cray in New York or is in
BASIC running on a Macintosh in Katmandu!
To be interesting, an algorithm has to solve a general, specified
problem.
What is a problem?
Definition
A mapping/relation between a set of input instances
(domain) and an output set (range)
Problem Specification
Specify what a typical input instance is
Specify what the output should be in terms of the input
instance
Example: Sorting
Input: A sequence of N numbers a1an
Output: the permutation (reordering) of the input
sequence such that a1 a2 an .
Types of Problems
Search: find X in the input satisfying property Y
Structuring: Transform input X to satisfy property Y
Construction: Build X satisfying Y
Optimization: Find the best X satisfying property Y
Decision: Does X satisfy Y?
Adaptive: Maintain property Y over time.
Efficiency
What does efficiency mean?
Factor the number and see if there are any twos in the
factorization.
Example: TSP
Input: A sequence of N cities with the
distances dij between each pair of cities
Output: a permutation (ordering) of the
cities <c1, , cn> that minimizes the
expression
{j =1 to n-1} dj,j+1 + dn,1
Possible Algorithm:
Nearest neighbor
Not Correct!
A Correct Algorithm
We could try all possible orderings of the points, then
select the ordering which minimizes the total length:
d=
For each of the n! permutations, Pi of the n points,
if cost(Pi) < d then
d = cost(Pi)
Pmin = Pi
return Pmin
Outline
Definitions
Algorithms
Problems
Course Objectives
Administrative stuff
Analysis of Algorithms
Course Objectives
1. Learning classic algorithms
2. How to devise correct and efficient algorithms
for solving a given problem
3. How to express algorithms
4. How to validate/verify algorithms
5. How to analyze algorithms
6. How to prove (or at least indicate) no correct,
efficient algorithm exists for solving a given
problem
7. Writing clear algorithms and
Classic Algorithms
Expressing Algorithms
Implementations
Pseudo-code
English
My main concern here is not the specific
language used but the clarity of your
expression
Loop invariants
Analyzing algorithms
Outline
Definitions
Algorithms
Problems
Course Objectives
Administrative stuff
Analysis of Algorithms
Asymptotic analysis
Asymptotic notation
Input Size
Measuring Complexity
F(1) = 3
F(2) = 5
F(3) = 7
F(n) = 2n+1
Num Exec.
(n-1) + 1
n-1
?
?
?
n-1
Drawbacks
Quicksort
simplex method for linear programming
Simplifications
Ignore constants
4n2 - 3n log n + 17.5 n - 43 n + 75 becomes
n2 n log n + n - n + 1
Asymptotic Efficiency
n2 n log n + n - n + 1 becomes n2
Simplification of analysis
Much easier to analyze if we focus only on n2 rather
than worrying about 3.7 n2 or 3.9 n2
Asymptotic Analysis
We focus on the infinite set of large n
ignoring small values of n
Usually, an algorithm that is asymptotically
more efficient will be the best choice for all
but very small inputs.
0
infinity
Big Oh Notation
O(g(n)) =
{f(n) : there exists positive constants c and n0
such that nn0, 0 f(n) c g(n) }
What are the roles of the two constants?
n0:
c:
O(g(n))
(g(n))
(g(n))
1
100
( g (n))
1
25
n2
Example Function
f(n) = 3n - 100n + 6
2
Quick Questions
c
3n2 - 100n + 6 = O(n2)
3n2 - 100n + 6 = O(n3)
3n2 - 100n + 6 O(n)
3n2 - 100n + 6 = (n2)
3n2 - 100n + 6 (n3)
3n2 - 100n + 6 = (n)
3n2 - 100n + 6 = (n2)?
3n2 - 100n + 6 = (n3)?
3n2 - 100n + 6 = (n)?
n0
Little Oh Notation
o(g(n)) =
{f(n) : c >0 n0 > 0 such that n n0
0 f(n) < cg(n)}
Intuitively, limn f(n)/g(n) = 0
f(n) < c g(n)
10
20
110-5 sec
n2
0.025 sec
0.036 sec
n3
0.001 sec
0.008 sec
0.027 sec
0.064 sec
0.125 sec
0.216 sec
n5
0.1 sec
3.2 sec
24.3 sec
1.7 min
5.2 min
13.0 min
2n
0.001sec
1.0 sec
17.9 min
12.7 days
35.7 years
366 cent
3n
0.59sec
58 min
6.5 years
3855 cent
2108cent
1.31013cent
log2 n
310-6 sec
510-6 sec
610-6 sec
610-6 sec
30
40
50
410-5 sec
60
Example Problems
1. What does it mean if:
f(n) O(g(n)) and g(n) O(f(n)) ???
2. Is 2n+1 = O(2n) ?
Is 22n = O(2n) ?
3. Does f(n) = O(f(n)) ?
4. If f(n) = O(g(n)) and g(n) = O(h(n)),
can we say f(n) = O(h(n)) ?