Professional Documents
Culture Documents
Unit1 1IntroAsympAnalysis
Unit1 1IntroAsympAnalysis
Unit1 1IntroAsympAnalysis
SE 208
Introduction
Divyashikha Sethia
Divyashikha@dtu.ac.in
DTU
2
1. Concept of Algorithm efficiency
Analysis of Algorithms..
• An algorithm is a finite set of precise instructions
for performing a computation or for solving a
problem.
• What is the goal of analysis of algorithms?
– To compare algorithms mainly in terms of running
time but also in terms of other factors (e.g., memory
requirements, programmer's effort etc.)
• What do we mean by running time analysis?
– Determine how running time increases as the size
of the problem increases.
3
1. Concept of Algorithm efficiency
Input Size
– polynomial degree
– # of elements in a matrix
4
2. Run Time analysis of algorithms
Types of Analysis
• Worst case
– Provides an upper bound on running time
– An absolute guarantee that the algorithm would not run longer,
no matter what the inputs are
• Best case
– Provides a lower bound on running time
– Input is the one for which the algorithm runs the fastest
7
2. Run Time analysis of algorithms
Example
• Associate a "cost" with each statement.
• Find the "total cost“ by finding the total number of times
each statement is executed.
Algorithm 1 Algorithm 2
Cost Cost
arr[0] = 0; c1 for(i=0; i<N; i++) c2
arr[1] = 0; c1 arr[i] = 0; c1
arr[2] = 0; c1
... ...
arr[N-1] = 0; c1
----------- -------------
c1+c1+...+c1 = c1 x N (N+1) x c2 + N x c1 =
(c2 + c1) x N + c2
8
2. Run Time analysis of algorithms
Another Example
• Algorithm 3 Cost
sum = 0; c1
for(i=0; i<N; i++) c2
for(j=0; j<N; j++) c2
sum += arr[i][j]; c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N2
9
3. Asymptotic Notations
Asymptotic Analysis
• To compare two algorithms with running
times f(n) and g(n), we need a rough
measure that characterizes how fast
each function grows.
• Hint: use rate of growth
• Compare functions in the limit, that is,
asymptotically!
(i.e., for large values of n)
10
3. Asymptotic Notations
Rate of Growth
• Consider the example of buying elephants and
goldfish:
Cost: cost_of_elephants + cost_of_goldfish
Cost ~ cost_of_elephants (approximation)
• The low order terms in a function are relatively
insignificant for large n
n4 + 100n2 + 10n + 50 ~ n4
11
3. Asymptotic Notations
Different notations
12
3. Asymptotic Notations
Big-O Notation
13
3. Asymptotic Notations
Visualizing Orders of Growth
• On a graph, as
you go to the
right, a faster
growing
Value of function →
function fA(n)=30n+8
eventually
becomes
larger... fB(n)=n2+1
Increasing n →
14
3. Asymptotic Notations
More Examples …
15
Back to Our Example
Algorithm 1 Algorithm 2
Cost Cost
arr[0] = 0; c1 for(i=0; i<N; i++) c2
arr[1] = 0; c1 arr[i] = 0; c1
arr[2] = 0; c1
...
arr[N-1] = 0; c1
----------- -------------
c1+c1+...+c1 = c1 x N (N+1) x c2 + N x c1 =
(c2 + c1) x N + c2
16
Example (cont’d)
Algorithm 3 Cost
sum = 0; c1
for(i=0; i<N; i++) c2
for(j=0; j<N; j++) c2
sum += arr[i][j]; c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N2 = O(N2)
17
Asymptotic notations
• O-notation
18
Big-O Visualization
19
Examples
– 1000n2+1000n = O(n2):
20
More Examples
• Show that 30n+8 is O(n).
21
Big-O example, graphically
• Note 30n+8 isn’t
less than n
anywhere (n>0).
cn =
• It isn’t even
31n 30n+8
Value of function →
less than 31n
everywhere.
• But it is less than 30n+8
31n everywhere to
the right of n=8.
n
O(n)
n>n0=8 →
Increasing n →
22
No Uniqueness
• There is no unique set of values for n0 and c in proving the
asymptotic bounds
for all n ≥ 5
24
Examples
– 5n2 = (n)
c, n0 such that: 0 cn 5n2 cn 5n2 c = 1 and n0 = 1
– 100n + 5 ≠ (n2)
c, n0 such that: 0 cn2 100n + 5
100n + 5 100n + 5n ( n 1) = 105n
cn2 105n n(cn – 105) 0
Since n is positive cn – 105 0 n 105/c
contradiction: n cannot be smaller than a constant
– n = (2n), n3 = (n2), n = (logn)
25
Asymptotic notations (cont.)
• -notation
26
Examples
– n2/2 –n/2 = (n2)
• ½ n2 - ½ n ≤ ½ n2 n ≥ 0 c2= ½
• ½ n2 - ½ n ≥ ½ n2 - ½ n * ½ n ( n ≥ 2 ) = ¼ n2
c1= ¼
– n ≠ (n2): c1 n2 ≤ n ≤ c2 n2
27
Examples
– 6n3 ≠ (n2): c1 n2 ≤ 6n3 ≤ c2 n2
c2 ≥ n/logn, n≥ n0 – impossible
28
Relations Between Different Sets
• Subset relations between order-of-growth sets.
R→R
O( f ) ( f )
•f
( f )
29
Common orders of magnitude
30
Common orders of magnitude
31
Logarithms and properties
• In algorithm analysis we often use the notation “log n”
without specifying the base
log b x = log a x
log a b
32
More Examples
• For each of the following pairs of functions, either f(n) is
O(g(n)), f(n) is Ω(g(n)), or f(n) = Θ(g(n)). Determine
which relationship is correct.
– f(n) = log n2; g(n) = log n + 5 f(n) = (g(n))
– f(n) = n; g(n) = log n2 f(n) = (g(n))
– f(n) = log log n; g(n) = log n f(n) = O(g(n))
– f(n) = n; g(n) = log2 n f(n) = (g(n))
– f(n) = n log n + n; g(n) = log n f(n) = (g(n))
– f(n) = 10; g(n) = log 10 f(n) = (g(n))
– f(n) = 2n; g(n) = 10n2 f(n) = (g(n))
– f(n) = 2n; g(n) = 3n f(n) = O(g(n))
33
Properties
• Theorem:
f(n) = (g(n)) f = O(g(n)) and f = (g(n))
• Transitivity:
– f(n) = (g(n)) and g(n) = (h(n)) f(n) = (h(n))
– Same for O and
• Reflexivity:
– f(n) = (f(n))
– Same for O and
• Symmetry:
– f(n) = (g(n)) if and only if g(n) = (f(n))
• Transpose symmetry:
– f(n) = O(g(n)) if and only if g(n) = (f(n))
34
• References:
• Analysis of Algorithms
CS 477/677 Asymptotic Analysis, Instructor: George Bebis
• Chapter 3 Cormen: Growth of Functions
35