3-Complexity Analysis

UNIT - 3
Complexity Analysis
Santanu Halder Design and Analysis of Algorithms

1
Posteriori Analysis
Priori Analysis

2
Asymptotic notation
Big-O notation: f(n) = O(g(n) iff f(n) ≤ c*g(n)
for all n ≥ n0, where c and n0 are some
positive constants.
The function g(n) indicates the upper

bound of f(n) and so g(n) should be as
small as possible for which the statement
f(n) = O(g(n)) is true.

3
Big (O) Notation

4
Big Oh (O) Notation
Example: f (n)  2n 3  3n 2  n  10
f (n)  2n 3  3n 2  n  10  5n 3 n  2
f ( n)  O ( n 3 )

5
Asymptotic notation
Omega notation: f(n) = Ω(g(n) iff f(n) ≥
c*g(n) for all n ≥ n0, where c and n0 are
some positive constants.
The function g(n) is only a lower bound of

f(n) and so g(n) should be as large as
possible for which the statement f(n) =
Ω(g(n)) is true.

6
Omega (Ω) Notation

7
Omega (Ω) Notation
Example: f (n)  2n 3  3n 2  n  10
f (n)  2n 3  3n 2  n  10  2n 3 n  1
f ( n)  ( n 3 )

8
Asymptotic notation
Theta notation: f(n) = Ө(g(n) iff c1*g(n) ≤ f(n)
≤ c2*g(n) for all n ≥ n0, where c1,c2 and n0
are some positive constants.
The function g(n) is a tight bound of f(n)

because g(n) is both upper and lower
bound on f(n).

9
Theta (θ) Notation

10
Theta (θ) Notation
Example: f (n)  2n 3  3n 2  n  10
2n 3  2n 3  3n 2  n  10  5n 3 n  2
f ( n)   ( n 3 )

11
Little oh (o) Notation
Little oh (o) notation: f(n) = o(g(n) iff
f ( n)
lim 0
n  g ( n)

12
Little oh (o) Notation
Example:
f (n)  2n 3  3n 2  n  10
2n 3  3n 2  n  10  2 3 1 10 
lim 4
 lim  2  3  4   0
n  n n  n
 n n n 
f ( n)  o( n 4 )

13
Little omega (ω) Notation
Little omega (ω) notation: f(n) = ω(g(n) iff
g ( n)
lim 0
n  f ( n)

14
Little omega (ω) Notation
Example:
f (n)  2n 3  3n 2
n  1 1 
lim 3  lim 2    0
n  2 n  3n 2 n  2n 3n 

f ( n)   ( n)

15
Theorem 1
f ( n )  a m n m  a m 1 n m 1     a1 n  a 0
m
f (n)  
i0
ai n i
m
f (n)  n m

i0
ai n i  m
m
1
f ( n )  n m  ai
i0 n m i

16
Theorem 1
1
If n>1 then n m i  1
m
f (n)  n m
a
i0
i
f (n)  O (n ) m

17
Properties of Big Oh
Theorem 1: If f1(n) = O(g1(n)) and f2(n) = O(g2(n)),

then f1(n) + f2(n) = O(max(g1(n), g2(n))).
f1(n) ≤ c1*g1(n) for n ≥ n1
f2(n) ≤ c2*g2(n) for n ≥ n2
Let n0 = max(n1,n2) and c0 = 2*max(c1,c2)

18
f 1( n )  f 2 ( n )  c1  g 1( n )  c 2  g 2 ( n ) for n≥n0
c 0  ( g 1( n )  g 2 ( n ))
f 1( n )  f 2 ( n ) 
2
f 1( n )  f 2 ( n )  c 0  max( g 1( n ), g 2 ( n ))
f 1( n )  f 2 ( n )  O (max( g1( n ), g 2 ( n )))

19
Theorem 2: If f1(n) = O(g1(n)) and f2(n) = O(g2(n)),

then f1(n) × f2(n) = O(g1(n) × g2(n))
f1(n) ≤ c1*g1(n) for n ≥ n1
f2(n) ≤ c2*g2(n) for n ≥ n2
Let n0 = max(n1,n2) and c0 = c1 × c2

20
f 1( n )  f 2 ( n )  ( c1  g 1( n ))  ( c 2  g 2 ( n )) for n≥n0
f 1( n )  f 2 ( n )  c 0  [ g1( n )  g 2 ( n )]
f 1( n )  f 2 ( n )  O ( g 1( n )  g 2 ( n ))

21
Examples
Prove that log n! O(n log n)
log n!  log( 1  2    n )
log n!  log 1  log 2    log n
log n!  log n  log n    log n
log n!  n log n
log n!  O ( n log n )

22
Recurrence Relations
T ( n )  2  T ( n  1)  1
n
T (n)  2  T    n
2
T (n)  2  T  n  log n
n
T ( n )  3T    n 2
2
n
T ( n )  3T    cn 2
4

23
Recurrence Relations
Way of solution of a recurrence relation
 Substitution Method
 Master’s Theorem
 Recursion Tree Method

24
Substitution Method
T ( n )  2  T ( n  1)  1
T (n)  2 2  T (n  2)  2  1
T ( n )  2 3  T ( n  3)  2 2  2  1

T ( n )  2 k  T ( n  k )  2 k 1    2 2  2  1
Let n-k=1
T ( n )  2 n 1  2 n  2    2 2  2  1

25
Substitution Method
2n  1
T (n) 
2 1
T (n)  2 n  1
T (n)  2 n For all n≥1
T (n)  O (2 n )

26
Substitution Method
n
T (n)  2  T    n
2
 n 
T (n)  2 2  T  2   2  n
2 
 n 
T (n)  2  T  k   k  n
k
2 
Let, 2k=n
T ( n )  n  T 1  n  log n  2 n log n For all n≥2
T ( n )  O ( n log n )

27
Substitution Method
T (n)  2  T  n  log n
Let, m=log2n => n=2m
 m

T ( 2 m )  2  T  2 2 m

 
Again, consider S(m)=T(2m)
m
S (m )  2  S    m
2
S ( m )  O ( m log m )  T ( n )  O (log n log log n )

28
Substitution Method
T (n)  2  T  n  1
 m

T ( 2 m )  2  T  2 2  1

 
m
S (m )  2  S    1
2
S ( m )  O ( m )  T ( n )  O (log n )

29
Substitution Method
m
S (m )  2  S    1
2
m
S (m )  2 2  S  2   2  1
2 
m
S (m )  2  S  3   2 2  2  1
3
2 
m
S (m )  2 k  S  k     2 2  2  1
2 
Let, 2k=m
S ( m )  2 k    2 2  2  1  S ( m )  O ( 2 k )  O ( m )

30
Master’s Theorem
Master’s Theorem solves recurrence relation of
the form:
n
 
T ( n )  aT     n k log p n
 b    
f (n)
Where, a ≥ 1, b > 1, k ≥ 0 and p is a real number.

31
Master’s Theorem
Case 1:
If a>bk then 
T ( n )   n log b a 
Case 2:
If a=bk and

i) If p < -1 then T ( n )   n log b a 
ii) If p = -1 then T ( n )   n log b a
. log 2 n 
iii) If p > -1 then T ( n )   n log b a
. log p 1 n 

32
Master’s Theorem
Case 3:
If a < bk and
 
i) If p < 0 then T ( n )  O n k
ii) If p ≥ 0 then T ( n )   n . log n 
k p

33
Master’s Theorem
Problem 1:
n
T ( n )  3T    n 2
n

T ( n )  aT     n k log p n 
2 b

34
Master’s Theorem
Problem 1:
n
T ( n )  3T    n 2
n

T ( n )  aT     n k log p n 
2 b
a=3 As, a<bk and p ≥ 0,
b=2 We follow, Case 3-(ii).
k=2
p=0
 
T ( n )   n k . log p n
Or, T ( n )   n . log n 
2 0
Or, T ( n )   n 
2

35
Master’s Theorem
Problem 2:
n
T ( n )  2T    n log n
n

T ( n )  aT     n k log p n 
2 b

36
Master’s Theorem
Problem 2:
n
T ( n )  2T    n log n
n
T ( n )  aT     n k log p n  
2 b
a=2 As, a = bk and p > -1,
b=2 We follow, Case 2-(iii).
k=1
p=1

T ( n )   n log b a . log p 1 n 
Or, T ( n )   n log 2 2
. log 11
n
Or, T ( n )   n log n  2

37
Master’s Theorem
Problem 3:
n
T ( n )  2T    n 0 .51
n

T ( n )  aT     n k log p n 
4 b

38
Master’s Theorem
Problem 3:
n
T ( n )  2T    n 0 .51
n

T ( n )  aT     n k log p n 
4 b
a=2 As, a < bk (40.51=2.03) and p ≥ 0,
b=4 We follow, Case 3-(ii).
k=0.51
p=0
 
T ( n )   n k . log p n
Or, T ( n )   n . log n 
0 .51 0
Or, T ( n )   n 
0 .51

39
Master’s Theorem
Problem 4:
T (n) 
n
2T    log n
n

T ( n )  aT     n k log p n 
2 b

40
Master’s Theorem
Problem 4:
T (n) 
n
2T    log n
n

T ( n )  aT     n k log p n 
2 b
a=√2 As, a > bk ,
b=2 We follow, Case 1.
k=0
p=1

T ( n )   n log b a 

Or, T ( n )   n log 2 2

 
Or, T ( n )   n 1 2 Or, T ( n )    n
41
Master’s Theorem
Problem 5:
n n
T ( n )  3T   
n

T ( n )  aT     n k log p n 
3 2 b
Rewrite the recurrence relation as:
n
T ( n )  3T    n
3
Because, in the general form of Master’s Theorem,
we have ϴ for function f(n) which hides constants
in it.

42
Master’s Theorem
Problem 5:
n
T ( n )  3T    n
n
T ( n )  aT     n k log p n  
3 b
a=3 As, a = bk and p ≥ -1,
k=1
p=0

T ( n )   n log b a . log p 1 n 
Or, T ( n )   n log 3 3
. log 0 1 n
Or, T ( n )   n log n 

43
Master’s Theorem
Problem 6:
T (n)  T  n  1 n

T ( n )  aT     n k log p n 
b
 m

T ( 2 m )  T  2 2  1

 
m
S (m )  S    1
2

44
Master’s Theorem
Problem 6:
m
S (m )  S    1
n

T ( n )  aT     n k log p n 
2 b
a=1 As, a = bk and p ≥ -1,
k=0
p=0

S ( m )   m log b a . log p 1 m 
Or, S ( m )   m log 2 1
. log 0 1
m
Or, S ( m )   m . log m  Or,
0 0 1
S ( m )   log m 

45
Master’s Theorem
Problem 6:
m
S (m )  S    1
n

T ( n )  aT     n k log p n 
2 b
a=1 As, a = bk and p ≥ -1,
k=0
p=0 S ( m )   log m 
Or, T ( n )   log log 2 n 

46
Master’s Theorem
Problem 7:
n
T ( n )  8T    n 2 log n
n

T ( n )  aT     n k log p n 
4 b
Above recurrence relation does not
correspond to the general form of Master’s
Theorem.
Therefore, it can not be solved using Master’s
Theorem.

47
Master’s Theorem
More Inadmissible Equations:
n a is not constant; number of sub-
T (n)  2 T    n n
n
2 problems should be fixed.
n a is less than 1; number of sub-

T ( n )  0 .5T    n problems can not be less than 1.
2
n Combination time is not
T ( n )  64 T    n 2 log n
8 positive
n
T ( n )  T    n ( 2  cos n ) Regularity violation
2

48
Recursion Tree
Step 1: Draw a recursion tree based on the given
recurrence relation.
Step 2: Determine cost of each level.
Step 3: Determine total number of levels in
recursion tree.
Step 4: Determine number of nodes in the last
level.
Step 5: Determine cost of last level.
Step 6: Add cost of all the level of recursion tree
and simplify the expression in terms of
Asymptotic notation.

49
Recursion Tree
Problem 1:
n
T ( n )  2T    n
2

50
Recursion Tree
Problem 1: n
T ( n )  2T    n
2
Step 1:

51
Recursion Tree
Problem 1: n
T ( n )  2T    n
2 Cost
Step 2: n

52
Recursion Tree
Problem 1: n
T ( n )  2T    n
2
recursion tree.
Size of Sub-Problem at Level 0 = n/20
……………………………………………..
……………………………………………..
Size of Sub-Problem at Level k = n/2k

53
Recursion Tree
Problem 1: n
T ( n )  2T    n
2
recursion tree.
If k becomes the last level, then the size of Sub-
Problem at level k is 1.
n/2k = 1
Or, 2k = n
Or, k = log2n
Therefore, total number of levels in the recursion
tree is log2n +1.

54
Recursion Tree
Problem 1: n
T ( n )  2T    n
2
level.
Level 0 has 20 nodes Level log2n has 2log2n nodes
Level 1 has 21 nodes Or, Level log2n has n nodes
Level 2 has 22 nodes
…………………………
…………………………
Level k has 2k nodes

55
Recursion Tree
Problem 1: n
T ( n )  2T    n
2
Cost of last level = n × T(1) = ϴ(n).

56
Recursion Tree
Problem 1: n
T ( n )  2T    n
2
T ( n )  n  log 2 n   ( n )
  ( n log n )

57
Recursion Tree
Problem 2: n
T ( n )  3T    cn 2
4
Step 1:

58
Recursion Tree
Problem 2: n
T ( n )  3T    cn 2
4
Step 2: Cost
cn2
(3/42)cn2
(9/162)cn2

59
Recursion Tree
Problem 2: n
T ( n )  3T    cn 2
4
recursion tree.
……………………………………………..
……………………………………………..
Size of Sub-Problem at Level k = n/4k

60
Recursion Tree
Problem 2: n
T ( n )  3T    cn 2
4
recursion tree.
If k becomes the last level, then the size of Sub-Problem at
level k is 1.
n/4k = 1
Or, 4k = n
Or, k = log4n
Therefore, total number of levels in the recursion tree is
log4n +1.

61
Recursion Tree
Problem 2: n
T ( n )  3T    cn 2
4
level.
Level 0 has 30 nodes Level log4n has 3log4n nodes
Level 1 has 31 nodes Or, Level log4n has nlog43
Level 2 has 32 nodes nodes.
…………………………
…………………………
Level k has 3k nodes

62
Recursion Tree
Problem 2: n
T ( n )  3T    cn 2
4
Cost of last level = nlog43 × T(1) = ϴ(nlog43).

63
Recursion Tree
Problem 2: n
T ( n )  3T    cn 2
4

64
Recursion Tree
Problem 2: n
T ( n )  3T    cn 2
4
 3  3 
2

T ( n )  cn 1 
2
    ....    ( n log 4 3 )
 16  16  
log 4 n
 3 
1  
 cn 2  16 
3

  n log 4 3 
1
16
log 4 n

16 2 16 2  3 
13
cn 
13
cn   
  n log 4 3 
 16 
 
 O n2

65
How can we improve the complexity?
Write an algorithm to calculate the sum of first n

natural numbers.
Approach 1
1. Algorithm Summation(n)
2. {
3. Sum = 1;
4. for(Counter = 1;Counter ≤ n;Counter++)
5. Sum = Sum + Counter;
6. return(Sum);
7. }//End of Algo

66
Approach 2
1. Algorithm Summation(n)
2. {
3. Result=(n*(n+1))/2;
4. }//End of Algo
Complexity O (1)

67
Write an algorithm to find the value of xn where n is

integral power of 2.
Approach 1
1. Algorithm Power(x, n)
2. {
3. Result = 1;
4. for(Counter = 1;Counter≤n;Counter++)
5. Result = Result*x;
6. return(Result);
7. }//End of Algo

68
Approach 2
1. Algorithm Power(x, n)
2. {
3. if (n=1) then return x;
4. else return Power(x*x,n/2);
5. }//End of Algo
Complexity
n
T ( n )  T    c  T ( n )  O (log n )
2

69
End of unit 3

70

3-Complexity Analysis

Uploaded by

Copyright:

Available Formats

You might also like

3-Complexity Analysis

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

3-Complexity Analysis

Uploaded by

Copyright:

Available Formats

UNIT - 3

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

The function g(n) indicates the upper

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

The function g(n) is only a lower bound of

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

The function g(n) is a tight bound of f(n)

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Little oh (o) notation: f(n) = o(g(n) iff

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Little omega (ω) notation: f(n) = ω(g(n) iff

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Theorem 1: If f1(n) = O(g1(n)) and f2(n) = O(g2(n)),

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Theorem 2: If f1(n) = O(g1(n)) and f2(n) = O(g2(n)),

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

 Recursion Tree Method

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Where, a ≥ 1, b > 1, k ≥ 0 and p is a real number.

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

Santanu Halder Design and Analysis of Algorithms

2 problems should be fixed.