UNIT - 3

Complexity Analysis

Chandan Sinha Roy Analysis of Algorithms

1
 Posterior Analysis
Priori Analysis

Chandan Sinha Roy Analysis of Algorithms

2
 Asymptotic notation
Big-O notation: f(n) = O(g(n) iff f(n) ≤ c*g(n)
for all n ≥ n0, where c and n0 are some
positive constants.

The function g(n) indicates the upper bound

of f(n) and so g(n) should be as small as
possible for which the statement f(n) =
O(g(n)) is true.

Chandan Sinha Roy Analysis of Algorithms

3
 Big (O) Notation

Chandan Sinha Roy Analysis of Algorithms

4
 Big Oh (O) Notation

Example: f (n)  2n 3  3n 2  n  10
f (n)  2n 3  3n 2  n  10  5n 3 n  2, C  5
f ( n)  O ( n 3 )

T(n) = 15n3 + 7n2 + 35 <=57n3 , f(n) = 15n3 + 7n2 + 35 and g(n)=n3, C=57
for n>=1
•T(n) = O(n3)
•Exercise : 10n3 + 5n + 7 є O(n4)

Chandan Sinha Roy Analysis of Algorithms

5
 Asymptotic notation
Omega notation: f(n) = Ω(g(n) iff f(n) ≥
c*g(n) for all n ≥ n0, where c and n0 are
some positive constants.

The function g(n) is only a lower bound of

f(n) and so g(n) should be as large as
possible for which the statement f(n) =
Ω(g(n)) is true.

Chandan Sinha Roy Analysis of Algorithms

6
 Omega (Ω) Notation

Chandan Sinha Roy Analysis of Algorithms

7
 Omega (Ω) Notation

Example: f (n)  2n 3  3n 2  n  10
f (n)  2n 3  3n 2  n  10  2n 3 n  1
f ( n)  ( n 3 )

T(n)= 2n3 + 37 >=2n3 , f(n)= 2n3 + 37 and g(n)= n3. C=2 for n>=1
•T(n) = Ω(n3)

Chandan Sinha Roy Analysis of Algorithms

8
 Asymptotic notation
Theta notation: f(n) = Ө(g(n) iff c1*g(n) ≤ f(n)
≤ c2*g(n) for all n ≥ n0, where c1,c2 and n0 are
some positive constants.
The function g(n) is a tight bound of f(n)
because g(n) is both upper and lower
bound on f(n).

Chandan Sinha Roy Analysis of Algorithms

9
 Theta (θ) Notation

Chandan Sinha Roy Analysis of Algorithms

10
 Theta (θ) Notation

Example: f (n)  2n 3  3n 2  n  10
2n 3  2n 3  3n 2  n  10  5n 3 n  2
f ( n)   ( n 3 )

f (n)  5n log n  10n   (n log n)

k
f   ( g ), so, f   ( g )
f (n)   a. n   (n )
i i k

i 0
but , ( g )  f
 ( g )  O( g ) ( g )

Chandan Sinha Roy Analysis of Algorithms

11
 Little oh (o) Notation

Little oh (o) notation: not asymptotically tight

o(g(n)) = { f(n) : for any (not some) positive constant c>0, there
exist a constant n0>0 such that 0<= f(n) <C.g(n) for all n>=n0

In case of Big-O, bound holds for some value of constant c >0

Incase of Small-o, bound holds for all values of constant c >0

So, 2n=o(n2) where n>=2 and any value of C,

but 2n2 ≠ o(n2) may satisfy rules, but not for all values of C.

f ( n)
lim 0
n  g ( n )

Chandan Sinha Roy Analysis of Algorithms

12
 Little oh (o) Notation

Example:
f (n)  2n 3  3n 2  n  10

2n 3  3n 2  n  10  2 3 1 10 
lim 4
 lim  2  3  4   0
n  n n  n
 n n n 

f ( n)  o( n 4 )

Chandan Sinha Roy Analysis of Algorithms

13
 Little omega (ω) Notation
Little omega (ω) notation: not asymptotically tight
ω(g(n)) = { f(n) : for any (not some) positive constant c>0, there
exist a constant n0>0 such that 0<=C.g(n) < f(n) for all n>=n0

So, n2 / 2 = ω(n), but n2 / 2 ≠ ω(n2)

Little omega (ω) notation: f(n) = ω(g(n) iff

g ( n)
lim 0
n  f ( n)

Chandan Sinha Roy Analysis of Algorithms

14
 Little omega (ω) Notation

Example:
f (n)  2n 3  3n 2

n  1 1 
lim 3  lim 2    0
n  2n  3n 2 n  2n 3n 


f ( n)   ( n)

Chandan Sinha Roy Analysis of Algorithms

15
 Theorem 1

f (n)  am n m  am 1n m 1    a1n  a0

m
f (n)   ai n i
i 0
m
f ( n)  n m

i 0
ai n i  m
m
1
f (n)  n m  ai
i 0 n m i

Chandan Sinha Roy

Analysis of Algorithms
16
 Theorem 1
1
If n>1 then n m i  1
m
f ( n)  n m
a
i 0
i

m
f ( n)  O ( n )

Chandan Sinha Roy Analysis of Algorithms

17
 Properties of Big Oh

Theorem 1: If f1(n) = O(g1(n)) and f2(n) = O(g2(n)),

then f1(n) + f2(n) = O(max(g1(n), g2(n))).
f1(n) ≤ c1*g1(n) for n ≥ n1
f2(n) ≤ c2*g2(n) for n ≥ n2
Let n0 = max(n1,n2) and c0 = 2*max(c1,c2)

Chandan Sinha Roy Analysis of Algorithms

18
 Properties of Big Oh

f 1(n)  f 2(n)  c1 g1(n)  c 2  g 2(n) for n≥n0

c0  ( g1(n)  g 2(n))
f 1(n)  f 2(n) 
2
f 1(n)  f 2( n)  c0  max( g1(n), g 2(n))
f 1(n)  f 2(n)  O(max( g1(n), g 2(n)))

Chandan Sinha Roy Analysis of Algorithms

19
 Properties of Big Oh

Theorem 2: If f1(n) = O(g1(n)) and f2(n) = O(g2(n)),

then f1(n) × f2(n) = O(g1(n) × g2(n))
f1(n) ≤ c1*g1(n) for n ≥ n1
f2(n) ≤ c2*g2(n) for n ≥ n2
Let n0 = max(n1,n2) and c0 = c1 × c2

Chandan Sinha Roy Analysis of Algorithms

20
 Properties of Big Oh

f 1(n)  f 2(n)  (c1 g1(n))  (c 2  g 2(n)) for n≥n0

f 1(n)  f 2(n)  c0  [ g1(n)  g 2(n)]
f 1(n)  f 2(n)  O ( g1(n)  g 2(n))

Chandan Sinha Roy Analysis of Algorithms

21
 More Examples
Prove that log n! O (n log n)
log n!  log(1 2    n)
log n!  log 1  log 2    log n
log n! log n  log n    log n
log n! n log n
log n! O (n log n)

Chandan Sinha Roy Analysis of Algorithms

22
 More Examples
T (n)  2  T (n  1)  1
T (n)  2 2  T ( n  2)  2  1
T (n)  23  T (n  3)  2 2  2  1

T (n)  2 k  T (n  k )  2 k 1    2 2  2  1
Let n-k=1
T (n)  2 n 1  2 n  2    2 2  2  1

Chandan Sinha Roy Analysis of Algorithms

23
 More Examples
2n  1
T ( n) 
2 1
T ( n)  2 n  1
T ( n)  2 n For all n≥1
T ( n)  O ( 2 n )

Chandan Sinha Roy Analysis of Algorithms

24
 More Examples
n
T ( n)  2  T    n
2
 n 
T ( n)  2 2  T  2   2  n
2 
k  n 
T ( n)  2  T  k   k  n
2 
Let, 2k=n
T (n)  n  T 1  n  log n  2n log n For all n≥2
T (n)  O(n log n)

Chandan Sinha Roy Analysis of Algorithms

25
 More Examples
T ( n)  2  T  n  log n
Let, m=log2n => n=2m
 m

T (2 )  2  T  2   m
m 2

 
Again, consider S(m)=T(2m)
m
S ( m)  2  S    m
2
S ( m)  O(m log m)  T (n)  O(log n log log n)

Chandan Sinha Roy Analysis of Algorithms

26
 More Examples
T ( n)  2  T  n  1
Let, m=log2n => n=2m
 m

T (2 )  2  T  2   1
m 2

 
Again, consider S(m)=T(2m)
m
S ( m)  2  S    1
2
S ( m)  O(m)  T (n)  O(log n)

Chandan Sinha Roy Analysis of Algorithms

27
 How can we improve the complexity?

Write an algorithm to calculate the sum of first n

natural numbers.
Approach 1
1. Algorithm Summation(n)
2. {
3. Sum = 1;
4. for(Counter = 1;Counter ≤ n;Counter++)
5. Sum = Sum + Counter;
6. return(Sum);
7. }//End of Algo

Chandan Sinha Roy Analysis of Algorithms

28
 How can we improve the complexity?
Approach 2
1. Algorithm Summation(n)
2. {
3. Result=(n*(n+1))/2;
4. }//End of Algo

Complexity O(1)

Chandan Sinha Roy Analysis of Algorithms

29
 How can we improve the complexity?

Write an algorithm to find the value of xn where n is

integral power of 2.
Approach 1
1. Algorithm Power(x, n)
2. {
3. Result = 1;
4. for(Counter = 1;Counter≤n;Counter++)
5. Result = Result*x;
6. return(Result);
7. }//End of Algo

Chandan Sinha Roy Analysis of Algorithms

30
 How can we improve the complexity?
Approach 2
1. Algorithm Power(x, n)
2. {
3. if (n=1) then return x;
4. else return Power(x*x,n/2);
5. }//End of Algo

Complexity
n
T (n)  T    c  T ( n)  O (log n)
2

Chandan Sinha Roy Analysis of Algorithms

31
 End of unit 3

Chandan Sinha Roy Analysis of Algorithms

32