Introduction to Algorithms

3rd ed.

Week 3 [Feb. …, 2024]

1
 Review
Analyzing merge sort
MERGE-SORT A[1 . . n]

If n = 1, done else calculate mid

C1, T(n)

C2, Q(1) Recursively Mergesort A[ 1 . . n/2 ] and

C3, 2T(n/2) Mergesort A[ n/2+1 . . n ]

C4, Q(n) “Merge” the 2 sorted lists

Should be T( n/2 ) + T( n/2 ) , but it turns out not to matter asymptotically.

T(n)=2T(n/2)+ Q(n) + C
T(1)=1

L1.2
 Kinds of analyses
Worst-case: (usually)
• T(n) = maximum time of algorithm on any
input of size n.
Average-case: (sometimes)
• T(n) = expected time of algorithm over all
inputs of size n.
• Need assumption of statistical distribution
of inputs.
Best-case: (NEVER)
• Cheat with a slow algorithm that works fast
on some input.
3
 Machine-independent time
What is insertion sort’s worst-case time?

• Ignore machine dependent constants,

otherwise impossible to verify and to compare algorithms

• Look at growth of T(n) as n → ∞ .

“Asymptotic Analysis”

4
Asymptotic notations
introduction

Which is larger:
10n2 or 2n3
What is n0
--------------------------------------------------------------
2n3<= 3n3+5 <=5n3 n0=2

5
 Q-notation
DEF:

Q(g(n)) = { f (n) : there exist positive constants c1, c2, and

n0 such that 0  c1 g(n)  f (n)  c2 g(n)
for all n  n0 }
Basic manipulations:

• Drop low-order terms; ignore leading constants.

• Example: 3n3 + 90n2 – 5n + 6046 = Q(n3)

3
2n <= 3
3n +5 <=5n3 n0=2
L1.6
Growth

7
 Asymptotic performance
When n gets large enough, a Q(n2) algorithm always beats a Q(n3) algorithm.

.
• Asymptotic analysis is a
useful tool to help to
structure our thinking
toward better algorithm
• We shouldn’t ignore
T(n)
asymptotically slower
algorithms, however.
• Real-world design
n n0 situations often call for a
L1.8 careful balancing
 Insertion sort analysis (back)
Worst case: Input reverse sorted.

n
T ( n) =  Q ( j ) = Q( )
[arithmetic series]
n2
j =2
Average case: All permutations equally likely.

n
T ( n) =  Q( j / 2) = Q(n 2 )
j =2
Is insertion sort a fast sorting algorithm?
• Moderately so, for small n.
• Not at all, for large n.
L1.9
 Example 2: Integer Multiplication
• Let X = A B and Y = C D where A,B,C and D are
integer digits
• Simple Method: XY = 2 x 2 multiplications
• Running Time Recurrence
T(n) is the max number of multiplications

• Solution T(n) = q(n2), n is number of digits in the

number.
• Can we do some enhancement? Yes. (advanced math)

L1.10
 Time Complexity- Asymptotic
Notations
Omega Ω: Lower
Big-oh O: Upper bound Theta Ɵ: Avg bound
bound

W(n) -------------A(n)-------------B(n)
nn --------------------------------------1
11
Growth

12
 O-notation: asymptotic upper bound
DEF:

O(g(n)) = { f (n) : there exist positive constants c and n0

such that 0  f (n)  c g(n) for all n  n0 }
Alternatively, we say f (n) is O(g(n)) ……
So, f(n) = O(g(n)) means that f(n) is
asymptotically less than or equal to g(n).
Basic manipulations:

• Drop low-order terms; ignore leading constants.

• Example: 3n3 + 90n2 – 5n + 6046 = O(n3)
=O(n4)
=O(n5)
=O(nm>=3);
the best O is the smallest. In this case n3 13
Ω-notation: asymptotic lower bound
DEF:

Ω(g(n)) = { f (n) : there exist positive constants c and

n0 such that 0  c g(n)  f (n) for all
n  n0 }
Basic manipulations:

• Drop low-order terms; ignore leading constants.

• Example: 3n3 + 90n2 – 5n + 6046 = Ω (n3)
=Ω (n2)
=Ω (n1)
=Ω (n<=3); the best Ω is the largest
14
 Q-notation
DEF:

Q(g(n)) = { f (n) : there exist positive constants c1, c2, and

n0 such that 0  c1 g(n)  f (n)  c2 g(n)
for all n  n0 }
Basic manipulations:

• Drop low-order terms; ignore leading constants.

• Example: 3n3 + 90n2 – 5n + 6046 = Q(n3)

15
Once again (definitions)

16
Once again

17
 Asymptotic performance
When n gets large enough, a Q(n2) algorithm always beats a Q(n3) algorithm.

.
• Asymptotic analysis is a
useful tool to help to
structure our thinking
toward better algorithm
• We shouldn’t ignore
T(n)
asymptotically slower
algorithms, however.
• Real-world design
n n0 situations often call for a
careful balancing 18
Asymptotic performance

19
 Examples
Big-Oh O(..)
i.e, O(g(n))
nn
.
.

f(n)=5n2+2n+1 ϵ O(n2)
Ɵ (g(n)) n
g(n)=n2

c=8, f(n)<=8n2, n>=1, n0 =1 Ω(g(n))

sqr(n)

lg(n)

Big-Oh O(..) 1

i.e, Can I do the following:

f(n)z=2n+3 ϵ O(n) 2n+3<=2n2+3n2, n>=1

g(n)=n 2n+3<=5n2, n>=1
2n+3<=2n+3n , 2n+3<=5n, n>=1

g(n)

20
 Examples
nn
.
Big-Oh O(..) .

i.e,
n
f(n)=2n+3 ϵ O(n)

2n+3<=2n+3n , 2n+3<=5n, n>=1 sqr(n)

lg(n)

Big-omega Ω 1
i.e,

f(n)=2n+3 ϵ Ω(n)

2n+3>=1n , 2n+3>=1n, n>=1

g(n) Ɵ(g(n))

21
Growth (back again)

22
 Note:
f (n) =Q(g(n)) implies that f (n) =O(g(n))
and f (n) =Ω (g(n))

However,
f (n) =O(g(n)) doesn’t imply that
f (n) =Q (g(n))
f (n) =Ω (g(n)) doesn’t imply that
f (n) =Q (g(n))

L1.23
Example:
Show that 2n2– 5n +4 is Q(n2)
Sol: Use the definition:
c1n2<=2n2– 5n +4 <=c2n2
Divide by n2:
c1<=2-5/n+4/n2<= c2; let n =1
0

Thus,
c1<1; take c1=1/2
c2>1; take c2=2
Since there are 3 positive constants that satisfy
the inequality; 2n2– 5n +4 is
Q(n2)
24
Theorem

25
Little o o-notation
• The bound 2n2 = O(n2) is asymptotically tight
• 2n = O(n2) is asymptotically not tight

• We use o-notation to denote an upper bound

that is not asymptotically tight
Little o
w-notation
Comparing Functions
With dropping lower non-dominants
analogy