The Design and Analysis of Algorithms

Chapter 2: Fundamentals of
the Analysis of Algorithm
Efficiency
Mathematical Analysis of Non-recursive and Recursive Algorithms

LAARNI D. PANCHO, MIT

CS & IT Department
Section 2.3. Mathematical Analysis
of Non-recursive Algorithms
Steps in mathematical analysis of non-
recursive algorithms
■ Decide on parameter n indicating input size
■ Identify algorithm’s basic operation
■ Determine worst, average, and best case for input
of size n
■ Set up summation for C(n) reflecting algorithm’s
loop structure
■ Simplify summation using standard formulas (see
Appendix A)

2
 Example: Selection sort 1
■ Input: An array A[0..n-1]
■ Output: Array A[0..n-1] sorted in ascending order
for i ← 0 to n-2 do
min ← i
for j = i + 1 to n – 1 do
if A[j] < A[min]
min ← j
swap A[i] and A[min]
3
 Example: Selection sort 2
■ Basic operation: comparison
Inner loop:
n-1
S(i) = Σ 1 = (n-1) – (i + 1) + 1 = n – 1 – i
j = i+1

Outer loop:
n-2 n-2 n-2 n-2
C(n) = Σ S(i) = Σ (n – 1 – i) = Σ (n – 1) – Σ i
i=0 i=0 i=0 i=0
n
Basic formula: Σi = n(n+1) / 2
i=0
C(n) = (n – 1 )(n -1 ) – (n-2)(n-1)/2 = (n – 1) [2(n – 1) – (n – 2)] / 2 =
= (n – 1) n / 2 = O(n2)
4
Section 2.4. Mathematical
Analysis of Recursive Algorithms
Steps in mathematical analysis of non-
recursive algorithms

■ Decide on parameter n indicating input size

■ Identify algorithm’s basic operation
■ Determine worst, average, and best case for input of
size n
■ Set up a recurrence relation and initial condition(s)
■ Solve the recurrence to obtain a closed form or
estimate the order of magnitude of the solution (see
Appendix B)
5
Important Recurrence Types

■ One (constant) operation reduces problem size by one.

T(n) = T(n-1) + c T(1) = d
Solution: T(n) = (n-1)c + d linear

■ A pass through input reduces problem size by one.

T(n) = T(n-1) + cn T(1) = d
Solution: T(n) = [n(n+1)/2 – 1] c + d quadratic

■ One (constant) operation reduces problem size by half.

T(n) = T(n/2) + c T(1) = d
Solution: T(n) = c log n + d logarithmic

■ A pass through input reduces problem size by half.

T(n) = 2T(n/2) + cn T(1) = d
Solution: T(n) = cn log n + d n n log n 6
 Example 1: Factorial

n! = n*(n-1)! Telescoping:
0! = 1 T(n) = T(n-1) + 1
T(n-1) = T(n-2) + 1
Recurrence T(n-2) = T(n-3) + 1
relation: …
T(2) = T(1 ) + 1
T(n) = T(n-1) + 1
Add the equations and cross equal
T(1) = 1 terms on opposite sides:

■ T(n) = T(1) + (n-1) =

= n
7
Example 2: Binary Search
■ Recurrence Relation
T(n) = T(n/2) + 1, T(1) = 1
■ Telescoping
T(n/2) = T(n/4) + 1
…
T(2) = T(1) + 1
■ Add the equations and cross equal terms on opposite sides:

T(n) = T(1) + log(n) = O(log(n)) 8

Master Theorem: A general
divide-and-conquer recurrence
T(n) = aT(n/b) + f (n) where f (n) ∈ Θ(nk)

a < bk T(n) ∈ Θ(nk)

a = bk T(n) ∈ Θ(nk log n )
a > bk T(n) ∈ Θ(n to the power of (logba))

Note: the same results hold with O instead of Θ.