CS6363: Design and Analysis of Computer Algorithms Prof.

Sergey Bereg

Recurrences
A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs.
Example. M ERGE -S ORT, the worst case running time is

Θ(1) if n = 1
T (n) =
2T (n/2) + Θ(n) if n > 1
Solution: T (n) = Θ(n lg n).

M ERGE -S ORT(A, p, r)
// Input: A[p..r] is the array of n numbers.
// Output: A[p..r] with sorted numbers.
1 if p < r then
2 q = b(p + r)/2c
3 M ERGE -S ORT(A, p, q)
4 M ERGE -S ORT(A, q + 1, r)
5 M ERGE(A, p, q, r)

Technicalities
We neglect certain technical details. For example, the actual recurrence for M ERGE -S ORT should be

Θ(1) if n = 1
T (n) =
T (dn/2e) + T (bn/2c) + Θ(1) if n > 1
We usually drop ceiling and floor function.
Example: T (n) = aT (n/b) + f (n).
Boundary conditions. The running time on a constant-sized input is always a constant, so T (n) = Θ(1)
for sufficiently small n. It is assumed by default. So we write the recurrence for M ERGE -S ORT as

T (n) = 2T (n/2) + Θ(n).

Substitution Method
1. Guess the form of the solution.

2. Use the induction to find the constants and

show that the guess is correct.

Example. T (n) = 3T (bn/3c) + 5n.

Guess: T (n) = O(n lg n).
Prove T (n) ≤ cn lg n by induction.
Base case. Does not work for n = 1 since 0 < T (1) ≤ c lg 1 = 0.
Base case should be n = 2, 3, 4, 5 since bn/3c = 1 for these values.
We assume that c ≥ nT lg
(n)
n for all n = 2, 3, 4, 5.
Induction step.

1
 T (n) ≤ 3(cbn/3c lg(bn/3c)) + 5n
≤ cn lg(n/3) + 5n (since bxc ≤ x)
≤ cn lg n − cn lg 3 + 5n

If cn lg n − cn lg 3 + 5n ≤ cn lg n, then the proof is finished.

Can we guarantee the inequality by selecting an appropriate c?
Yes: the inequality cn lg n − cn lg 3 + 5n ≤ cn lg n is the same as −cn lg 3 + 5n ≤ 0 or c ≥ 5/ lg 3 ≈
3.15 We can pick c = 4 for example.

Subtleties
T (n) = T (dn/2e) + T (bn/2c) + 1.
Right guess T (n) = O(n).
Try substitution method T (n) ≤ cn.

T (n) ≤ cdn/2e + cbn/2c + 1

= cn + 1

Doesn’t work. But T (n) ≤ cn − b works:

T (n) ≤ cdn/2e − b + cbn/2c − b + 1

= cn − 2b + 1
≤ cn − b if b ≥ 1

As before we select c large enough to handle the boundary conditions.

Changing variables
√ √
T (n) = 2T (b nc) + lg n. For simplicity T (n) = 2T ( n) + lg n.
Rename m = lg n. Then n = 2m .
T (2m ) = 2T (2m/2 ) + m
Rename S(m) = T (2m ). New recurrence S(m) = 2S(m/2) + m.
S(m) = O(m lg m) and T (n) = S(lg n) = O(lg n lg lg n).

2
Recursion-tree method
T (n) = 3T (bn/4c) + n2
n2
2
n
( n4 )2 ( n4 )2 ( n4 )2
T ( n4 ) T ( n4 ) T ( n4 )
n n n n n n n n n
T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 )

n2 n2

3
( 16 )n2 ( n4 )2 ( n4 )2 ( n4 )2

3 2 2 n 2
( 16 )n ( 16 )
...
3 k 2
( 16 ) n T ( 4nk )

3k

Recursion-tree method
T (n) = 3T (bn/4c) + n2 .
Select k = log4 n. Then 3k = 3log4 n = nlog4 3 < n and T (n/4k ) = O(1).
 2  k
3 2
2 3 2 3
T (n) = n + n + n + ··· + n2 + O(3k )
16 16 16
k 
3 i
X 
2
= n + O(n)
16
i=0
∞ 
3 i
X 
2
< n + O(n)
16
i=0
1
= n2 + O(n)
1 − 3/16
= O(n2 ).

3
Master Method
Theorem. Let a ≥ 1 and b > 1 be constants, f be a non-negative function, and T (n) is defined by the
recurrence
T (n) = aT (n/b) + f (n).
1. If f (n) = O(nlogb a−ε ) for some constant ε > 0, then T (n) = Θ(nlogb a ).
2. If f (n) = Θ(nlogb a ), then T (n) = Θ(nlogb a lg n).
3. If f (n) = Ω(nlogb a+ε ) for some constant ε > 0, and if af (n/b) ≤ cf (n) for some constant c < 1
and all sufficiently large n, then T (n) = Θ(f (n)).

Example. T (n) = 100T (n/10) + n lg n

Case 1 since n lg n = O(n2−ε ) for ε = 1/2.
T (n) = Θ(n2 ).

Example. T (n) = 4T (n/2) + 3n2 + 5n

Case 2 since 3n2 + 5n = Θ(n2 ).
T (n) = Θ(n2 lg n).

Example. T (n) = 3T (n/3) + n1.5

Case 3 since n1.5 = Ω(n1+ε ) for ε = 0.5.
T (n) = Θ(n1.5 ).