Design and Analysis of

Algorithms

Recurrences
Fall 2020

1
 Recurrences

• The expression:
 c n 1

T ( n)  
2T  n   cn n  1
  2 
is a recurrence.
– Recurrence: an equation that describes a
function in terms of its value on smaller
functions
2
 Recurrence Examples

 0 n0  0 n0
s ( n)   s ( n)  
c  s (n  1) n  0 n  s (n  1) n  0

 c n 1 
  c n 1

T ( n)   T ( n)  
2T  n   c n  1  n
aT    cn n  1
  2  
 b
3
 Methods to solve recurrence
• Substitution method
• Iteration method
• Recursion tree method
• Master Theorem

4
5
 Example
 1 n 1


T ( n)  
2T 

n
n n 1

  2

1. Guess: T (n) = n lg n + n.

2. Induction:
 Basis: n = 1⇒n lg n + n = 1 = T (n)
 Inductive step: Inductive hypothesis is that T (k) = k
lg k + k for all k < n.
 We’ll use this inductive hypothesis for T (n/2).

6
Example (Cont…)

7
Generally, we use asymptotic notation:
•We would write T (n) = 2T (n/2) + (n).
•We assume T (n) = O(1) for sufficiently
small n.
•We express the solution by asymptotic
notation: T (n) = ϴ(n lg n).

8
 Substitution Method
• For the substitution method:
– Name the constant in the additive term.
– Show the upper (O) and lower (Ω)
bounds separately. Might need to use
different constants for each.

9
 Example:
• T (n) = 2T (n / 2)+ϴ(n).
– If we want to show an upper bound
of T (n) = 2T (n/ 2) + O(n),
• We write T (n) ≤ 2T (n / 2) + cn for
some positive constant c.

10
 Upper bound
Guess:
•T (n) ≤ dn lg n for some positive constant d.
We are given c in the recurrence.
•We get to choose d as any positive
constant.
•It’s OK for d to depend on c.

11
 Upper Bound
• Substitution:

12
 Lower bound
• T (n) ≥ 2T (n/2) + cn for some positive
constant c.
• Guess:
– T (n) ≥ dn lg n for some positive constant d.

13
 Lower Bound
• Substitution:

Therefore, T (n) = Ω(n lg n). 14

15
16
17
18
 Methods to solve recurrence
• Substitution method
• Iteration method
• Recursion tree method
• Master Theorem

19
 Solving Recurrences
• Another option is “iteration method”
– Expand the recurrence
– Work some algebra to express as a
summation
– Evaluate the summation
• We will show several examples

20
  0 n0
s ( n)  
c  s (n  1) n  0
• s(n) =
c + s(n-1)
c + c + s(n-2)
2c + s(n-2)
2c + c + s(n-3)
3c + s(n-3)
…
kc + s(n-k) = ck + s(n-k)
21
  0 n0
s ( n)  
c  s (n  1) n  0

• So far for n >= k we have

– s(n) = ck + s(n-k)
• What if k = n?
– s(n) = cn + s(0) = cn

22
  0 n0
s ( n)  
c  s (n  1) n  0
• So far for n >= k we have
– s(n) = ck + s(n-k)
• What if k = n?
– s(n) = cn + s(0) = cn
• So  0 n0
s ( n)  
c  s (n  1) n  0
• Thus in general
– s(n) = cn
23
  0 n0
s ( n)  
n  s (n  1) n  0
• s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
= …
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)

24
  0 n0
s ( n)  
• s(n) n  s (n  1) n  0
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
= …
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)

i
i  n  k 1
 s(n  k )

25
  0 n0
s ( n)  
n  s (n  1) n  0

• So far for n >= k we have

n

i
i  n  k 1
 s(n  k )

26
  0 n0
s ( n)  
n  s (n  1) n  0

• So far for n >= k we have

n

i
i  n  k 1
 s(n  k )
• What if k = n?

27
  0 n0
s ( n)  
n  s (n  1) n  0
• So far for n >= k we have

i
i  n  k 1
 s(n  k )

• What if k = n?
n
n 1
n


i 1
i  s (0)   i  0  n
i 1 2

28
  0 n0
s ( n)  
n  s (n  1) n  0
• So far for n >= k we have
n

i
i  n  k 1
 s(n  k )

• Whatn if k = n?
n 1
n


i 1
i  s(0)   i  0  n
i 1 2

• Thus in general n 1
s ( n)  n
2
29
 Methods to solve recurrence
• Substitution method
• Iteration method
• Recursion tree method
• Master Theorem

30
31
32
33
34
35
36
37
38
39
40
 Methods to solve recurrence
• Substitution method
• Iteration method
• Recursion tree method
• Master Theorem

41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58