Divide-and-Conquer:

Master Theorem
Neil Rhodes
Department of Computer Science and Engineering
University of California, San Diego

Algorithmic Design and Techniques

Algorithms and Data Structures
 Outline

1 What is the Master Theorem

2 Proof of Master Theorem

 (︀ n )︀
T (n) = T 2 + O(1)
 (︀ n )︀
T (n) = T 2 + O(1)

T (n) = O(log n)
 (︀ n )︀
T (n) = 4T 2 + O(n)
 (︀ n )︀
T (n) = 4T 2 + O(n)

T (n) = O(n2)
 (︀ n )︀
T (n) = 3T 2 + O(n)
 (︀ n )︀
T (n) = 3T 2 + O(n)

T (n) = O(nlog2 3)
 (︀ n )︀
T (n) = 2T 2 + O(n)
 (︀ n )︀
T (n) = 2T 2 + O(n)

T (n) = O(n log n)

 Master Theorem
Theorem
(︀ ⌈︀ n ⌉︀ )︀
If T (n) = aT b + O(nd )
 Master Theorem
Theorem
(︀ ⌈︀ n ⌉︀ )︀
If T (n) = aT b + O(nd ) (for constants
a > 0, b > 1, d ≥ 0), then:
 Master Theorem
Theorem
(︀ ⌈︀ n ⌉︀ )︀
If T (n) = aT b + O(nd ) (for constants
a > 0, b > 1, d ≥ 0), then:
⎧
⎪
⎪
⎨
T (n) =
⎪
⎪
⎩
 Master Theorem
Theorem
(︀ ⌈︀ n ⌉︀ )︀
If T (n) = aT b + O(nd ) (for constants
a > 0, b > 1, d ≥ 0), then:
⎧
d
⎨O(n ) if d > logb a
⎪
⎪
T (n) =
⎪
⎪
⎩
 Master Theorem
Theorem
(︀ ⌈︀ n ⌉︀ )︀
If T (n) = aT b + O(nd ) (for constants
a > 0, b > 1, d ≥ 0), then:
⎧
d
⎨O(n ) if d > logb a
⎪
⎪
T (n) = O(nd log n) if d = logb a
⎪
⎪
⎩
 Master Theorem
Theorem
(︀ ⌈︀ n ⌉︀ )︀
If T (n) = aT b + O(nd ) (for constants
a > 0, b > 1, d ≥ 0), then:
⎧
d
⎨O(n ) if d > logb a
⎪
⎪
T (n) = O(nd log n) if d = logb a
⎪
⎩O(nlogb a )
⎪
if d < log ab
Master Theorem Example 1

(︁ n )︁
T (n) = 4T + O(n)
2
Master Theorem Example 1

(︁ n )︁
T (n) = 4T + O(n)
2
a=4
Master Theorem Example 1

(︁ n )︁
T (n) = 4T + O(n)
2
a=4
b=2
Master Theorem Example 1

(︁ n )︁
T (n) = 4T + O(n1)
2
a=4
b=2
d =1
Master Theorem Example 1

(︁ n )︁
T (n) = 4T + O(n)
2
a=4
b=2
d =1

Since d < logb a, T (n) = O(nlogb a ) = O(n2)

Master Theorem Example 2

(︁ n )︁
T (n) = 3T + O(n)
2
Master Theorem Example 2

(︁ n )︁
T (n) = 3T + O(n)
2
a=3
Master Theorem Example 2

(︁ n )︁
T (n) = 3T + O(n)
2
a=3
b=2
Master Theorem Example 2

(︁ n )︁
T (n) = 3T + O(n1)
2
a=3
b=2
d =1
Master Theorem Example 2

(︁ n )︁
T (n) = 3T + O(n)
2
a=3
b=2
d =1

Since d < logb a,

T (n) = O(nlogb a ) = O(nlog2 3)
Master Theorem Example 3

(︁ n )︁
T (n) = 2T + O(n)
2
Master Theorem Example 3

(︁ n )︁
T (n) = 2T + O(n)
2
a=2
Master Theorem Example 3

(︁ n )︁
T (n) = 2T + O(n)
2
a=2
b=2
Master Theorem Example 3

(︁ n )︁
T (n) = 2T + O(n1)
2
a=2
b=2
d =1
Master Theorem Example 3

(︁ n )︁
T (n) = 2T + O(n)
2
a=2
b=2
d =1

Since d = logb a,
T (n) = O(nd log n) = O(n log n)
Master Theorem Example 4

(︁ n )︁
T (n) = T + O(1)
2
Master Theorem Example 4

(︁ n )︁
T (n) = 1T + O(1)
2
a=1
Master Theorem Example 4

(︁ n )︁
T (n) = T + O(1)
2
a=1
b=2
Master Theorem Example 4

(︁ n )︁
T (n) = T + O(n0)
2
a=1
b=2
d =0
Master Theorem Example 4

(︁ n )︁
T (n) = T + O(1)
2
a=1
b=2
d =0

Since d = logb a, T (n) = O(nd log n) =

O(n0 log n) = O(log n)
Master Theorem Example 5

(︁ n )︁
T (n) = 2T + O(n2)
2
Master Theorem Example 5

(︁ n )︁
T (n) = 2T + O(n2)
2
a=2
Master Theorem Example 5

(︁ n )︁
T (n) = 2T + O(n2)
2
a=2
b=2
Master Theorem Example 5

(︁ n )︁
T (n) = 2T + O(n2)
2
a=2
b=2
d =2
Master Theorem Example 5

(︁ n )︁
T (n) = 2T + O(n2)
2
a=2
b=2
d =2

Since d > logb a, T (n) = O(nd ) = O(n2)

 Outline

1 What is the Master Theorem

2 Proof of Master Theorem

 Master Theorem
Theorem
(︀ ⌈︀ n ⌉︀ )︀
If T (n) = aT b + O(nd ) (for constants
a > 0, b > 1, d ≥ 0), then:
⎧
d
⎨O(n ) if d > logb a
⎪
⎪
T (n) = O(nd log n) if d = logb a
⎪
⎩O(nlogb a )
⎪
if d < log ab
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

n
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level
n
n/b ... n/b
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level
0 n
1 n/b ... n/b
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level
0 n
1 n/b ... n/b
..
.
...
i · · · n/b i · · ·
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level
0 n
1 n/b ... n/b
..
.
...
i · · · n/b i · · ·
..
.
...
logb n 1 ··· 1
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level #
0 n 1
1 n/b ... n/b
..
.
...
i · · · n/b i · · ·
..
.
...
logb n 1 ··· 1
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level #
0 n 1
1 n/b ... n/b a
..
.
... ...
i · · · n/b i · · ·
..
.
...
logb n 1 ··· 1
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level #
0 n 1
1 n/b ... n/b a
..
.
... ...
i · · · n/b i · · · ai
..
.
...
logb n 1 ··· 1
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level #
0 n 1
1 n/b ... n/b a
..
.
... ...
i · · · n/b i · · · ai
..
.
... ...
logb n 1 ··· 1 alogb n
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level work
0 n 1
1 n/b ... n/b a
..
.
... ...
i · · · n/b i · · · ai
..
.
... ...
logb n 1 ··· 1 alogb n
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level work
0 n O(nd )
1 n/b ... n/b a
..
.
... ...
i · · · n/b i · · · ai
..
.
... ...
logb n 1 ··· 1 alogb n
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level work
0 n O(nd )
1 n/b ... n/b aO( bn )d = O(nd ) bad
..
.
... ...
i · · · n/b i · · · ai
..
.
... ...
logb n 1 ··· 1 alogb n
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level work
0 n O(nd )
1 n/b ... n/b aO( bn )d = O(nd ) bad
..
.
... ...
(︀ )︀d (︀ )︀i
i · · · n/b i · · · ai O bni = O(nd ) bad
..
.
... ...
logb n 1 ··· 1 alogb n
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level work
0 n O(nd )
1 n/b ... n/b aO( bn )d = O(nd ) bad
..
.
... ...
(︀ )︀d (︀ )︀i
i · · · n/b i · · · ai O bni = O(nd ) bad
..
.
... ...
logb n 1 ··· 1 alogb n = O(nlogb a )
 (︀ ⌈︀ n ⌉︀ )︀
T (n) = aT b + O(nd )

level work
0 n O(nd )
1 n/b ... n/b aO( bn )d = O(nd ) bad
..
.
... ...
(︀ )︀d (︀ )︀i
i · · · n/b i · · · ai O bni = O(nd ) bad
..
.
... ...
logb n 1 ··· 1 alogb n = O(nlogb a )
∑︀logb n d
(︀ a )︀i
Total: i=0 O(n ) bd
 Geometric Series
For r ̸= 1:

a + ar + ar 2 + ar 3 + · · · + ar n−1
 Geometric Series
For r ̸= 1:

a + ar + ar 2 + ar 3 + · · · + ar n−1
1 − rn
=a
1−r
 Geometric Series
For r ̸= 1:

a + ar + ar 2 + ar 3 + · · · + ar n−1
1 − rn
=a
{︃1 − r
=
 Geometric Series
For r ̸= 1:

a + ar + ar 2 + ar 3 + · · · + ar n−1
1 − rn
=a
{︃1 − r
O(a) if r < 1
=
 Geometric Series
For r ̸= 1:

a + ar + ar 2 + ar 3 + · · · + ar n−1
1 − rn
=a
{︃1 − r
O(a) if r < 1
=
O(ar n−1) if r > 1
Case 1: bad < 1 (d > logb a)

logb n (︁ a )︁i
∑︁
d
O(n ) d
b
i=0
Case 1: bad < 1 (d > logb a)

logb n (︁ a )︁i
∑︁
d
O(n ) d
b
i=0
=O(nd )
Case 2: bad = 1 (d = logb a)

logb n (︁ a )︁i
∑︁
d
O(n ) d
b
i=0
Case 2: bad = 1 (d = logb a)

logb n (︁ a )︁i
∑︁
d
O(n ) d
b
i=0
logb n
∑︁
= O(nd )
i=0
Case 2: bad = 1 (d = logb a)

logb n (︁ a )︁i
∑︁
d
O(n ) d
b
i=0
logb n
∑︁
= O(nd )
i=0
=(1 + logb n)O(nd )
Case 2: bad = 1 (d = logb a)

logb n (︁ a )︁i
∑︁
d
O(n ) d
b
i=0
logb n
∑︁
= O(nd )
i=0
=(1 + logb n)O(nd )
=O(nd log n)
Case 3: bad > 1 (d < logb a)

logb n (︁ a )︁i
∑︁
O(nd ) d
b
i=0
Case 3: bad > 1 (d < logb a)

logb n (︁ a )︁i
∑︁
O(nd ) d
b
i=0
(︂ (︁ a )︁logb n )︂
=O O(nd ) d
b
Case 3: bad > 1 (d < logb a)

logb n (︁ a )︁i
∑︁
O(nd ) d
b
i=0
(︂ (︁ a )︁logb n )︂
=O O(nd ) d
b
logb n )︁
d a
(︁
=O O(n ) d log n
b b
Case 3: bad > 1 (d < logb a)

logb n (︁ a )︁i
∑︁
O(nd ) d
b
i=0
(︂ (︁ a )︁logb n )︂
=O O(nd ) d
b
logb n )︁
d a
(︁
=O O(n ) d log n
b b
logb a )︁
d n
(︁
=O O(n ) d
n
Case 3: bad > 1 (d < logb a)

logb n (︁ a )︁i
∑︁
O(nd ) d
b
i=0
(︂ (︁ a )︁logb n )︂
=O O(nd ) d
b
logb n )︁
d a
(︁
=O O(n ) d log n
b b
logb a )︁
d n
(︁
=O O(n ) d
n
logb a
=O(n )
 Summary
Master theorem is a shortcut:
 Summary
Master theorem is a shortcut:
Theorem
(︀ ⌈︀ n ⌉︀ )︀
If T (n) = aT b + O(nd ) (for constants
a > 0, b > 1, d ≥ 0), then:
⎧
d
⎨O(n ) if d > logb a
⎪
⎪
T (n) = O(nd log n) if d = logb a
⎪
⎩O(nlogb a )
⎪
if d < log ab