A N A LY T I C C O M B I N AT O R I C S

PART ONE

4. Asymptotic
Approximations
http://aofa.cs.princeton.edu
 A N A LY T I C C O M B I N AT O R I C S
PART ONE

4. Asymptotic Approximations
• Standard scale
• Manipulating expansions
• Asymptotics of finite sums
OF
• Bivariate asymptotics

http://aofa.cs.princeton.edu

4a.Asympt.scale
Asymptotic approximations

Goal: Develop accurate and concise estimates of quantities of interest

(log ) ✗ not accurate

= ✗ not concise

ln + + ( ) ✓

Informal definition of concise:

‟easy to compute with constants and standard functions”

3
Notation (revisited)

“Big-Oh” notation for upper bounds

( )= ( ( )) | ( )/ ( )|

“Little-oh” notation for lower bounds

( ) = ( ( )) ( )/ ( )

“Tilde” notation for asymptotic equivalence

( ) ( ) ( )/ ( )

4
Notation for approximations

“Big-Oh” approximation

( )= ( )+ ( ( ))
Error will be at most within a constant factor of h(N ) as N increases.

“Little-oh” approximation

( ) = ( ) + ( ( ))
Error will decrease relative to h(N ) as N increases.

“Tilde” approximation

( ) ( )
Weakest nontrivial o-approximation.

5
Standard asymptotic scale
sequence
Definition. A decreasing series gk(N) with gk+1(N) = o(gk(N )) is called an asymptotic scale.
The series
( ) ( )+ ( )+ ( ) + ...
is called an asymptotic expansion of f. The expansion represents the collection of formulae
( )= ( ( ))
( )= ( )+ ( ( ))
( )= ( )+ ( )+ ( ( ))
( )= ( )+ ( )+ ( )+ ( ( ))

The standard scale is products of powers of N, log N, iterated logs and exponentials.
Typically, we:
• use only 2, 3, or 4 terms (continuing until unused terms are extremely small)
• use ~-notation to drop information on unused terms.
• use O-notation or o-notation to specify information on unused terms.
Methods extend in principle to any desired precision.
6
Example: Asymptotics of linear recurrences ≠

Theorem. Assume that a rational GF f (z)/g(z) with f (z) and g(z) relatively prime and g(0)=0 has
a unique pole 1/β of smallest modulus and that the multiplicity of β is ν. Then
( ) ( ) ( / )
[ ] = ( )( / )
( )

Proof sketch Ex. +

! ! ! 7 2187 275
β + β + ... + β 8 6561 17811
≤< ≤< ≤< 9 19683 20195
Largest term dominates. 10 59049 60073
11 177147 179195

Notes:
Example 1.
• Pole of smallest modulus usually dominates.
3N + 2N ~ 3N
✓
• Easy to extend to cover multiple poles in neighborhood
of pole of smallest modulus. Example 2.
2N + 1.99999N ~ 2N
✗
7
Asymptotics of linear recurrences ≠

Theorem. Assume that a rational GF f (z)/g(z) with f (z) and g(z) relatively prime and g(0)=0 has
a unique pole 1/β of smallest modulus and that the multiplicity of β is ν. Then
( ) ( ) ( / )
[ ] = ( )( / )
( )

Example from earlier lectures.

= = =
Make recurrence valid for all n. = +

Multiply by zn and sum on n. ( )= ( ) ( )+

Solve. ( )=
+
Smallest root of denominator is 1/3. ( )( / )
= =
+ /

8
Fundamental asymptotic expansions
are immediate from Taylor’s theorem.

exponential = + + + + ( )

logarithmic ln( + ) = + + ( )

binomial ( + ) = + + + + ( )

geometric = + + + + ( )

as x ➛ 0.

9
Fundamental asymptotic expansions
are immediate from Taylor’s theorem.

Substitute x = 1/N to get expansions as N ➛ ∞.

exponential /
= + + + + ( )

logarithmic ln( + )= + + ( )

binomial ( + ) = + + + + ( )

geometric = + + + ( )

as N ➛ ∞.

10
Inclass exercise

Develop the following asymptotic approximations

ln( + ) + ln( ) ( / )

= + ( ) + ( )

= + ( )

ln( + ) ln( ) ( / )

= + ( )+ + + ( )

= + ( )

11
 A N A LY T I C C O M B I N AT O R I C S
PART ONE

4. Asymptotic Approximations
• Standard scale
• Manipulating expansions
• Asymptotics of finite sums
OF
• Bivariate asymptotics

http://aofa.cs.princeton.edu

4a.Asympt.scale
 A N A LY T I C C O M B I N AT O R I C S
PART ONE

4. Asymptotic Approximations
• Standard scale
• Manipulating expansions
• Asymptotics of finite sums
OF
• Bivariate asymptotics

http://aofa.cs.princeton.edu

4b.Asympt.manip
Manipulating asymptotic expansions

Goal.
Develop expansion on the standard scale for any given expression.

! "
− ( )
+ ln( + )

Techniques. Why?

simplification Facilitate comparisons of different quantities.

Simplify computations.
substitution ! "
factoring Ex.
N = 106 ??
multiplication
division
composition
exp/log

14
Manipulating asymptotic expansions

Simplification. An asymptotic series is only as good as its O-term.

Discard smaller terms.
ln + + ( ) ✗

ln + ( ) ✓

Substitution. Change variables in a known expansion.

Taylor series ln( + ) = + + ( )

Substitute x = 1/N ln( + )= + + ( ))

15
Manipulating asymptotic expansions

Factoring. Estimate the leading term, factor it out, expand the rest.

Factor out 1/N2. =

+ /

Expand the rest. = + ( )

Distribute. = + ( )

16
Manipulating asymptotic expansions

Multiplication. Do term-by-term multiplication, simplify, collect terms.

! "! "
Ex. ( ) = ln +γ+ ( ) ln +γ+ ( )

log
Term-by-term = (ln ) + ln + ( )
multiplication.
+ ln + + ( )
log
+ ( )+ ( )+ ( )
slight improvement big improvement
in precision in precision
log
Collect terms. = (ln ) + ln + + ( )

( ) (ln ) + γ ln +γ
1000 56.032 47.717 55.692 56.025

10000 95.797 84.830 95.463 95.796

May need trial-and-error to get desired precision.
100000 146.172 132.547 145.838 146.172 ✓ 17
Manipulating asymptotic expansions

Division. Expand, factor denominator, expand 1/(1−x), multiply.

Ex.
ln( + )
ln +γ+ ( )
Expand. =
ln + ( )

γ
Factor denominator. + + ( ) OK to simplify by replacing
= ln
O(1/N log N) by O(1/N)
+ ( )

! γ "! "
Expand 1/(1−x). = + + ( ) + ( )
ln

Multiply.
γ
= + + ( )
ln
18
Manipulating asymptotic expansions

Composition. Substitute an expansion.

Substitute HN expansion.
= ln +γ+ ( / )
Lemma.
γ ( / ) ( / )
= = + ( )

Expand ex. = ( + ( )+ ( )

γ
!
Simplify. = ( + ( )
γ
γ
Distribute. = + ( ) 1000 1782 1781

10000 17812 17811

big improvement
in precision 100000 178108 178107

1000000 1781073 1781072 ✓

19
Manipulating asymptotic expansions

Exp/log. Start by writing f (x) = exp(ln(f (x)).

! " Q. How would you
− compute values
for large N?
! "" # #$
Exp/log. = exp ln −
! " #$
= exp ln −
! " #$
Expand ln(1+x) = exp − + ( )
! " Lemma.
Distribute. = exp − + ( ) ( / )
= + ( )

= / + ( ) ! "
− /
10000 0.367861 0.367879
big improvement
in precision 100000 0.367878 0.367879

1000000 0.367879 0.367879 ✓

20
Inclass exercises

Develop asymptotic approximations for

ln( − ) ( / )

= ln + ln( − ) Factor out ln N.

= ln − + ( ) Expand the rest.

( ) ( / )

! "! "
( ) = ln +γ+ + ( ) ln +γ+ + ( )

ln
= (ln ) + ln + + + ( )

21
 A N A LY T I C C O M B I N AT O R I C S
PART ONE

4. Asymptotic Approximations
• Standard scale
• Manipulating expansions
• Asymptotics of finite sums
OF
• Bivariate asymptotics

http://aofa.cs.princeton.edu

4b.Asympt.manip
 A N A LY T I C C O M B I N AT O R I C S
PART ONE

4. Asymptotic Approximations
• Standard scale
• Manipulating expansions
• Asymptotics of finite sums
OF
• Bivariate asymptotics

http://aofa.cs.princeton.edu

4c.Asympt.sums
Asymptotics of finite sums

Bounding the tail. Make a rapidly decreasing sum infinite.

( ) ( )
! = ! = !
! !
! >
= + ( ) | |< + + + ... =
+ ( + ) ( + )

Using the tail. The last term of a rapidly increasing sum may dominate.
!
!= ! + + = ! + ( )
!

N −1 terms, each less than 1/N(N −1)

Approximating with an integral.

= = ln see text for proofs;

stay tuned for
better approximations
ln ! = ln ln = ln +

24
Euler-Maclaurin Summation
is a classic formula for estimating sums with integrals.

Theorem. (Euler-Maclaurin summation). Let f be a function defined on [1, ∞)

whose derivatives exist and are absolutely integrable. Then

( )= ( ) + ( )+ + ( ) ( ) + ...

Asymptotic series diverges; need to check bound on last term (see text for many details).
BUT this form is useful for many applications.

Classic example 1. = ln + + + ( )

Classic example 2. ln ! = ln + ln + + ( )

25
Inclass exercise

Given Stirling’s approximation ln != ln + ln + ( )

Develop an asymptotic approximation for ( / )

= exp ln !) ln !

= exp ln( ) + ln + ( / )
( ln( ) + ln + ( / )) ln ln = ln ln ln

= exp ln ln + ( / ) = ln

Ex. ! "
= + ( )
∼ √

26
Asymptotics of the Catalan numbers: an application

Q. How many bits to represent a binary tree with N internal nodes?

! " N = 106 ??

A. At least lg ✗ not concise

+

A. At least ∼ lg √ ∼ − . lg ✓

Note: Can do it with 2N bits

preorder traversal
0 for internal nodes
1 for external nodes
0000101100010101101011001101100000111001011011001011011010011011000001111000011
0110000111001001001111011000101111010000110110110010110000110011001110101100111
27
 A N A LY T I C C O M B I N AT O R I C S
PART ONE

4. Asymptotic Approximations
• Standard scale
• Manipulating expansions
• Asymptotics of finite sums
OF
• Bivariate asymptotics

http://aofa.cs.princeton.edu

4c.Asympt.sums
 A N A LY T I C C O M B I N AT O R I C S
PART ONE

4. Asymptotic Approximations
• Standard scale
• Manipulating expansions
• Asymptotics of finite sums
OF
• Bivariate asymptotics

http://aofa.cs.princeton.edu

4d.Asympt.bivariate
Bivariate asymptotics
is often required to analyze functions of two variables.

Ex. applications in analysis of algorithms involve

• N (size)
• k (cost)

Challenges:
• asymptotics depends on relative values of variables
• may need to approximate sums over whole range of relative values.

for k = 0
Example 1: Binomial distribution
exponentially small for k close to N

! 1 for k = 0
Example 2: Ramanujan Q-distribution
( )! exponentially small for k close to N

30
Binomial distribution

.5
.5

.75

.25
.25

.125
.125
.103
.103

00
00 N 2N

k (scaled by a factor of 2N )
31
Ramanujan Q-distribution

11
!
( )!
.75
.75

.5
.5

.25
.25

0
0
00 N/2 N

k (scaled by a factor of N )
32
Ramanujan Q-distribution

!
= exp ln ! ln( )! ln
( )!
ln ! = ( + ) ln + ln + ( )
Stirling's approximation. = exp ( + ) ln + ln

( + ) ln( )+ ln ln + ( )

Collect terms = exp ( + ) ln( ) + ( )

ln( )= + ( )

Expand ln(1 − k/N) = exp + + ( )+ ( )

k k/N k 3/N 2

Simplify. = / + ( )+ ( ) N 2/5 1/N 3/5 1/N 4/5

N 1/2 1/N 1/2 1/N 1/2

N 3/5 1/N 2/5 1/N 1/5

33
Normal approximation to the binomial distribution

= exp ln !) ln( )! ln( + )!

ln ! = ( + ) ln + ln + ( )

Stirling's = exp ( + ) ln( ) + ln + ( / )

approximation.
( + ) ln( ) + ln + ( / )

( + + ) ln( + ) ln + ( / )

Collect terms = exp ( ) ln ln ( + ) ln( ) ( + + ) ln( + )+ ( / )

Rearrange terms = exp ) ln ln

( + )(ln( ) + ln( + )) + (ln( ) ln( + )) + ( / )

Expand ln(1 − k/N) = exp ( ) ln ln + ( )+ ( ) ln( ) + ln( + )= + ( )

and ln(1 + k/N).
ln( ) ln( + )= + ( )
/
= + ( )+ ( )
34
Fundamental bivariate approximations

uniform central

/ /
normal + ( /
) + ( )+ ( )

Poisson + ( ) + ( )+ ( )
! !

! /( ) /( )
Q + ( ) + ( )+ ( )
( )!

35
 Next challenge: Approximating sums via bivariate asymptotics

Example: Ramanujan Q-function 1

.75

! .5
( )
( )!
.25

0
0

What is the area under this curve?

Observations:
• nearly 1 for small k
• negligable for large k
• bivariate asymptotics needed to give different estimates in different ranges.
negligible
36
Laplace method

To approximate a sum:
• Restrict the range to an area that contains the largest summands.
• Approximate the summand.
• Extend the range by bounding the tails to get a simpler sum.
• Approximate the new sum with an integral.

37
Laplace method for Ramanujan Q-function

!
( )
( )!

Restrict the range to an area that ! ! Take k0 = o(N2/3) to

contains the largest summands. ( )= + make tail
( )! <
( )!
exponentially small.

! /
Approximate the summand. Q-distribution
( )! approximation.

Extend the range by bounding / Tail is also

( ) exponentially small
the tails to get a simpler sum.
for this sum.

Approximate the new sum /

with an integral. ( ) = /

38
 A N A LY T I C C O M B I N AT O R I C S
PART ONE

4. Asymptotic Approximations
• Standard scale
• Manipulating expansions
• Asymptotics of finite sums
OF
• Bivariate asymptotics

http://aofa.cs.princeton.edu

4d.Asympt.bivariate
Exercise 4.9

How small is "exponentially small"?

AN INTRODUCTION

ANALYSIS
TO THE

ALGORITHMS
OF

S E C O N D E D I T I O N

ROBERT SEDGEWICK
PHILIPPE FLAJOLET

40
Exercise 4.71

Asymptotics of another Ramanujan function.

AN INTRODUCTION

ANALYSIS
TO THE

ALGORITHMS
OF

S E C O N D E D I T I O N

ROBERT SEDGEWICK
PHILIPPE FLAJOLET

41
Assignments for next lecture

1. Write a program that takes N and k from the command line

and prints ! "
lg

2. Write up solutions to Exercises 4.9 and 4.70.

3. Read pages 149-215 (Asymptotic Approximations) in text.

42
 A N A LY T I C C O M B I N AT O R I C S
PART ONE

4. Asymptotic
Approximations
http://aofa.cs.princeton.edu