Announcements

On Thursday and following classes: sit in your working

group positions
Classroom exercise 3-person groups and associated seating
have been announced
The three videos are up with audio, both as ppts on
sakai and as youtube items. Discard the addresses of the
previous youtube video.
Class schedule
Today: lecture on Asymptotic Notation
Thursday: end of lecture; exercises on Asymptotic Notation;
answering your questions from videos; exercises from material
on videos
Next Tuesday: more exercises from material on videos

UNC Chapel Hill

 Announcements (2)

Homework schedule
Assignment 1 is online now, but the lecture material
on it will not complete until this Thursday
Assignment 1 is due on a week from Thursday (9/7)

UNC Chapel Hill

 Announcements (3):
Assignment Schedule
Assignment 1 will be made on next Tuesday
The first collection of video lectures will need to be
viewed by next Thursday
These 3 videos are on youtube and sakai
The youtube locations will be announced via email and a Sakai
announcement
The worksheets for these videos must be passed in to
gradescope by the time of Thursdays class
Bring your cell phone cameras to class every time. If
there are in-class exercises in that class you will need
either to write your answer to them on your computer and
submit them to gradscope or write them on paper, take a
picture of it on your camera, and submit that.

UNC Chapel Hill

 Announcements (5): LAs and TAs
550 LAs
Nathan Rahul 919-360-1534, rahuln@live.unc.edu
Tyler Thrower 336-409-4198, tthrower@live.unc.edu
Duy Duc Nguyen 919-741-3136, duyn@email.unc.edu
Hayden Lawler 910-685-1048, haydenl@live.unc.edu
Han Bit Yoon 816-806-2888, hbyoon@live.unc.edu
550 TAs
Jisan Mahmud 984-528-6686, jisan.unc.ta@gmail.com
Rohan Prinja 984-244-4669, rohan@cs.unc.edu
Yixin Nie 773-219-4989, yixin1@cs.unc.edu (presently
unavailable)
UNC Chapel Hill
 LA Schedule and Place

LA place for help sessions: SN 155, SN006 (see next slide)

UNC Chapel Hill

 LA Help Sessions Place

LA place for help sessions in SN 155:

Mondays 11-3 and 4-5
Tuesdays 2-3
Wednesdays 11-1
Thursdays 2-3
Fridays 11-2

All others in SN 006 or elsewhere per notice on its door

UNC Chapel Hill

 Getting the Registrations
Regularized
The following registered late but are in the class
Wen Dou (we need a questionnaire), Wai Poon. Jiyu Xu
A few of the 3-person groups are being reconfigured
2 of them have become 4-person groups

UNC Chapel Hill

 Asymptotic Notation

Q(g(n)), O(g(n)), W(g(n)), o(g(n)), w(g(n))

Used to describe the running times of
algorithms
Instead of exact running time, say Q(n2)
Defined for functions whose domain is the
set of natural numbers
i.e., the size of the input
Determine sets of functions, in practice
used to compare two functions
UNC Chapel Hill
 Q-notation

For a given function g(n),

we denote by Q(g(n)) the
set of functions
Q(g(n)) = {f(n): there exist
positive constants c1, c2 and
n0 such that
0 c1g(n) f(n) c2g(n),
for all n some n0 }

We say g(n) is an asymptotically tight bound for f(n)

UNC Chapel Hill

 Example

10n2 - 3n = Q(n2)
What constants for n0, c1, and c2 will
work?
Make c1 a little smaller than leading
coefficient, and c2 a little bigger
To compare orders of growth, look at
leading term
The terms that grow less fast become
negligible as n gets large
UNC Chapel Hill
 Comparing g(n) values in speed

Q(n) < Q(n) if <

Q(log n) < Q(n ) for all >0
Q(log n) is independent of the base of the log
Q(log log n) < Q(log n)
Q(logkn) < Q(log jn) if k > j
When multiplying factors, the non-
common factor decides
Example: Q(n log n) < Q(n n), i.e., Q(n2)

UNC Chapel Hill

 O-notation

For a given function g(n),

we denote by O(g(n))
the set of functions
O(g(n)) = {f(n): there exist
positive constants c and n0
such that
0 f(n) cg(n)
for all n n0 }

We say g(n) is an asymptotic upper bound for f(n)

Typically wed like it to be an asymptotic least upper bound
UNC Chapel Hill
 O-notation: example

For a given function g(n),

we denote by O(g(n))
the set of functions
O(g(n)) = {f(n): there exist
positive constants c and n0
such that
0 f(n) cg(n)
for all n n0 }

What functions are O(n)?

UNC Chapel Hill

 O-notation: example

For a given function g(n),

we denote by O(g(n))
the set of functions
O(g(n)) = {f(n): there exist
positive constants c and n0
such that
0 f(n) cg(n)
for all n n0 }
What functions are O(n)?
Those that grow more slowly than cn: sublinear
Examples: f(n) =d; f(n) =d lg n, f(n) =n
UNC Chapel Hill
 W-notation

For a given function

g(n), we denote by
W(g(n)) the set of
functions
W(g(n)) = {f(n): there
exist positive constants
c and n0 such that
0 cg(n) f(n)
for all n n0 }
We say g(n) is an asymptotic lower bound for f(n)
Typically wed like it to be an asymptotic greatest lower bound
UNC Chapel Hill
 W-notation

For a given function

g(n), we denote by
W(g(n)) the set of
functions
W(g(n)) = {f(n): there
exist positive constants
c and n0 such that
0 cg(n) f(n)
for all n n0 }
What functions are W(n)?

UNC Chapel Hill

 W-notation
For a given function
g(n), we denote by
W(g(n)) the set of
functions
W(g(n)) = {f(n): there
exist positive constants
c and n0 such that
0 cg(n) f(n)
for all n n0 }
What function are W(n)?
Those that grow faster than cn: superlinear
Examples: f(n) = d n lg n, f(n) = n2
UNC Chapel Hill
 Relations Between Q, W, O

For any two functions g(n) and f(n),

f(n) Q(g(n)) if and only if f(n) O(g(n))
and f(n) W(g(n)).

That is, Q(g(n)) = O(g(n)) W(g(n))

Same high order term up to its constant

multiplier

Example: 3n2 +5 lg n Q(4n2 +n) and Q(n2)

UNC Chapel Hill

 Worst Case Running Times
The worst case running time is O(g(n))
Worst case is no worse than O(g(n))
The worst case running time is W(h(n))
The worst case is no better than W(h(n))
If h(n) = g(n), then worst case running time
is Q(g(n))
Same ideas apply for best case and
average case
So we can say, for example, worst case is
O(g(n)) and best case is O(h(n))
UNC Chapel Hill
 Example

Insertion sort takes Q(n2) worst case time,

so sorting (as a problem) is O(n2)

Any sort algorithm must look at each item,

so sorting is W(n)

In fact, using guaranteed

equal-subdivision divide and conquer,
e.g., merge sort, sorting is Q(n lg n) in the
worst case

UNC Chapel Hill

 Asymptotic Notation in Equations

Used to replace expressions containing

lower-order terms
For example,
4n3 + 3n2 + 2n + 1 = 4n3 + 3n2 + Q(n)
= 4n3 + Q(n2) = Q(n3)
In equations, Q(g(n)) always stands for an
anonymous function f(n) Q(g(n))
In the example above, Q(n2) stands for
3n2 + 2n + 1

UNC Chapel Hill

 o-notation

For a given function g(n), we denote by o(g(n)) the

set of functions:
o(g(n)) = {f(n): for any positive constant c > 0, there
exists a constant n0 > 0 such that 0 f(n) < cg(n) for
all n n0 }
f(n) becomes insignificant relative to g(n) as n
approaches infinity: lim [f(n) / g(n)] = 0
n

We say g(n) is an upper bound for f(n) that is not

asymptotically tight. That is, o(g(n)) O(g(n))

UNC Chapel Hill

 o relations among functions

1 o( lg n)
lg n o(n)
n o(n lg n)
n lg n o(n2)
n2 o(n3)
n3 o(n20)
n20 o(2n)

UNC Chapel Hill

 w-notation

For a given function g(n), we denote by w(g(n)) the

set of functions:
w(g(n)) = {f(n): for any positive constant c > 0, there
exists a constant n0 > 0 such that 0 cg(n) < f(n) for
all n n0 }
f(n) becomes arbitrarily large relative to g(n) as n
approaches infinity: lim [f(n) / g(n)] =
n

We say g(n) is a lower bound for f(n) that is not

asymptotically tight. That is, w(g(n)) W(g(n))

UNC Chapel Hill

 w relations among functions
lg n w(1)
n w(lg n)
n lg n w(n)
n2 w(n lg n)
n3 w(n2)
n20 w(n3)
2n w(n20)

UNC Chapel Hill

 Limits
lim
n
[f(n) / g(n)] = 0 f(n) o (g(n))
lim [f(n) / g(n)] < f(n) O(g(n))- o(g(n))
n

0 <nlim

[f(n) / g(n)] < f(n) Q(g(n))
0 <nlim

[f(n) / g(n)] f(n) W(g(n))- w(g(n))
lim [f(n) / g(n)] = f(n) w(g(n))
n

lim [f(n) / g(n)] undefined cant say

n

UNC Chapel Hill

 Comparison of Functions

fg ab

f (n) O(g(n)) a b
f (n) W(g(n)) a b
f (n) Q(g(n)) a = b
f (n) o(g(n)) a < b
f (n) w(g(n)) a > b

UNC Chapel Hill

 Properties
Transitivity
f(n) Q(g(n)) & g(n) Q(h(n)) f(n) Q(h(n))
f(n) O(g(n)) & g(n) O(h(n)) f(n) O(h(n))
f(n) W(g(n)) & g(n) W(h(n)) f(n) W(h(n))
f(n) o(g(n)) & g(n) o(h(n)) f(n) o(h(n))
f(n) w(g(n)) & g(n) w(h(n)) f(n) w(h(n))
Symmetry
f(n) Q(g(n)) if and only if g(n) Q(f(n))
Transpose Symmetry
f(n) O(g(n)) if and only if g(n) W(f(n))
f(n) o(g(n)) if and only if g(n) w((f(n))
UNC Chapel Hill
 (Another Property)
Let h be a monotonically increasing
function and
h(f(n)) G(h(g(n))) with G being some
order operator Q,O, o, W, or w
Then if
h(f) is a positive power of f, the relation
between h(f) and h(g) is the same as for f and g
h(f) is logarithmic in f, the relation between h(f)
and h(g) may be scrunched but not reordered
h(f) is exponential in f, the relation between
h(f) and h(g) may be separated but not
reordered
UNC Chapel Hill
 Useful Properties of Logarithms
Since for logs base a,
log (alog b) = log b log a = (log b) 1, so b= alog b
Thus (alog b )log c = blog c
And (alog b )log c = (alog c )log b = clog b
We conclude that for any logarithm base a
blog c = clog b
Also, for base a except when specified,
log c logcb = logc(blog c) = logc(clog b) = log b logcc
= log b, so to change log bases,
logcb = logab / logac
UNC Chapel Hill
 Useful Facts

For a 0, b > 0, lim n ( lga n / nb ) = 0, so

lga n o(nb), and nb w(lga n )
Prove using LHopitals rule repeatedly

lg(n!) = Q(n lg n)

UNC Chapel Hill

 Examples Comparing Functions
Column A vs. Column B

A B
5n2 + 100n 3n2 + 2

log3(n2) log2(n3)

nlg4 3lg n

lg2n n1/2

UNC Chapel Hill

 Examples Comparing Functions
Column A vs. Column B

A B
5n2 + 100n 3n2 + 2 A Q(B)
A Q(n2), n2 Q(B) A Q(B)

log3(n2) log2(n3)

nlg4 3lg n

lg2n n1/2

UNC Chapel Hill

 Examples Comparing Functions
Column A vs. Column B

A B
5n2 + 100n 3n2 + 2 A Q(B)
A Q(n2), n2 Q(B) A Q(B)

log3(n2) log2(n3) A Q(B)

logba = logca / logcb; A = 2 lg n / lg 3, B = 3 lg n, A/B =2/(3 lg 3)

nlg4 3lg n

lg2n n1/2

UNC Chapel Hill

 Examples Comparing Functions
Column A vs. Column B

A B
5n2 + 100n 3n2 + 2 A Q(B)
A Q(n2), n2 Q(B) A Q(B)

log3(n2) log2(n3) A Q(B)

logba = logca / logcb; A = 2lgn / lg3, B = 3lgn, A/B =2/(3lg3)

nlg 4 3lg n A w(B)

alog b = blog a; B =3lg n=nlg 3; A/B =nlg(4/3) as n

lg2n n1/2

UNC Chapel Hill

 Examples Comparing Functions
Column A vs. Column B

A B
5n2 + 100n 3n2 + 2 A Q(B)
A Q(n2), n2 Q(B) A Q(B)

log3(n2) log2(n3) A Q(B)

logba = logca / logcb; A= 2 lg n / lg 3, B = 3 lg n, A/B =2/(3 lg 3)

nlg 4 3lg n A w(B)

alog b = blog a; B =3lg n=nlg 3; A/B =nlg(4/3) as n

lg2n n1/2 A o(B)

lim ( lga n / nb ) = 0 (here a = 2 and b = 1/2) A o(B)
n

UNC Chapel Hill

 Common Levels on Asymptotic
Complexity
Q(1) The best
Q(lgkn), k>1 Faster than Q(lg n), but seldom
encountered
Q(lg n) Quite fast
Q(n) Rather good
Q(n lg n) A lot better than Q(n2)
Q(nk), k>1 Slowish
Q(an), a>1 Unusably slow

UNC Chapel Hill

 Exercises in Class

For each pair of expressions (A,B) below, indicate whether A is , o, or

of B. Explain your reasoning.

UNC Chapel Hill

 Review on Summation

Why do we need summation formulas?

We need them for computing the running times
of some algorithms. (CLRS/Chapter 3)
Example: Maximum Subvector
Given an array a[1n] of numeric values (can be
positive, zero and negative) determine the
subvector a[ij] (1 i j n) whose sum of
elements is maximum over all subvectors.

1 -2 2 2

UNC Chapel Hill

 Reviews on Summation
MaxSubvector(a, n)
maxsum 0;
for i 1 to n //all possible starting positions
for j = i to n //all possible ending positions
sum 0;
for k i to j
sum += a[k]
maxsum max(sum, maxsum);
return maxsum;
n n j
T(n) = 1
i=1 j=i k=i

NOTE: This is not a simplified solution. Exercise due at

beginning of next class: what is the final answer, using formulas
in the next 2 slides?
UNC Chapel Hill
 Reviews on Summation

Constant series: For n 0,

b
1=b-a+1
i=a

Linear series: For n 0,

n
i = 1 + 2 + + n = n(n+1) / 2

Log series: For n 0,
i=1n log i = Q(n log n)
Power series: For n 0, 0<a<1,
i=0n ai = (1- an+1 )/(1-a) = Q(1)

UNC Chapel Hill

 Reviews on Summation

Quadratic Series: For n 0,

n
i2 = 12 + 22 + + n2 = (2n3 + 3n2 + n) / 6
i=1

Monic polynomial series: For n 0,

n ik = Q(nk+1)

Linear-Geometric Series: For n > 0, c1

n
ici = c + 2c2 + + ncn = [ncn+2 (n+1)cn+1 + c] / (c-1)2
i=1

UNC Chapel Hill