Design and Analysis of

Algorithms

Lecture # 4
Muhammad Nasir
Department of Computer Science
COMSATS University Islamabad,
Lahore Campus
mnasir@cuilahore.edu.pk
Running Time Analysis (Revisited)
• Depends upon the input size e.g. n
• Depends upon the structure of input
Complexity with input size
Asymptotic Notation
Goal: to simplify analysis by getting rid of
unneeded information (like “rounding”
1,000,001≈1,000,000)
We want to say in a formal way 3n2 ≈ n2
Asymptotic Notation
Categorize algorithms based on asymptotic growth
rate e.g. linear, quadratic, polynomial, exponential.
Ignore small constant and small inputs.
Estimate upper bound and lower bound on growth
rate of time complexity function.
Describe running time of algorithm as n grows to
infinity.
Limitations:
Not always useful for analysis on fixed-size inputs.
All results are for sufficiently large inputs.
Asymptotic Notation
Concentrate on order of growth of an
algorithm’s basic operation as the input
size grows.

Asymptotic Notations O, Ω, Θ, o, 𝜔.
Θ-notation to mean “order exactly”.
O-notation to mean “order at most”.
Ω-notation to mean “order at least”.
Little o to mean “tight upper bound”.
Little 𝜔 to mean “tight lower bound”.
Classes of Functions
Let {g(n)} denote the class (set) of functions that are
O(g(n))
We have
{n}  {n2}  {n3}  {n4}  {n5}  …
where the containment is strict

{n3}
{n2}
{n}
Asymptotic analysis - terminology
Special classes of algorithms:
• Logarithmic: O(log n)
• Linear: O(n)
• Quadratic: O(n2)
• Polynomial: O(nk), k ≥ 1
• Exponential: O(an), n > 1
Polynomial vs. exponential ?
Logarithmic vs. polynomial ?
Asymptotic Running Times
Here is a list of common asymptotic running times:
Θ(1): Constant time; can’t beat it!
Θ(logn): Binary Search (Sorted array of length n).
Θ(n): About the fastest that an algorithm can run.
Θ(nlogn): Best sorting algorithms.
Θ(𝑛2 ), Θ(𝑛3 ): Polynomial time. Acceptable when n is not to
large (𝑛 ≤ 1000).
Θ(2𝑛 ), Θ(3𝑛 ): Exponential time. Acceptable only if n is small,
e.g., 𝑛 ≤ 50.
Θ(𝑛!), Θ(𝑛𝑛 ): Acceptable only for really small n, e.g. 𝑛 ≤ 20.
O-notation (Informal Introduction)
O(g(n)) is the set of all functions with a lower or
same order of growth as g(n), within a constant
multiple, as n goes to infinity
n ∈ O(n2) and ½ n(n−1) ∈ O(n2)
n3 ∉ O(n2) and n4+n+1 ∉ O(n2)
O-notation (Formal Introduction)
A function t(n) is said to be of O(g(n)),
denoted t(n)∈O(g(n)), if t(n) is bounded above
by some constant multiple of g(n) for all large
n,
i.e., if there exist some positive constant
c and some non-negative integer n0 such that
t(n) ≤ cg(n) for all n ≥ n0.
O-notation (Formal Introduction)
O-notation (Formal Introduction)

• Let t(n) = 5n + 25
• find the constants c and n0 to prove
formally that t(n) ∈ O(n2)
Ω-notation (Informal Introduction)

Ω(g(n)), stands for the set of all functions with a

higher or same order of growth as g(n), within a
constant multiple, as n goes to infinity
n3 ∈ Ω(n2)
12n(n−1) ∈ Ω(n2)
100n+5∉ Ω(n2)
Ω-notation (Formal Introduction)
A function t(n) is said to be in Ω(g(n)),
denoted t(n)∈Ω(g(n)), if t(n)is bounded below
by some positive constant multiple of g(n) for
all large n,
i.e., if there exist some positive constant
c and some non-negative integer n0 such that
t(n) ≥ cg(n) for all n ≥ n0.
Ω-notation (Formal Introduction)
Ω-notation (Formal Introduction)

Let t(n) = 12n(n−1)

Find the constants c and n0 to prove formally
that t(n) ∈ Ω(n2)
Θ-notation (Informal Introduction)

Θ(g(n)) is the set of all functions that have the

same order of growth as g(n)
Θ-notation (Formal Introduction)
A function t(n) is said to be in Θ(g(n)),
denoted t(n)∈Θ(g(n)), if t(n)is bounded both
above and below by some positive constant
multiples of g(n) for all large n,
i.e., if there exist some positive
constants c1 and c2 and some nonnegative
integer n0 such that
c1 g(n) ≤ t(n) ≤ c2g(n) for all n ≥ n0.
Θ-notation (Formal Introduction)
Θ-notation (Formal Introduction)
For example, functions like

are all asymptotically equivalent. As n

becomes large, the dominant (fastest
growing) term is some
constant times n2.
Θ-notation-Example

Consider the function:

𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑

Lower bound 𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 grows

asymptotically at least as fast as 𝑛2 ,
Upper bound f(n) grows no faster asymptotically
than 𝑛2 ,
Θ-notation-Example

Lower bound: f(n) grows asymptotically at least

as fast as 𝒏𝟐 . For this, need to show that there
exist positive constants 𝑐1 and 𝑛0 , such that
f(n) ≥ 𝑐1 𝑛2 for all 𝑛 ≥ 𝑛0 . Consider the reasoning:

𝒇(𝒏) = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 ≥ 𝟖𝒏𝟐 − 𝟑 = 𝟕𝒏𝟐 + 𝒏𝟐 − 𝟑 ≥ 𝟕𝒏𝟐

Thus c1 = 7. We implicitly assumed that 2𝑛 ≥ 0

and 𝑛2 − 3 ≥ 0 These are not true for all 𝑛 but if
𝑛 ≥ 3, then both are true. So select 𝑛0 ≥ 3.
We then have 𝑓(𝑛) ≥ 𝑐1𝑛2 for all n ≥ 𝑛0 .
Θ-notation-Example

Upper bound: f(n) grows asymptotically no faster

than 𝒏𝟐 . For this, need to show that there exist
positive constants 𝑐2 and 𝑛0 , such that f(n) ≤ 𝑐2 𝑛2
for all 𝑛 ≥ 𝑛0 . Consider the reasoning:

𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 ≤ 𝟖𝒏𝟐 + 𝟐𝒏 ≤ 𝟖𝒏𝟐 + 𝟐𝒏𝟐 = 𝟏𝟎𝒏𝟐

Thus c2 = 10. We implicitly assumed that 2𝑛 ≤ 𝟐𝒏𝟐

This is not true for all 𝑛 but if 𝑛 ≥1. So select 𝑛0 ≥
1. We then have f(n) ≤ 𝑐2 𝑛2 for all n ≥ 𝑛0 .
Θ-notation-Example

From lower bound we have 𝑛0 ≥ 3

From upper bound we have 𝑛0 ≥ 1
But we need one….
Combining the two, we let 𝑛0 be the larger of the
two: 𝑛0 ≥ 3. In conclusion, if we let c1 = 7, c2 =
10 and 𝑛0 ≥ 3, we have:
𝟕𝒏𝟐 ≤ 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 ≤ 𝟏𝟎𝒏𝟐 for all 𝑛 ≥ 3
We established,
0 ≤ 𝑐1 𝑔 𝑛 ≤ 𝑓 𝑛 ≤ 𝑐2 𝑔(𝑛) for all 𝑛 ≥ 𝑛0
Asymptotic Notation (Other Bounds)
We established that 𝑓 𝑛 ∈ 𝑛2 .
Let’s show why f(n) is not in some other asymptotic
class. First,
Show that 𝑓(𝑛) ∉ 𝜃(𝑛). If this were true, we would
have had to satisfy both the upper and lower
bounds.
Here, the lower bound is satisfied because
𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 does grow at least as fast
asymptotically as n. But the upper bound is false.
Upper bounds requires that there exist positive
constants c2 and n0 such that f(n) ≤ 𝑐2 𝑛 for all 𝑛 ≥
𝑛0 .
Asymptotic Notation (Other Bounds)
Informally we know that 𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 will
eventually exceed 𝑐2 𝑛 no matter how large we
make c2. Let check it…
Suppose we assume that constants c2 and n0 did
exist such that 𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 ≤ 𝑐2 𝑛 for all
𝑛 ≥ 𝑛0 Since this is true for all sufficiently large n
then it must be true in the limit as n tends to
infinity. If we divide both sides by n, we have:
3
lim 8𝑛 + 2 − ≤ 𝑐2
𝑛→∞ 𝑛
The left side tends to ∞. So, no matter how large c2
is, the statement is violated. Thus 𝑓(𝑛) ∉ 𝜃(𝑛).
 Asymptotic Notation (Other Bounds)
Show that 𝑓(𝑛) ∉ 𝜃(𝑛3 ).
Here, the lower bound 𝑓(𝑛) ≥ 𝑐1 𝑛3 for all 𝑛 ≥ 𝑛0 is
violated.
If we divide both sides by 𝑛3 :
8 2 3
lim + 2 − 3 ≥ 𝑐1
𝑛→∞ 𝑛 𝑛 𝑛
The left side tends to 0. The only way to satisfy this is to set c1 = 0. But
by hypothesis, c1 is positive. This means that 𝑓(𝑛) ∉ 𝜃(𝑛3 ).
Sometimes we only interested in proving one bound
upper or lower bound. In that case, O-notation and
Ω-notation is used.