CSE 225 Data Structures and Algorithms: Mirza Mohammad Lutfe Elahi

CSE 225 Data Structures and Algorithms
Mirza Mohammad Lutfe Elahi
Department of Electrical and Computer Engineering

North South University
Fall 2016
Comparison of Algorithms
Chapter 02
Mirza Mohammad Lutfe Elahi (ECE@NSU) C++ Plus Data Structures, Nell Dale (5th) Chapter 02 1 / 36
Comparison of Algorithm?
There is more than one way to solve most problems.
The ways to solve the problem are not same.
But the solutions are functionally correct.
Figure: Direction to Joe’s Diner
Which Cost More to Feed?
Algorithm Efficiency
There are often many approaches (algorithms) to solve a problem.

- How do we choose between them?
At the heart of computer program design are two (sometimes
conflicting) goals to design an algorithm.
Easy to understand, code, debug.
Makes efficient use of the resources.
Goal one is the concern of Software Engineering.
Goal two is the concern of data structures and algorithm analysis.
When goal two is important,
- how do we measure an algorithm’s cost?
How to Measure Efficiency?
Critical resources:
- Time, memory, programmer effort, user effort
Factors affecting running time:
- For most algorithms, running time depends on ”size” of the input.
- Running time is expressed as T (n) for some function T on input
size n.
Analysis of Algorithms
An algorithm is a finite set of precise instructions for performing a

computation or for solving a problem.
What is the goal of analysis of algorithms?
- To compare algorithms mainly in terms of running time but also in
terms of other factors (e.g., memory requirements, programmer’s
effort etc.)
What do we mean by running time analysis?
- Determine how running time increases as the size of the problem
increases.
Input Size
Size of an array.
Polynomial degree.
Number of elements in a matrix.
Number of bits in the binary representation of the input.
Number of vertices and edges in a graph.
Time Analysis
Provides upper and lower bounds of running time.
Lower Bound ≤ Running Time ≤ Upper Bound
Different types of analysis:

Worst Case
Best Case
Average Case
Worst Case
Provides an upper bound on running time.

An absolute guarantee that the algorithm would not run longer, no
matter what the inputs are.
Best Case
Provides an lower bound on running time.

An absolute guarantee that input is the one for which the algorithm
runs the fastest.
Average Case
Provides an estimate of average running time.

Assumes that the input is random.
Useful when best or worst cases do not happen very often.
How Do We Analyze an Algorithm?
Need to define objective measures.

1 Compare execution times?
- Empirical comparison (run programs)
- Not good: times are specific to a particular machine.
2 Count the number of statements?
- Empirical comparison (run programs)
- Not good: number of statements varies with programming language
and programming style.
3 Express running time t as a function of problem size n
- (i.e., t = f (n) )
Rate of Growth ≡ Asymptotic Analysis
Given two algorithms having running times f (n) and g (n), how do we
decide which one is faster?
Compare ”rates of growth” of f (n) and g (n).
Using rate of growth as a measure to compare different functions
implies comparing them asymptotically, i.e., as n → ∞
If f (n) is faster growing than g(x), then f(x) always eventually
becomes larger than g (n) in the limit, i.e., for large enough values of n
The Growth of Functions
A problem that can be solved with polynomial worst-case complexity

is called tractable.
Problems of higher complexity are called intractable.
Problems that no algorithm can solve are called unsolvable.
Asymptotic Notation - Big-O
Figure: g (n) is asymptotically upper bound for f (n)
Asymptotic less than.
O(g (n)) = f (n) : there exist positive constants c and n0 such that
0 ≤ f (n) ≤ cg (n) for all n ≥ n0
Asymptotic Notation - Big-Ω
Figure: g (n) is asymptotically lower bound for f (n)
Asymptotic greater than.
Ω(g (n)) = f (n) : there exist positive constants c and n0 such that
0 ≤ cg (n) ≤ f (n) for all n ≥ n0
Asymptotic Notation - Big-Θ
Figure: g (n) is asymptotically tight bound for f (n)
Asymptotic equality.
Θ(g (n)) = f (n) : there exist positive constants c1 , c2 and n0 such that
0 ≤ c1 g (n) ≤ f (n) ≤ c2 g (n) for all n ≥ n0
Relation between Order of Growth Sets
Growth Functions
Growth Functions
Comparing Growth
N log2 N Nlog2 N N2 2N
1 0 0 1 2
2 1 2 4 4
4 2 8 16 16
8 3 24 64 256
16 4 64 256 65,536
32 5 160 1024 4.29X109
64 6 384 4096 1.84X1019
128 7 896 16,384 3.40X1038
Estimation Techniques
Known as ”back of the envelope” or ”back of the napkin” calculation

Determine the major parameters that effect the problem.
Derive an equation that relates the parameters to the problem.
Select values for the parameters, and apply the equation to yield and
estimated solution.
Estimation Example
Known as ”back of the envelope” or ”back of the napkin” calculation

How many library bookcases does it take to store books totaling one
million pages?
Estimate:
- Pages/inch
- Feet/shelf
- Shelves/bookcase
Understanding the Growth Rate
Consider the example of buying elephants and goldfish:
Cost: (cost of elephants) + (cost of goldfish)
Approximation:
Cost ' cost of elephants
Understanding the Growth Rate
How to Find f (n)?

Associate a ”cost” with each statement.
Find total number of times each statement is executed.
Add up the costs.
The low order terms of a function are relatively insignificant for large
n.
n4 + 100n2 + 10n + 50
Approximately
n4
Highest order term determines rate of growth!
Running Time Examples
Algorithm 1 Cost
arr[0] = 0; c1
arr[1] = 0; c1
arr[2] = 0; c1
...
arr[N-1] = 0; c1
——
c1 + c1 + ... + c1 = c1 ∗ N
Algorithm 2 Cost
for(i = 0; i < N; i++) c2
arr[1] = 0; c1
——
(N + 1) ∗ c2 + N ∗ c1 = (c2 + c1 ) ∗ N + c2
Algorithm Cost
sum = 0; c1
for(i = 0; i < N; i++) c2
for(j = 0; j < N; i++) c2
sum += arr[i][j]; c3
——
c1 + c2 ∗ (N + 1) + c2 ∗ N ∗ (N + 1) + c3 ∗ N ∗ N
= (c2 + c3 ) ∗ N 2 + 2c2 ∗ N + (c1 + c2 )
Big-O Visualization
Algorithm Cost in Big − O

i = 0;
while(i<N)
{
X = X + Y; O(1)
result = mystery(X); O(N)
i++; O(1)
}
Cost of the body of the while loop: O(N)
Loop executed N times
Total cost: N ∗ O(N) = O(N 2 )
Algorithm Cost in Big − O

if(i<j)
for(i = 0; i < N; i++) O(N)
X = X + i;
else
X = 0; O(1)
MAX(O(N), O(1)) = O(N)

What does the following algorithm compute?
i n t max_diff1 ( i n t a [ ] , int n)
{
int m = 0;
f o r ( i n t i = 0 ; i < n ; i++)
f o r ( i n t j = i + 1 ; j < n ; j++)
i f ( abs ( a [ i ] a [ j ]) > m)
m = abs ( a [ i ] a[j]) ;
return m ;
}
It returns the maximum difference between any two numbers in the

input array.
Comparisons:
(n − 1) + (n − 2) + (n − 3) + ... + 1 = (n−1)
n = 0.5n2 − 0.5n
Time Complexity: O(n ) 2
Another algorithm solving the same problem.
i n t max_diff2 ( i n t a [ n ] , i n t n )
{
i n t min = a [ 0 ] ;
i n t max = a [ 0 ] ;
f o r ( i n t i = 1 ; i < n ; i++)
i f ( a [ i ] < min )
min = a [ i ] ;
e l s e i f ( a [ i ] > max )
max = a [ i ] ;
r e t u r n max − min ;
}
Comparisons: 2n − 2
Time Complexity: O(n)
Orders of Magnitude
Listed from slowest to fastest growth:

O(1) Constant time
O(log2 N) Logarithmic time
O(N) Linear time
O(Nlog2 N) N logarithmic time
O(N 2 ) Quadratic time
O(N 3 ) Cubic time
O(2N ) Exponential time
Examples
Rank the following functions by order of growth from the slowest to the
fastest. (notation: lgn = log2 n)
4n2 , 10lgn, 1000, n2 , 2n , 100n, 2lgn
1000 10lgn 2lgn 100n n2 4n2 2n
Slowest =⇒ Fastest
Analyze Complexity of Code Segments
Analyze Complexity of Code Segments

CSE 225 Data Structures and Algorithms: Mirza Mohammad Lutfe Elahi

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CSE 225 Data Structures and Algorithms: Mirza Mohammad Lutfe Elahi

Uploaded by

Copyright:

Available Formats

CSE 225 Data Structures and Algorithms

Mirza Mohammad Lutfe Elahi

Department of Electrical and Computer Engineering

Figure: Direction to Joe’s Diner

Which Cost More to Feed?

There are often many approaches (algorithms) to solve a problem.

How to Measure Efficiency?

An algorithm is a finite set of precise instructions for performing a

Provides upper and lower bounds of running time.

Lower Bound ≤ Running Time ≤ Upper Bound

Different types of analysis:

Provides an upper bound on running time.

Lower Bound ≤ Running Time ≤ Upper Bound

Provides an lower bound on running time.

Lower Bound ≤ Running Time ≤ Upper Bound

Provides an estimate of average running time.

Lower Bound ≤ Running Time ≤ Upper Bound

How Do We Analyze an Algorithm?

Need to define objective measures.

Rate of Growth ≡ Asymptotic Analysis

The Growth of Functions

A problem that can be solved with polynomial worst-case complexity

Asymptotic Notation - Big-O

Figure: g (n) is asymptotically upper bound for f (n)

Asymptotic less than.

Asymptotic Notation - Big-Ω

Figure: g (n) is asymptotically lower bound for f (n)

Asymptotic greater than.

Asymptotic Notation - Big-Θ

Figure: g (n) is asymptotically tight bound for f (n)

Relation between Order of Growth Sets

Known as ”back of the envelope” or ”back of the napkin” calculation

Known as ”back of the envelope” or ”back of the napkin” calculation

Understanding the Growth Rate

Consider the example of buying elephants and goldfish:

Cost: (cost of elephants) + (cost of goldfish)

Cost ' cost of elephants

Understanding the Growth Rate

How to Find f (n)?

Running Time Examples

Running Time Examples

Running Time Examples

Algorithm Cost in Big − O

Running Time Examples

Algorithm Cost in Big − O

MAX(O(N), O(1)) = O(N)

Running Time Examples

It returns the maximum difference between any two numbers in the

Running Time Examples

Another algorithm solving the same problem.

Listed from slowest to fastest growth:

4n2 , 10lgn, 1000, n2 , 2n , 100n, 2lgn

1000 10lgn 2lgn 100n n2 4n2 2n

Analyze Complexity of Code Segments

Analyze Complexity of Code Segments

You might also like