Unit I

DESIGN
DesignAND ANALYSIS
and OF of
Analysis
ALGORITHMS
Algorithms
From,
Ashwini Janagal
Associate Professor
Department of ISE,
JNN College of Engineering,
Shivamogga-577204,
Karnataka,
India.
What is an Algorithm?
I. Introduction: What is an Algorithm
 By Persian Author Abu Ja’far Mohammed ibn al

Khowarizmi.
 DEFINITION:
An ALgorithm is a finite set of instructions that, if followed,
accomplishes a particular tas. In addition, all algorithms must satisfy
the 5 crieteria.
 DEFINITION:
An ALgorithm is a finite set of instructions that, if followed,
accomplishes a particular tas. In addition, all algorithms must satisfy
the 5 crieteria.
1. Input: Zero or More
2. Output : At least One Quantity
3. Definitness: Clear and Unambiguous.
4. Finiteness: Algorithm terminates
5. Effectiveness: Feasible and Basic.
INPUT ALGORITHM OUTPUT
DEFINITENESS FINITENESS EFFECTIVENESS

What are the possible inputs given for the task (0 or more) ?
Example:
Task: To add 2 numbers.
Input: 2 signed integers / 2signed real numbers / 1 integer and one real
number etc.
What are the possible inputs given for the task (0 or more) ?
Example:
Task: To add 2 whole numbers.
Input: 2 signed integers.

Quantity Produced
Example:
Task: To add 2 whole numbers.
OUTPUT-1: Result of Addition
OUTPUT-2 : Error message(if user enters something other than numbers)

Instructions are unambiguous.

Example:
if (a<b)
-----------------
else if (a<b)
--------------
For all the different I/Ps the program is going to end somewhere.
Example:
While(i>=100)
{
printf(“HI WE ARE IN WHILE LOOP\n”);
}
Do you think this programme will terminate?
Instructions are feasible

Study of Algorithm
How to Device algorithm?
How to Validate algorithm?
How to Analyze algorithm?
How to Test algorithm?

Study of Algorithm
There are some methods which

guarantee good algorithms.
1. Can we automate algorithm writing?

Ans: To some extent yes. But there is no
guarantee that it is a good algorithm.
Study of Algorithm
1. I want to organize your blue books according to your

register number. Can you write a algorithm for it?
Study of Algorithm
Process of making sure that algorithm gives

proper output for all possible inputs.
Program. Predicate Calculus

Study of Algorithm
How to Validate algorithm? Analysis of Algorithm/ Performance Analysis

Time Spent on Amount of
CPU. Memory Used
Best Case Performance
Average Case Performance
Worst Case Performance

Study of Algorithm
Example;
Task: Binary Search. (If numbers are not sorted then sort them and then search)
Best: If numbers are already in Ascending Order
Average?
Worst?
Study of Algorithm
Program testing
Debugging Profiling
Study of Algorithm
Program Testing

Debugging Profiling
Makes sure that program provides proper output
Program is run on sample data and output is checked

Study of Algorithm

Program Testing

Debugging Profiling
Program is run on actual data and output correctness, Timing and

Space requirement are studied.
Algorithm Specification
Pseudocode Conventions
1. Comments Begin with //
2. Blocks are indicated with matching braces: { and }
3. Identifier Begins with a Letter.
Data Types are not explicitly Declared.
4. Assignment of Values to Variables is done using Assigment
Statement
<variable>:= <Expression>
5. Two Boolean Values True and False
Three Logical Operators and, or and not.
Relational operators <, ≤,>,≥,= and ≠.
6. Multidimentional array is accessed using A[i , j].

7. Looping Statements
while <condition> do
{
<statement 1>
:
:
<statement n>
}
for variable:=val1 to val2 step step do
{
<statement 1>
:
:
<statement n>
}
Can you convert a for loop to while loop????
Only loop where loop will
repeat
execute as long as condition
is false
<statement 1>
:
:
<statement n>
until <condition>
8. Conditional Statement
 if <condition> then <statement>
 if <condition> then <statement 1> else <statement 2>
 case
{
:<condition1>:<statement 1>
:
:
:<condition n>:<statement n>
:else:<statement n+1>
}
9. Input and output are done using read and write.

10. Procedure
Algorithm Name (<Parameter List>)
Example 1.1: Selection Sort

Logic: From those elements that are currently unsorted, find the smallest
and place it next in the sorted list
for i:=1 to n do
{
Exaxmine a[i] to a[n] and suppose a[k] is the smallest;
Interchange a[i] and a[k];

}

Logic: From those elements thatHow
are to interchange???
currently unsorted, find the smallest
and place it next in the sorted list t:=a[i];
a[i]:=a[j];
a[j]:=temp;
for i:=1 to n do
{
Exaxmine a[i] to a[n] and suppose a[j] is the smallest;
Interchange a[i] and a[j];

}
Theorem 1.1 Algorithm SelectionSort(a,n) correctly sorts

a set of n≥1 elements; the result remains in a[1..n] such
that a[1]≤a[2]≤.....≤a[n].

Algorithm SelectionSort(a,n)
{
for i:=1 to n do
{
k:=i;
for j:=i+1 to n do
if(a[j]<a[k]) then k:=j;
t:=a[i];a[i]:=a[k]; a[k]:=t;
}
}
 Consider the selected lines.

 At some iteration say i=q; q<r≤n; a[1..q] is sorted
after the execution of these statements.
 Following last execution for i-loop (i=n) the above
theorem holds.
Recursive Algorithms
 Recursive Function: A function Defined in terms of

itself.
 Recursive ALgorithm; Same Algorithm is invoked in the
body.
 Direct Recursive Algorithm: An Algorithm which calls
itself.
 Indirect Recursive Algorithm: Algorithm A calls
Algorithm B which in-turn call Algorithm A.
 Any algorithm with following feature can be converted

to a recursion like this
Assignment
Assignment
if-then-else
if-then-else
Recursion
While Loop
 Best Suitable Example-I: Factorial

n!=n×(n-1)×(n-2)........×2×1
 Best Suitable Example-II: Binomial Co-Efficient

 Best Suitable Example-I: Tower of Hanoi

 Move disks from A to C
with B as temporay stop
 At a time one disk
 Always in decreasing order
of their size
 Move (n-1) disks from A to
B with C as temporay stop
 Move nth disk from A to C
 Move (n-1) disks from B to
C with A as temporary
stop.
Algorithm TowersOfHanoi(n,A,C,B)
//TO me Disks from A to C
{
if(n≥1)
{
//n-1 disks from A to B
TowersOfHanoi(n-1,A,B,C);
Move the Disk from A to C
//n-1 disks from B to C
TowersOfHanoi(n-1,B,C,A);
}
}
PERFORMANCE ANALYSIS
DEFINITION: SPACE/TIME COMPLEXITY

The Space Complexity of an algorithm is the amount of
memory it needs to run to completion. The Time
Complexityof an algorithm is the amount of computer time
it needs to run to completion
TIME EFFICIENCY SPACE EFFICIENCY
HOW MUCH
HOW FAST ?
MEMORY IS USED ?
More scope to enhance time Now a days Computers have

efficiency than space efficiency. large amount of memory
But cache memories are quite

small and costly.
Space Complexity
Algorithm abc(a,b,c)
{
Types of Space
return a+b+b*c+(a+b-c)/(a+b)+4.0;
}
1. Fixed Part: Size of data types, Space for code,
Algorithm Sum(a,n)
{ simple variables, fixed size variables etc.
s:=0.0;
2. Variable
for i:=1Part:
to n Size
do varies in the course of
s:=s+a[i];
execution.
return Example
s; may be a pointer variable and
}
Dynamic Memory Allocation.
Algorithm RSum(a,n)
{
if(n≤0) then return 0.0;
else return RSum(a,n-1)+a[n]
}
Space Complexity
{
} Space requirement of an Algorithm P = S(P)=c+Sp
Algorithm Sum(a,n)
c = Constant (Fixed Part)
{
s:=0.0; Part (Instance Characteristics)
Sp=Variable
for i:=1 to n do
s:=s+a[i];
return s;
}
Algorithm RSum(a,n)
{
}
Space Complexity
{
}
Algorithm Sum(a,n)
{
s:=0.0;
for i:=1 to n do
s:=s+a[i];
1. return
Fixed Part:
s; a, b and c need 3 words.
}
2. Variable Part Sp = 0
Algorithm RSum(a,n)
{
}
1. Fixed Part: Array “a” need “n” words
Space Complexity
“n”, “i” and “s” need 3 words.
2. Variable Part Sp= 0
{
Therefore SSum(n)≥ n+3
}
Algorithm Sum(a,n)
{
s:=0.0;
for i:=1 to n do
s:=s+a[i];
return s;
}
Algorithm RSum(a,n)
{
}
PERFORMANCE
 Here we ANALYSIS
need to find the depth of
Space Complexity
Stack....Because when function call is made
arguments are pushed to system Stack.
{  Assumption: Return Address 1 word.
return
 Each a+b+b*c+(a+b-c)/(a+b)+4.0;
time function call is made ‘n’, a[n] and
}
return address are pushed to stack
Algorithm Sum(a,n)
 Depth of recursion is (n+1) (n, n-1, n-2,..., 3,
{
2,1 and 0 also for false conditioning)
s:=0.0;
 SRSum(a,n)≥
for i:=1 to n 3(n+1)
do
s:=s+a[i];
return s;
}
Algorithm RSum(a,n)
{
}
Time Complexity
 Time T(P) -- > Time taken by Program P for compiling
and Execution.
 Once Compiled Program may be executed many times
without compiling.
 --> Just Run(Execution) Time.

Time Complexity
 --> Just Run(Execution) Time.
 A program may have lots of Loads, stores, addition,
subtraction, Multiplication etc.
 n-> Instance Characteristics.
 etc. are time taken for Addition, Subtraction,
multiplication etc.
Time Complexity
 Application of This formula is practically impossible ......
 So sometime we just notedown the time taken for execution.
 Well this also is not accurate as many other programs are running in
your machine and execution time varies on them.
 Other important notion is to COUNT NUMBER OF PROGRAM STEPS.

Time Complexity
 What is PROGRAM STEPS?
 Meaningful segments of program which are independent of execution
instance.
Example:
This whole statement is one step....(Irrespective of

instance)
Time Complexity
 What is PROGRAM STEPS?
 Meaningful segments of program which are independent of execution
step.
Rules:
1. Comments --> 0 Steps;
2. ASsignment (Without any function call) --> 1 step;
3. while (<expr>) has step count of expr.
4. for i:=<expr> to <expr1> do
First entry step count=step count of <expr> + step count of <expr1>
Remaining entry= step count <expr1>
Time Complexity
 Example Algorithm to do step count
Original Code
Algorithm Sum(a,n)
{
s:=0.0;
for i:=1 to n do
s:=s+a[i];
return s;
}
Time Complexity
 Example Algorithm to do step count

Code with Step Count Embedded
Algorithm Sum(a,n)
{
s:=0.0;
count:=count+1;
// count is a global variable so initalized to 0
for i:=1 to n do
{
count := count + 1;
s:=s+a[i];
count:= count +1;
}
count:= count+1;
return s;
count := count +1;
}
Time Complexity
 Example Recursive Algorithm to do step count

Code with Step Count Embedded
Algorithm RSum(a,n)
{
}
Time Complexity

Original Code
Algorithm RSum(a,n)
{
count := count+1 //for if
if(n≤0) then
{
count := count+1; // for return;
return 0.0;
}
else
{
count := count+1;
return RSum(a,n-1)+a[n]
}
}
Time Complexity

:
Original Code
:
Algorithm RSum(a,n)
{
count := count+1 //for if
if(n≤0) then
{
count := count+1; // for return;
return 0.0;
}
else
{
count := count+1;
return RSum(a,n-1)+a[n]
}
}
Time Complexity
 Definition 1.3: [Input Size] One of the instance Characteristics that is
frequently used in literature is “Input Size”.
 The Input size is defined to be the number of words needed to describe
that instance.
 Input Size for problem of Summing an Array of size n elements is n+1. (n
is size of array and 1 for storing the variable.

Time Complexity
 MATRIX ADDITION
Original Code
Algorithm Add(a,b,c,m,n)
{
for i:=1 to m do
for j:=1 to n do
c[i,j]:=a[i,j]+b[i,j];
}
Time Complexity
Original Code + Step CountEmbedded
{
for i:=1 to m do
{
count:=count+1;
for j:=1 to n do
{
count:=count+1;
count:=count+1;
}
count:=count+1;
}
count:=count+1;
}
Time Complexity
{ n times for each value of i
for i:=1 to m do
{ So this for loop is executed m×n times
count:=count+1;
for j:=1 to n do
{
count:=count+1;
count:=count+1;
}
count:=count+1;
}
count:=count+1;
}
 Time Complexity
n times for each value of i
Original CodeSo+ this
Step for loop is executed m×n times
CountEmbedded
 Also we can see two increments in
{ count so 2×m×n
for i:=1 to m do
{
count:=count+1;
for j:=1 to n do
{
count:=count+1;
count:=count+1;
}
count:=count+1;
}
count:=count+1;
}
 Time Complexity
This for loop is executed m times
 Also
Original Code we CountEmbedded
+ Step can see two increments in
count so 2×m
{
for i:=1 to m do
{
count:=count+1;
for j:=1 to n do
{
count:=count+1;
count:=count+1;
}
count:=count+1;
}
count:=count+1;
}
 Time Complexity
{
for i:=1 to m do
{
count:=count+1;
for j:=1 to n do
{
count:=count+1;
Last count of 1
count:=count+1;
}
count:=count+1;
}
count:=count+1;
}
 Time Complexity
{
for i:=1 to m do
{
 Everything will add upto
count:=count+1;
for j:=1 to n do
{
count:=count+1;
count:=count+1;
}
count:=count+1;
}
count:=count+1;
}
Time Complexity

{
 If m is greater than n then we
for i:=1 to m do
{
better interchange it.
count:=count+1;
for j:=1 to n do
{
count:=count+1;
count:=count+1;
}
count:=count+1;
}
count:=count+1;
}
 Time Complexity

{
 If m is greater than n then we
for i:=1 to m do
{
better interchange it.
count:=count+1;
for j:=1 to n do
{
count:=count+1;
 Input size is 2mn+2 (2 matrices of
count:=count+1;
size m, n and two words to store m
}
count:=count+1;
and n.
}
count:=count+1;
}
 Time Complexity
Second method of finding step count is by building table.
 Number of Steps per Execution(s/e) of the statement.
 Number of times the statement is executed.

 Time Complexity
Example :1
Statements s/e Frequency Total
Steps
Algorithm Sum(a,n) 0 0
{ 0 0
1
s:=0.0; 1 1
n+1
for i:=1 to n do 1 n+1
n
s:=s+a[i]; 1 n
1
return s; 1 1
} 0 0
2n+3
 Time Complexity : Example
x=
Statements s/e Frequency Total
n=0 n>0 Steps
Algorithm RSum(a,n) 0 0 0
{ 0
1 1
if(n≤0) then 1 1 1
1 0
return 0.0; 1 1 0
else 0
0 1
return RSum(a,n-1)+a[n]; 1+x 0 1+x
2 2+x
 Time Complexity
Fibonocci series (0,1,1,2,3,5,8....)
Algorithm Fibonocci(n)
{
if(n≤1) then
write(n);
else
{
fnm2:=0; fnm1:=1;
for i:=2 to n do
{
fn:=fnm1+fnm2;
fnm2:=fnm1; fnm1:=fn;
}
write(fn);
}
}
1+1--> For loop Entry
Statements s/e Frequency Total Steps
n=0/1 n>1 n=0/1 n>1
Algorithm Fibonocci(n) 0 0 0
{
if(n≤1) then 1 1 0 1 0
write (n); 1 1 0 1 0
else
{
fnm2:=0; fnm1:=1; 2 0 1 0 2
for i:=2 to n do 1 n 0 n
0
{
fn:=fnm1+fnm2; 1 0 n-1 0 n-1
fnm2:=fnm1; fnm1:=fn; 2 n-1 0 2(n-1)
0
}
write (fn); 1 0 1 0 1+1
}
}
Statements = s/e Frequency Total Steps
= n=0/1 n>1 n=0/1 n>1
Algorithm Fibonocci(n) 0 0 0
{
if(n≤1) then 1 1 0 1 0
write (n); 1 1 0 1 0
else
{
fnm2:=0; fnm1:=1; 2 0 1 0 2
for i:=2 to n do 1 n 0 n
0
{
fn:=fnm1+fnm2; 1 0 n-1 0 n-1
fnm2:=fnm1; fnm1:=fn; 2 n-1 0 2(n-1)
0
}
write (fn); 1 0 1 0 1+1
}
}
2
2+n+n-
1+2n-2+1+1
Efficiency
PERFORMANCE ANALYSIS Framework
 Units for Measuring Running Time
FIRST METHOD: Whats the problem with using same

Time as we measure (Second, minute etc.)?
Well it depends on hardware, speed of computer, compilers used etc.,

which varies from computer to computer
So, What is the next Option
Count Each Operation That is Performed
Well that's very large and tedious job

Efficiency
What next?
Find “BASIC OPERATIONS” in program
Count how many times they are executed.

Efficiency
So what is “BASIC
OPERATION”
Efficiency
So what is “BASIC
OPERATION”?
Most Time Consuming Part of the Algorithms Innermost Loop

Efficiency
So what is “BASIC
OPERATION”? in SORTING
the array
Efficiency
So what is “BASIC
OPERATION”? in SORTING
the array
So what is “BASIC
OPERATION”? in MATRIX
MULTIPLICATION
Efficiency
So how do you count them?
It depends on input size 'n'

Efficiency
for(int i = 0; i < n; i++)

{
So what is the count in if (values[i] == target)
LINEAR SEARCH {
return i;
}
}
BEST CASE? AVEAGE CASE? WORST CASE?
ARRAY
10 15 20 80 44 64 35 67
Efficiency
If = Time taken for one Basic Step
If = Number of Basic Operations
Running Time
APPROXIMATELY!!!! (Because we are considering only basic operation)

Efficiency Framework
1
If
t ( n )  n ( n  1)
2
and we double the input size then compare t(n)
1
t ( n )  n ( n  1)
2
1 2 1
t (n)  n  n
2 2
If 1
t ( n )  n ( n  1)
2
and we double the input size then compare
t(n)
1
t ( n )  n ( n  1)
2
1 2 1 VERY SMALL
t (n)  n  n
2 2
If 1
t ( n )  n ( n  1)
2
and we double the input size then compare
t(n)
1
t ( n )  n ( n  1)
2
1 2 1 1 2
t (n)  n  n  n
2 2 2
 Units for Measuring Running Time NOW WE
COMPARE
If 1
t ( n )  n ( n  1)
2
1 2 1 2
( 2n ) 4n
t ( 2n ) 2 2
  4
t (n) 1 2 1 2
n n
2 2
 Units for Measuring Running Time NOW WE
COMPARE
If 1
t ( n )  n ( n  1)
2
1 2 1 2
( 2n ) 4n
t ( 2n ) 2 2
  4
t (n) 1 2 1 2
n n
2 2 Time taken
is 4 times
more
Asymptotic Notation
Asymptotic Growth
Asymptotic Notation for Growth in Algorithms
running time Vs. Input Size
Order of Growth
a set of functions whose asymptotic growth behavior
is considered equivalent
Ex: 2n, 200n, (n+1) all have same order of

growth
We ignore constant multiples. (For large
I/P it doesn't matter).
Order of Growth
Function Which Grows SLOW : log
logan=logab+logbn --> We simply write log n

(Without base)
Order of Growth
Function Which Grows FAST : 2n and n!

2n and n! --> Exponential Growth Functions
2100 Needs --> If we have a computer capable of
executing 1012 instr. per second then we need 1010
years in that machine to calculate 2100
Order of Growth
Should we not write algorithm which take

exponential growth????
You CAN.... But for smaller inputs.
Compare Growth of log n if input is double.

Input is double --> now we take (2n)
t(n)= log(2n) = log22+log2n = 1 + log2n
Increases by 1
Order of Growth
Compare Growth of n logn if input is double.
t(n)= 2nlog(2n)= 2n(log22+log2n)

= 2n(1+log2n)
= 2n+2nlog2n
Increase
Order of Growth
Compare Growth of n2 if input is double.

t(n)= n2 = (2n)2 =4n2
4 times more
Compare Growth of n3 logn if input is double.
t(n)= n3 = (2n)3 = 8n3

8 times more
Worst-Case, Best-Case, and Average-Case Efficiencies
ALGORITHM SequentialSearch(A[O .. n - 1], K)

i <- 0
while i < n and do
i <- i+1
if i < n return i
else return -1
NOTES OF SEQUENTIAL SEARCH


i <- 0
while i < n and do
i <- i+1
if i < n return i
else return -1
So, How many times While Loop was Executed?


i <- 0
while i < n and do
i <- i+1
if i < n return i
else return -1
So, How many times While Loop was Executed?
For the given Example it was 5???


i <- 0
while i < n and do
i <- i+1
if i < n return i
else return -1
What is the Best Case?
What is the Best Case?


i <- 0
while i < n and do
i <- i+1
if i < n return i
else return -1
 What is the Best Case? -- CBest(n)
 What is the Worst Case? -- CWorst(n)


i <- 0
while i < n and do
i <- i+1
if i < n return i
else return -1
 What is the Best Case? -- CBest(n)
 If key was in ‘0’th Position
 “while loop” executed once - So CBest(n) = 1


i <- 0
while i < n and do
i <- i+1
if i < n return i
else return -1
 What is the Worst Case? -- CWorst(n)
 If key was in Last (n)th Position
 “while loop” executed ‘n’ times -- So CWorst(n) = n


i <- 0
while i < n and do
i <- i+1
if i < n return i
else return -1
 What is the AVERAGE CASE?

 If p = probability of Successful Search (0 ≤ p ≤ 1)

 probability of the first match occurring in the ith
position of the list is the same for every i.

 probability of the first match occurring in the ith
position of the list is the same for every i.
 From the above statement, probability of searched
element in 'i' th position is p/n

 So if served element is in 1st position -> 1× p/n (while

loop once)
 if served element is in 2nd position -> 2× p/n (while loop
twice)
 Successful (p=1)
 Unsuccessful (p=0)
 Amotized Efficiency
 Worst case in single run but when mixed with
other operation and run together give better
performance.
Asymptotic Notations and Basic Efficiency Classes
To compare the order of growth

O (Big Oh)
Ω (Big Omega)
Ө (Big Theta)

O (Big Oh)
Ω(Big
Important
Omega) Notations
t(n) --> Function for Algorithm Running time.
Ө (Big Theta)
C(n) --> t(n) is usually represented in C(n) (Counter of step)
g(n) --> Some simple function to compare with.

O (Big Oh)
Ω (Big Omega)
Ө (Big Theta)
Asymptotic Notations and
Basic Efficiency Classes
Informal Introduction (Big Oh)
 is the set of all functions with a SMALLER or SAME order of growth as

g(n)
 Examples
n  O(n 2 ) n 3  O(n 2 )
100n  5  O ( n 2 ) 0.00001n 3  O ( n 2 )
1
n ( n  1)  O ( n 2 ) n 4  n  1  O(n 2 )
2
Informal Introduction Ω (Big Omega)
Ω(g(n)) is the set of all functions with a LARGER or SAME ORDER of

growth as g(n)
 Examples
n 3  ( n 2 )
1
2
n ( n  1)  ( n 2 ) 100n  5  ( n ) 2
n 4  n  1  ( n 2 )
Informal Introduction (Big Theta)
 is the set of all functions with a SAME order of growth as g(n)
 Examples
an  bn  c   ( n ) Ifa  0
2 2
n  sin n   ( n )
2 2
Definition:
A function t(n) is said to be in O(g(n)), denoted
if t(n) is bounded above by some constant multiple of g(n) for all large n, i.e.,
if there exist some positive constant c and some non negative integer n0 such
that
Example: Remember
Example: Remember
Definition:
A function t(n) is said to be in Ω(g(n)), denoted
if t(n) is bounded above by some constant multiple of g(n) for all large n, i.e.,
if there exist some positive constant c and some non negative integer n0 such
that
Example: Remember
Example: Remember
Definition:
A function t(n) is said to be in Ө(g(n)), denoted
if t(n) is bounded both below and above by some constant multiple of g(n) for
all large n, i.e., if there exist some positive constants c1 and c2 and some non
negative integer n0 such that

Example: Remember
Prove
Upper Bound =
Lower Bound =
So we choose
Example: Remember 𝑐 2 𝑔(𝑛)≤ 𝑡(𝑛)≤ 𝑐1 𝑔 ¿

 Useful Property Involving the Asymptotic Notations
Theorem:
If
(The analogous assertions are true for the notations as well.)

Theorem:
If
Step 1: If we take arbitrary 4 real numbers like a1, b1, a2 and b2.
And if , Then a1+a2 ≤ 2max{b1,b2}
Step 2: Since
Similarly
Step 3:
Theorem:
If
Step 4:
Theorem:
If
Step 5:
Therefore,
Constant c=
Asymptotic Notations and Basic
Asymptotic Efficiency
Notations and Classes
lim =
n−>∞ g(n)
{
t(n) 0Impliesthat (n)has Smal erOrderof growthcompared
¿
g(n)¿c>0Impliesthat (n)has Sameorder of growthcompared¿g(n)¿∞Impliesthat (n)has Larger order of growthcompared¿g(n)¿ }
{ }
𝑡 (𝑛) 0 𝐼𝑚𝑝𝑙𝑖𝑒𝑠𝑡ℎ𝑎𝑡 𝑡 (𝑛)∈ 𝑂 (𝑔 (𝑛))
lim = 𝑐 >0 𝐼𝑚𝑝𝑙𝑖𝑒𝑠 𝑡ℎ𝑎𝑡 𝑡 (𝑛)∈ 𝜃 (𝑔 (𝑛))
𝑛−>∞ 𝑔(𝑛)
∞ 𝐼𝑚𝑝𝑙𝑖𝑒𝑠 𝑡ℎ𝑎𝑡 𝑡 ( 𝑛) ∈ 𝛺 (𝑔 (𝑛))
 Using Limits for Comparing Order of Growth
{ }
t (n) 0 Implies that t (n) ∈O ( g (n))
lim = c > 0 Implies that t ( n) ∈ θ( g (n))
n −>∞ g (n)
∞ Implies that t ( n)∈ Ω ( g (n))
EXAMPLE 1: Compare the orders of growth of n(n - 1) and n2.
= ==
A Constant. Therefore
{ }
n −>∞ g (n)
EXAMPLE 2 Compare the orders of growth of log2nand
L'Hopital's rule =
= = 2 =0 log2n
{ }
n −>∞ g (n)
EXAMPLE 2 Compare the orders of growth of n! and 2.
Stirlings Formula says n! ≈
= == =
n!
 Basic Efficiency Classes
Asymptotic
Mathematical Analysis Notations and
of Non-Recursive Basic Efficiency Classes
Algorithms
Ex1: Finding the value of the largest element in a list of ‘n’ numbers
ALGORITHM MaxElernent(A[O .. n -1])

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
A 12 34 23 67 45 16 26 55 1
i
Maxval
n 9
Asymptotic
Algorithms

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4 5 6 7 8
A 12 34 23 67 45 16 26 55 1
i
Maxval 12
n 9
Mathematical Analysis of Non-Recursive Algorithms

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4 5 6 7 8
A 12 34 23 67 45 16 26 55 1
i 1
Maxval 12
n 9
Asymptotic
Algorithms

maxval <-- A[0]
A[i=1] > maxval for i<--1 to n-1 do
34 > 12 if A[i]>maxval
True
maxval<-- A[i]
return maxval
0 1 2 3 4 5 6 7 8
A 12 34 23 67 45 16 26 55 1
i 1
Maxval 12
n 9
Asymptotic
Algorithms
ALGORITHM MaxElernent(A[O .. n -
1])
maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 1
Maxval 12 34
n 9

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 12
Maxval 12 34
n 9

maxval <-- A[0]
A[i=2] > maxval
23 > 34 for i<--1 to n-1 do
FALSE if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 12
Maxval 12 34
n 9

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 123
Maxval 12 34
n 5

maxval <-- A[0]
A[i=3] > maxval
67 > 34 for i<--1 to n-1 do
TRUE if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 123
Maxval 12 34
n 5

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 123
Maxval 12 34 67
n 5

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 123 4
Maxval 12 34 67
n 5

maxval <-- A[0]
A[i=4] > maxval
45 > 67 for i<--1 to n-1 do
FALSE if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 123 4
Maxval 12 34 67
n 5

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 12345
Maxval 12 34 67
n 5

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
0 1 2 3 4
A 12 34 23 67 45
i 12345
Maxval 12 34 67
n 5

Heart of this maxval <-- A[0]
program for i<--1 to n-1 do
if A[i]>maxval
Comparison is maxval<-- A[i]
performed every
time return maxval
0 1 2 3 4
A 12 34 23 67 45
i 12345
Maxval 12 34 67
n 5
Asymptotic
Algorithms

Heart of this maxval <-- A[0]
program for i<--1 to n-1 do
if A[i]>maxval
Comparison is maxval<-- A[i]
performed every
time return maxval
0 1 2 3 4
Therefore we A 12 34 23 67 45
consider comparison i 12345
as “basic step”
Maxval 12 34 67
n 5

maxval <-- A[0]
for i<--1 to n-1 do
if A[i]>maxval
maxval<-- A[i]
return maxval
 c(n) The number of times comparison is executed

 c(n) = = n-1 = (n)
SOME GENERAL PLANS
 Decide on a parameter (or parameters) indicating an input's

size.
 Identify the algorithm's basic operation. (As a rule, it is located
in its innermost loop.)
 Check whether the number of times the basic operation is
executed depends only on the size of an input.
 It may also depend on some additional property,
SOME GENERAL PLANS
 The worst-case, average-case, and, if necessary, best-case

efficiencies have to be investigated separately.
 Set up a sum expressing the number of times the algorithm's
basic operation is executed.
 Using standard formulas and rules of sum manipulation,
either find a closed form formula for the count or, at the very
least, establish its order of growth.
SOME GENERAL PLANS

SOME GENERAL PLANS

Ex1: Element Uniqueness Problem
ALGORITHM UniqueElement(A[O .. n -1])

for i←0 to n-2 do
for j←i+1 to n-1 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i
j
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 0
j
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 0
j 1
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false A[0](12)=A[1](34)
return true FALSE
0 1 2 3 4
A 12 34 23 67 23
i 0
j 1
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 0
j 12
n 5

for i←0 to n-2 do
if A[i]=A[j] A[0](12)=A[2](23)
return false FALSE
return true
0 1 2 3 4
A 12 34 23 67 23
i 0
j 12
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 0
j 1 2 3
n 5

for i←0 to n-2 do
if A[i]=A[j]
return true FALSE
0 1 2 3 4
A 12 34 23 67 23
i 0
j 123
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 0
j 1 23 4
n 5

for i←0 to n-2 do
if A[i]=A[j]
FALSE
return true
0 1 2 3 4
A 12 34 23 67 23
i 0
j 1 23 4
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 0
j 1 234 5
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 01
j 1 234 5
n 5
Asymptotic
Algorithms

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 01
j 1 2345 2
n 5
Asymptotic
Algorithms

for i←0 to n-2 do
if A[i]=A[j] A[1](34)=A[2](23)
return false FALSE
return true
0 1 2 3 4
A 12 34 23 67 23
i 01
j 1 2345 2
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 01
j 1 2345 23
n 5

for i←0 to n-2 do
if A[i]=A[j]
FALSE
return true
0 1 2 3 4
A 12 34 23 67 23
i 01
j 1 2345 23
n 5
Asymptotic
Algorithms

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 01
j 1 2345 23 4
n 5

for i←0 to n-2 do
if A[i]=A[j]
return true FALSE
0 1 2 3 4
A 12 34 23 67 23
i 01
j 1 2345 23 4
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 01
j 1 2345 234 5
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 01 2
j 1 2345 234 5
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 01 2
j 1 2345 2345 3
n 5

for i←0 to n-2 do
if A[i]=A[j]
A[2](23)=A[3](67)
return false
FALSE
return true
0 1 2 3 4
A 12 34 23 67 23
i 01 2
j 1 2345 2345 3
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 01 2
j 1 2345 2345 3
4
n 5

for i←0 to n-2 do
if A[i]=A[j]
TRUE
return true
0 1 2 3 4
A 12 34 23 67 23
i 01 2
j 1 2345 2345 3
4
n 5

for i←0 to n-2 do
if A[i]=A[j]
return false
return true
0 1 2 3 4
A 12 34 23 67 23
i 01 2
j 1 2345 2345 3
4
n 5
Asymptotic
Algorithms

for i←0 to n-2 do
We consider if A[i]=A[j]
comparison as “basic
return false
step”
return true
 Best case = First two elements are same

 i=0, j=1 and comparison=true

for i←0 to n-2 do
We consider if A[i]=A[j]
comparison as “basic
return false
step”
return true
 Worst Case: Last two elements are same

or all elements are unique

for i←0 to n-2 do
i=0 to n-2
if A[i]=A[j]
j=i+1 to n-1 return false
return true
 cworst(n) =
 cworst(n) =
=
 cworst(n) = =
n-1 = n-1
n-1
(n-1)(n-2-0+1)-
𝑛 −2
𝐶 𝑊𝑜𝑟𝑠𝑡 (𝑛)=(𝑛 −1)(𝑛 −1 − )
2
𝑛 −2
𝐶 𝑊𝑜𝑟𝑠𝑡 (𝑛)=(𝑛 −1)(𝑛 −1 − )
2
𝐶 𝑊𝑜𝑟𝑠𝑡 (𝑛)=(𝑛 −1)

(𝑛 −1) 2 𝑛 −2
2
−
2 [ ]
2𝑛 − 2− 𝑛+2
2 [ ]

2𝑛 − 2− 𝑛+2
2 [ ]
2𝑛 − 2− 𝑛+2
2 [ ]
𝐶 𝑊𝑜𝑟𝑠𝑡 (𝑛)=(𝑛 − 1 )
[ ]
𝑛
2
𝐶 𝑊𝑜𝑟𝑠𝑡 (𝑛)=(𝑛 − 1 )
[ ]
𝑛
2
𝑛 (𝑛− 1)
𝐶 𝑊𝑜𝑟𝑠𝑡 (𝑛)=
2
=
Mathematical Analysis of Non-Recursive Algorithms A 0 1
0 11 12
Ex1: MATRIX MULTIPLICATION 1 13 14
ALGORITHM MatrixMultiplication(A[O .. n- 1, O ..
n- 1],
00 01 10 11
B[O .. n- 1, O .. n- 1])
A 11 12 13 14
for ito n-1 do B
C
for j
i
c[i,j]=0
j
for k
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
B 0 1
0 15 16
Ex1: MATRIX MULTIPLICATION
1 17 18
n- 1],
00 01 10 11
B[O .. n- 1, O .. n- 1]) A 11 12 13 14
for ito n-1 do B 15 16 17 18

C
for j
i
c[i,j]=0 j
for k
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
A 0 1
Mathematical Analysis of Non-Recursive Algorithms0 11 12

1 13 14
B 0 1
0 15 16
n- 1],
1 17 18
B[O .. n- 1, O .. n- 1])
00 01 10 11
for ito n-1 do
A 11 12 13 14
for j B 15 16 17 18
c[i,j]=0 C
i
for
j
C[i, j]+= C[i, j]+ A[i, k]*B[k, j] k
return c n 2
A 0 1

1 13 14
Ex1: MATRIX MULTIPLICATION B 0 1

0 15 16
1 17 18
ALGORITHM MatrixMultiplication(A[O .. n- 1, O .. n-
1], 00 01 10 11
B[O .. n- 1, O .. n- 1]) A 11 12 13 14
B 15 16 17 18
for ito n-1 do
C
for j i 0
c[i,j]=0 j
k
for
n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
A 0 1

1 13 14
Ex1: MATRIX MULTIPLICATION B 0 1

0 15 16
1 17 18
ALGORITHM MatrixMultiplication(A[O .. n- 1, O .. n- 1],
B[O .. n- 1, O .. n- 1]) 00 01 10 11
A 11 12 13 14
for ito n-1 do
B 15 16 17 18
for j C
c[i,j]=0 i 0
j 0
for
k
C[i, j]+= C[i, j]+ A[i, k]*B[k,
n j]
2
return c
Mathematical Analysis A Algorithms
of Non-Recursive 0 1 B 0 1
C 0 1
0 11 12 0 15 16
0 0
Ex1: 1MATRIX MULTIPLICATION 1 13 14 1 17 18

00 01 10 11
B[O .. n- 1, O .. n- 1])
A 11 12 13 14
for ito n-1 do
B 15 16 17 18
for j C 0
c[i,j]=0 i 0
j 0
for
k
C[i, j]+= C[i, j]+ A[i, k]*B[k,
n j]
2
return c
C 0 1 A 0 1 B 0 1
Mathematical
0 0 Analysis of Non-Recursive Algorithms
0 11 12 0 15 16
1 1 13 14 1 17 18
00 01 10 11
A 11 12 13 14
ALGORITHM MatrixMultiplication(A[O .. n- 1, O ..15
n- 1],
B 16 17 18
B[O .. n- 1, O .. n- 1]) C 0
for ito n-1 do i 0

j 0
for j
k 0
c[i,j]=0 n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
A 0 1
B 0 1
C 0 1 0 and
11 Basic
12 Efficiency Classes
Mathematical Analysis ofAsymptotic Notations
Non-Recursive Algorithms 0 15 16
0 0 1 13 14
1 17 18
1
ALGORITHM MatrixMultiplication(A[O .. n- 1, O ..00 01

n- 1], 10 11
A 11 12 13 14
B[O .. n- 1, O .. n- 1])
B 15 16 17 18
forC[0,0]=C[0][0]+A[0][0]*B[0
ito n-1 do 0] C 165
C[0,0]=0+11*15=165
for j i 0
j 0
c[i,j]=0
k 0
for
n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
A 0 1 B 0 1
C 0
Mathematical 1
Analysis of Non-Recursive Algorithms
0 11 12 0 15 16
0 165
1 13 14 1 17 18
1
ALGORITHM MatrixMultiplication(A[O .. n- 1, O ..00 01

n- 1], 10 11
A 11 12 13 14
B[O .. n- 1, O .. n- 1])
B 15 16 17 18
for ito n-1 do C 0
for j i 0
j 0
c[i,j]=0
k 01
for
n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1 A 0 1
Mathematical Analysis of Non-Recursive Algorithms B 0 1
0 165 0 11 12
0 15 16
1 1 13 14
00 01 10 11
A 11 12 13 14
B 15 16 17 18
B[O .. n- 1, O .. n- 1])
C 369
forC[0,0]+=C[0]
ito n-1 [0]+A[0]
do [1]*B[1] [0]
i 0
C[0,0]+=165+12*17=369
for j j 0
k 1
c[i,j]=0
n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C
Mathematical 0 1 of Non-Recursive Algorithms
Analysis A 0 1 B 0 1
0 369 0 11 12 0 15 16
1
Ex1: MATRIX MULTIPLICATION 1 13 14 1 17 18
00 01 10 11
ALGORITHM MatrixMultiplication(A[O .. n- 1, O .. 11n- 1],
A 12 13 14
B[O .. n- 1, O .. n- 1]) B 15 16 17 18
for ito n-1 do C 369

i 0
for j
j 0
c[i,j]=0 k 01 2
for n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
A 0 1
C 0 1 B 0 1
Mathematical Analysis of Non-Recursive0 Algorithms
11 12
0 369 0 15 16
1 13 14
1 1 17 18
1], 00 01 10 11
A 11 12 13 14
B[O .. n- 1, O .. n- 1])
B 15 16 17 18
for ito n-1 do
C 369
for j i 0
c[i,j]=0 j 01
k 01 2
for
n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
A 0 1 B 0 1
C 0
Mathematical 1
0 11 12 0 15 16
0 369 0
1 13 14 1 17 18
Ex1:1 MATRIX MULTIPLICATION
00 01 10 11
ALGORITHM MatrixMultiplication(A[O .. n- 1, AO .. 11n- 12 13 14
1], B 15 16 17 18
B[O .. n- 1, O .. n- 1]) C 0
i 0
for ito n-1 do
j 01
for j k 01 2
c[i,j]=0 n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
A 0 1 B 0 1
C 0
Mathematical 1
0 11 12 0 15 16
0 369 0
1 13 14 1 17 18
Ex1:1 MATRIX MULTIPLICATION
00 01 10 11
A 11 12 13 14
B 15 16 17 18
B[O .. n- 1, O .. n- 1])
C 0
for ito n-1 do i 0
for j j 01
k 01 2 0
c[i,j]=0
n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
B 0 1
C 0 1 A 0 1
Mathematical Analysis of Non-Recursive Algorithms 0 15 16
0 369 0 0 11 12
1 17 18
1 1 13 14
00 01 10 11
A 11 12 13 14
B[O .. n- 1, O .. n- 1])
B 15 16 17 18
forC[0,1]+=C[0]
ito n-1 do[1]+A[0] [0]*B[0] [1] C 369 176
C[0,1]+=0+11*16=176
for j i 0
j 01
c[i,j]=0
k 01 2 0
for n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1
A 0
Mathematical Analysis of Non-Recursive Algorithms 1 B 0 1
0 369 176
0 11 12 0 15 16
1
00 n- 01
ALGORITHM MatrixMultiplication(A[O .. n- 1, O .. 1], 10 11
A 11 12 13 14
B[O .. n- 1, O .. n- 1])
B 15 16 17 18
for ito n-1 do C 369 176
for j i 0
j 01
c[i,j]=0
k 01 2 01
for
n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1 A 0 1 B 0 1
Mathematical
0 369 176 Analysis of Non-Recursive
0 11 Algorithms
12 0 15 16
1 1 13 14 1 17 18
00 01 10 11
A 11 12 13 14
B 15 16 17 18
B[O .. n- 1, O .. n- 1])
C 369 392
forC[0,1]+=C[0][1]+A[0][1]*B[1][1]
ito n-1 do i 0
C[0,1]+=176+12*18=392
for j j 01
k 01 2 01
c[i,j]=0
n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1
A
Mathematical Analysis of Non-Recursive 0 1
Algorithms B 0 1
0 369 392
0 11 12 0 15 16
1
00 01 10 11
ALGORITHM MatrixMultiplication(A[O .. n- 1,AO ..11n- 1],
12 13 14
B 15 16 17 18
B[O .. n- 1, O .. n- 1])
C 369 392
for ito n-1 do
i 0
for j j 01
c[i,j]=0 k 01 2 01 2
n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1 A 0 1 B 0 1
0 369 392 0 11 12 0 15 16
1 1 13 14 1 17 18
B[O .. n- 1, O .. n- 1])
00 01 10 11
for ito n-1 do
A 11 12 13 14
for j
B 15 16 17 18
c[i,j]=0 C 369 392
i 0
for
j 01 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
k 01 2 01 2
return c n 2
C 0
Mathematical1Analysis of Non-Recursive
A 0 Algorithms
1 B 0 1
0 369 392 0 11 12 0 15 16
Ex1:1 MATRIX MULTIPLICATION1 13 14 1 17 18
00 n- 01
ALGORITHM MatrixMultiplication(A[O .. n- 1, O .. 1], 10 11
A 11 12 13 14
B[O .. n- 1, O .. n- 1])
B 15 16 17 18
for j i 01
j 01 2
c[i,j]=0
k 01 2 01 2
for
n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1
Mathematical A Algorithms
0 369 392Analysis of Non-Recursive 0 1 B 0 1
1 0 11 12 0 15 16
ALGORITHM MatrixMultiplication(A[O .. n- 1, O00.. n-

01 1],10 11
A 11 12 13 14
B[O .. n- 1, O .. n- 1])
B 15 16 17 18
for j i 01
j 01 2 0
c[i,j]=0
k 01 2 01 2
for
n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1 A 0 1 B 0 1
Mathematical
0 369 392
Analysis of Non-Recursive
0 11 12
Algorithms
0 15 16
1 0 1 13 14 1 17 18
00 01 10 11
A 11 12 13 14
B 15 16 17 18
B[O .. n- 1, O .. n- 1])
C 369 392 0
for ito n-1 do i 01
for j j 012 0
k 012 01 2
c[i,j]=0
n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1
A 0 1 B 0 1
Mathematical
0 369 392 Analysis of Non-Recursive Algorithms
0 11 12 0 15 16
1 0
1 13 14 1 17 18
00 01 10 11
A 11 12 13 14
ALGORITHM MatrixMultiplication(A[O .. n- 1,15O ..16n- 1],
B 17 18
B[O .. n- 1, O .. n- 1]) C 369 392 0
for ito n-1 do i 01

j 012 0
for j
k 012 012 0
c[i,j]=0 n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1 A 0 1 B 0 1
Mathematical
0 369 392 Analysis of Non-Recursive
0 Algorithms
11 12 0 15 16
1 0 1 13 14 1 17 18
B[O .. n- 1, O .. n- 1])
00 01 10 11
forC[1,0]+=C[1][0]+A[1][0]*B[0][0]
ito n-1 do
A 11 12 13 14
for j
C[1,0]+=0+13*15=195 B 15 16 17 18
c[i,j]=0 C 369 392 195
i 01
for
j 012 0
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
k 012 012 0
return c n 2
C 0 1 A 0 1
Mathematical Analysis of Non-Recursive Algorithms B 0 1
0 369 392 0 11 12
0 15 16
1 195 1 13 14
B[O .. n- 1, O .. n- 1])
00 01 10 11
for ito n-1 do
A 11 12 13 14
for j
B 15 16 17 18
c[i,j]=0 C 369 392 195
i 01
for
j 012 0
C[i, j]+= C[i, j]+ A[i, k]*B[k, 012
j]
k 012 01
return c n 2
C 0 1 A 0 1 B 0 1
Mathematical Analysis of Non-Recursive
0 Algorithms
11 12
0 369 392 0 15 16
1 195 1 13 14 1 17 18
B[O .. n- 1, O .. n- 1])
00 01 10 11
forC[1,0]+=C[1][0]+A[1][1]*B[1][0]
ito n-1 do
A 11 12 13 14
for j
C[1,0]+=195+14*17=433 B 15 16 17 18
c[i,j]=0 C 369 392 433
i 01
for
j 012 0
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
k 012 012 01
return c n 2
C 0 1 A 0 1 B 0 1
0 369 392 0 11 12 0 15 16
1 433 1 13 14 1 17 18
B[O .. n- 1, O .. n- 1])
for ito n-1 do 00 01 10 11

A 11 12 13 14
for j
B 15 16 17 18
c[i,j]=0 C 369 392 433
for i 01
j 012 0
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
k 012 012 01 2
return c n 2
C 0 1 A 0 1 B 0 1
0 369 392 0 11 12 0 15 16
1 433 1 13 14 1 17 18
00 01 10 11
A 11 12 13 14
B 15 16 17 18
B[O .. n- 1, O .. n- 1])
C 369 392 433
for ito n-1 do i 01
for j j 012 01
k 012 012 012
c[i,j]=0
n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1 A 0 1
B
Mathematical Analysis of Non-Recursive Algorithms 0 1
0 369 392 0 11 12
0 15 16
1 433 0 1 13 14
00 01 10 11
ALGORITHM MatrixMultiplication(A[O .. n- A
1, O 11
.. n-121], 13 14
B[O .. n- 1, O .. n- 1]) B 15 16 17 18
for ito n-1 do C 369 392 433 0

i 01
for j
j 012 01
c[i,j]=0 k 012 012 012
for n 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1 A 0 1 B 0 1
0 369 392 0 11 12 0 15 16
1 433 0 1 13 14 1 17 18
B[O .. n- 1, O .. n- 1])
for ito n-1 do 00 01 10 11

A 11 12 13 14
for j
B 15 16 17 18
c[i,j]=0 C 369 392 433 0
for i 01
j 012 01
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
k 012 012 012 0
return c n 2
C 0 1
Mathematical Analysis of Non-Recursive
A 0 1
Algorithms
B 0 1
0 369 392
1 433 0 0 11 12 0 15 16
1 13 14 1 17 18
B[O .. n- 1, O .. n- 1])
forC[1,1]+=C[1][1]+A[1][0]*B[0][1]
ito n-1 do 00 01 10 11
A 11 12 13 14
for j
C[1,1]+=0+13*16=208 B 15 16 17 18
c[i,j]=0 C 369 392 433 208
for i 01
j 012 01
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
k 012 012 012 0
return c n 2
C 0
Mathematical 1Analysis of Non-Recursive Algorithms
A 0 1 B 0 1
0 369 392
0 11 12 0 15 16
Ex1:1MATRIX
433 208
MULTIPLICATION
1 13 14 1 17 18
B[O .. n- 1, O .. n- 1]) 00 01 10 11
for ito n-1 do A 11 12 13 14

B 15 16 17 18
for j
C 369 392 433 208
c[i,j]=0 i 01
for j 012 01
k 012 012 012 01
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
n 2
return c
C 0 1 B 0 1
Mathematical Analysis of ANon-Recursive
0 1 Algorithms
0 369 392 0 15 16
0 11 12
1 433 208 1 17 18
1 13 14
00 01 10 11
ALGORITHM MatrixMultiplication(A[O .. n- 1,A O ..11n- 1],

12 13 14
B[O .. n- 1, O .. n- 1]) B 15 16 17 18
C 369 392 433 460
forC[1,1]+=C[1][1]+A[1][1]*B[1][1]
ito n-1 do
i 01
for j
C[1,1]+=208+14*18=460 j 012 01
c[i,j]=0 k 012 012 012 01

n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1 A 0 1 B 0 1
0 369 392 0 11 12 0 15 16
1 433 208 1 13 14 1 17 18
00 01 10 11
A 11 12 13 14
B 15 16 17 18
B[O .. n- 1, O .. n- 1])
C 369 392 433 460
for ito n-1 do i 01
for j j 012 01
k 012 012 012 01 2
c[i,j]=0
n 2
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
C 0 1 A 0 1 B 0 1
0 369 392 0 11 12 0 15 16
1 433 208 1 13 14 1 17 18
B[O .. n- 1, O .. n- 1])
for ito n-1 do 00 01 10 11

A 11 12 13 14
for j
B 15 16 17 18
c[i,j]=0 C 369 392 433 460
for i 01
j 012 01 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
k 012 012 012 01 2
return c n 2
C 0 1 A 0 1 B 0 1
0 369 392 0 11 12 0 15 16
1 433 208 1 13 14 1 17 18
B[O .. n- 1, O .. n- 1])
for ito n-1 do 00 01 10 11

A 11 12 13 14
for j
B 15 16 17 18
c[i,j]=0 C 369 392 433 460
for i 01 2
j 012 01 2
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
k 012 012 012 01 2
return c n 2
00 01 10 11
A 11 12 13 14
B 15 16 17 18
C 369 392 433 460
i 01 2
ALGORITHM MatrixMultiplication(A[O .. n- 1,j O .. n- 1],

012 01 2
k 012 012 012 01
B[O .. n- 1, O .. n- 1])
n 2
for ito n-1 do
BASIC OPERATION
for j
M(n) --> Total
c[i,j]=0 multiplications
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
B[O .. n- 1, O .. n- 1])
for ito n-1 do 𝑛− 1
for j ∑ 1
𝑘= 1
c[i,j]=0
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
𝑛 − 1 𝑛 −1
∑ ∑
1
B[O .. n- 1, O .. n- 1]) 𝑗 =0 𝑘= 0
for ito n-1 do
for j
c[i,j]=0
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
𝑛 −1 𝑛−1 𝑛 −1
𝑀 (𝑛)= ∑ ∑ ∑ 1
1], 𝑖=0 𝑗=0 𝑘=0
B[O .. n- 1, O .. n- 1])
for ito n-1 do
for j
c[i,j]=0
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
Mathematical Analysis ofAsymptotic Notations
Non-Recursive and Basic Efficiency Classes
Algorithms
B[O .. n- 1, O .. n- 1])
for ito n-1 do
for j
c[i,j]=0
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
B[O .. n- 1, O .. n- 1])
for ito n-1 do
for j
c[i,j]=0
for
C[i, j]+= C[i, j]+ A[i, k]*B[k, j]
return c
B[O .. n- 1, O .. n- 1])
for ito n-1 do
for j
BASIC OPERATION HAS
c[i,j]=0
1 Addition + 1 Multiplication
for
C[i, j]= C[i, j] + (A[i, k] * B[k, j])

return c

B[O .. n- 1, O .. n- 1])
BASIC OPERATION HAS
for ito n-1 do
1 Addition + 1 Multiplication
for j
c[i,j]=0
for
C[i, j]+= C[i, j] + (A[i, k] * B[k, j])

return c
M(n) = Number of times multiplication is performed =

A(n) = Number of times Addition is Performed =
B[O .. n- 1, O .. n- 1])
for ito n-1 do

t(n)=Time Efficiency of our Program,
for j
c[i,j]=0 then
for t(n)
C[i, j]+= C[i, j] + (A[i, k] * B[k, j])
=
return c
=(+)
Mathematical Analysis of Recursive Algorithms
Ex1:FACTORIAL
ALGORITHM F(n)
if n=0
return 1
else
return F(n-1)*n
Ex1:FACTORIAL
ADDRESS SPACE OF FIRST INSTANCE

FALSE
n 5
ALGORITHM F(n)
if n=0
return 1
else
return F(n-1)*n
Ex1:FACTORIAL

n 5 × F(4)
ADDRESS SPACE OF SECOND INSTANCE
n 4
ALGORITHM F(n)
if n=0
return 1
else
return F(n-1)*n
Ex1:FACTORIAL

n 5 × F(4)
FALSE
n 4
ALGORITHM F(n)
if n=0
return 1
else
return F(n-1)*n
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
ALGORITHM F(n)
if n=0
n 3
return 1
else
return F(n-1)*n
Ex1:FACTORIAL

n 5×F(4)
FALSE ADDRESS SPACE OF SECOND INSTANCE
n 4× F(3)
ALGORITHM F(n) n 3
if n=0
return 1
else
return F(n-1)*n
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
n 3× F(2)
ALGORITHM F(n)
n 2
if n=0
return 1
else
return F(n-1)*n
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
FALSE
n 3× F(2)
ALGORITHM F(n)
n 2
if n=0
return 1
else
return F(n-1)*n
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
n 3× F(2)
ALGORITHM F(n)
n 2 × F(1)
if n=0
return 1
n 1
else
return F(n-1)*n
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
FALSE
n 3× F(2)
ALGORITHM F(n)
n 2 × F(1)
if n=0
return 1
n 1
else
return F(n-1)*n
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
ADDRESS SPACE OF THIRD INSTANCE
n 3× F(2)
ADDRESS SPACE OF FOURTH INSTANCE
ALGORITHM F(n)
n 2 × F(1)
if n=0
ADDRESS SPACE OF FIFTH INSTANCE
return 1
n 1 × F(0)
else
ADDRESS SPACE OF SIXTH INSTANCE
return F(n-1)*n
n 0
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
TRUE
n 3× F(2)
ALGORITHM F(n) ADDRESS SPACE OF FOURTH INSTANCE
if n=0 n 2 × F(1)
return 1 ADDRESS SPACE OF FIFTH INSTANCE
else n 1 × F(0)
return F(n-1)*nADDRESS SPACE OF SIXTH INSTANCE
n 0
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
TRUE
n 3× F(2)
ALGORITHM F(n)
n 2 × F(1)
if n=0
return 1
n 1 × F(0) =1 × 1=1
else
return F(n-1)*n
n 0 Return 1
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
TRUE
n 3× F(2)
ALGORITHM F(n)
n 2 × F(1) = 2 × 1 = 2
if n=0
return 1
n 1 × F(0) =1 × 1=1
else
return F(n-1)*n
n 0 Return 1
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3)
TRUE
n 3× F(2) = 3 × 2 = 6
ALGORITHM F(n)
n 2 × F(1) = 2 × 1 = 2
if n=0
return 1
n 1 × F(0) =1 × 1=1
else
return F(n-1)*n
n 0 Return 1
Ex1:FACTORIAL

n 5×F(4)
n 4× F(3) = 4 × 6 = 24
TRUE
n 3× F(2) = 3 × 2 = 6
ALGORITHM F(n)
n 2 × F(1) = 2 × 1 = 2
if n=0
return 1
n 1 × F(0) =1 × 1=1
else
return F(n-1)*n
n 0 Return 1
Ex1:FACTORIAL

n 5×F(4) = 5 × 24 = 120
n 4× F(3) = 4 × 6 = 24
TRUE
n 3× F(2) = 3 × 2 = 6
ALGORITHM F(n)
n 2 × F(1) = 2 × 1 = 2
if n=0
return 1
n 1 × F(0) =1 × 1=1
else
return F(n-1)*n
n 0 Return 1
Ex1:FACTORIAL
ALGORITHM F(n)
if n=0
return 1
else
return F(n-1)*n
1. Number of multiplication=M(n)=M(n-1)+1
Ex1:FACTORIAL
ALGORITHM F(n)
if n=0
return 1
else
return F(n-1)*n
M(n)=M(n-1)+1
=(M((n-1)-1)+1)+1 = M(n-2)+2
=(M((n-2)-1)+1)+2 = M(n-3)+3
...........................
= M(n-i)+i
...........................
= M(n-n)+n = n
Ex2 : Towers of Hanoi
CLICK To see the Tower of Hanoi Problem
Recursion
1. Move the (n-1) disks from PEG 1 to PEG2
2. Move last one disk from PEG1 to PEG3
3. Move (n-1) disks from PEG2 to PEG3
M(n)=M(n-1)+1+M(n-1)
=2M(n-1)+1 for n>1
=1 for n=1
Recursion
Continued for some i,

Recursion
So,
Recursion
For ith Iteration
for n=n-1th Iteration

Tree of Recursive calls made

Important Problem Types
 Sorting
 Searching
 String Processing
 Graph Problems
 Combinatorial Problems
 Geometric Problems
 Neumerical Problems
Sorting
What kind of applications?

 Library Books Ordering
 Student data maintenence
Information on which to sort (Key)

 Alphabetical order of name
 Register number
 Book ISSN number etc.
Why Sorting?
 It makes searching easy.
Sorting
What is the best algorithm efficiency we have?

 We have few good algos with efficiency of nlog2n ... And research
is still on for best.
 There is no algorithm whi can perform better for any order list.
 Stable:
Before d e i h j
a[i]=a[j]
Sorting where i<j
After d e i h j
e<j
Sorting
Searching
Finding the given value [SEARCH KEY]
Every algorithm has a best case and worst case
Sometimes SEARCHING is part of ADDITION and
DELETION
String Operation
Non-numerical data.
Sequence of characters(may be alphabets, letters, special
symbols etc.)
Applications like: String Matching

Graph Problems
Verices and Edges.
Applications are huge: Traffic problem, game scheduling,
project deadlines etc.
Latest : Web Diameter
Traveling Salesman Problem
Graph Color Problem.

Combinatorial Problems
Permutation
Combination
Subset
Considered
Very Difficult
Problem Size increses fast
Lots of algorithms present but not acceptable.

Geometrical Problems
Ex: The convex-hull problem asks to find the
smallest convex polygon that would include all the
points of a given set.

Neumerical Problems
Involves Mathematical Objects.
Solving Equations or Set of Equations.
Integrals
Functions.
Neumerical Problems
In the last 25 years or so, the computing industry has shifted
its focus to business applications.
These new applications require primarily algorithms for
information storage, retrieval, transportation through
networks, and presentation to users.
As a result of this revolutionary change numerical analysis
has lost its formerly dominating position in both industry and
computer science programs.

Unit I

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Unit I

Uploaded by

Copyright:

Available Formats

DESIGN

 By Persian Author Abu Ja’far Mohammed ibn al

INPUT ALGORITHM OUTPUT

DEFINITENESS FINITENESS EFFECTIVENESS

INPUT ALGORITHM OUTPUT

DEFINITENESS FINITENESS EFFECTIVENESS

Task: To add 2 numbers.

INPUT ALGORITHM OUTPUT

DEFINITENESS FINITENESS EFFECTIVENESS

Task: To add 2 whole numbers.

Input: 2 signed integers.

INPUT ALGORITHM OUTPUT

DEFINITENESS FINITENESS EFFECTIVENESS

OUTPUT-1: Result of Addition

OUTPUT-2 : Error message(if user enters something other than numbers)

INPUT ALGORITHM OUTPUT

DEFINITENESS FINITENESS EFFECTIVENESS

Instructions are unambiguous.

INPUT ALGORITHM OUTPUT

DEFINITENESS FINITENESS EFFECTIVENESS

INPUT ALGORITHM OUTPUT

DEFINITENESS FINITENESS EFFECTIVENESS

Instructions are feasible

How to Device algorithm?

How to Validate algorithm?

How to Analyze algorithm?

How to Test algorithm?

How to Device algorithm?

How to Validate algorithm?

There are some methods which

1. Can we automate algorithm writing?

How to Device algorithm?

How to Validate algorithm?

How to Analyze algorithm?

How to Test algorithm?

1. I want to organize your blue books according to your

How to Device algorithm?

How to Validate algorithm?

How to Analyze algorithm?

How to Test algorithm?

Process of making sure that algorithm gives

Program. Predicate Calculus

How to Device algorithm?

How to Validate algorithm? Analysis of Algorithm/ Performance Analysis

How to Analyze algorithm?

How to Test algorithm?

Best Case Performance

Average Case Performance

Worst Case Performance

How to Device algorithm?

How to Validate algorithm?

How to Analyze algorithm?

How to Test algorithm?

How to Device algorithm?

How to Validate algorithm?

How to Analyze algorithm?

How to Test algorithm?

How to Validate algorithm?

How to Test algorithm?

Makes sure that program provides proper output

Program is run on sample data and output is checked

How to Device algorithm?