BANGALORE INSTITUTE OF TECHNOLOGY

Department of Information Science and Engineering

Subject Name: Design and Analysis of Algorithms
Subject Code: 18CS42
MODULE - 1

Staff In-charge
S MERCY
Assistant Professor
 COURSE OUTCOMES
• At the end of the course, the students will be able to attain the following skills.

Understand the various techniques for designing algorithm such as divide and
conquer, greedy, dynamic programming and backtracking.

Apply appropriate data structures and design technique to solve various problems.

Analyze the complexity of different algorithms to identify their efficiency.

Applied suitable algorithm for the given real-world problems and compared their
difference.
 MODULE - 1
• Introduction: What is an Algorithm?
Algorithm Specification
Analysis Framework
• Performance Analysis: Space complexity
Time complexity
• Asymptotic Notations: Big-Oh notation (O)
Omega notation (Ω)
Theta notation (Θ)
Little-oh notation (o)
Mathematical analysis of Non-Recursive and Recursive Algorithms with Examples.
• Important Problem Types: Sorting, Searching, String processing, Graph Problems, Combinatorial Problems.
• Fundamental Data Structures: Stacks, Queues, Graphs, Trees, Sets and Dictionaries.
 INTRODUCTION-WHAT IS AN ALGORITHM?
• An algorithm is a step by step procedure for solving the given problem.
• An algorithm is a finite sequence of effective steps to solve a particular
problem where each step is unambiguous(definite) and which terminates for
all possible inputs in a finite amount of time.

• An algorithm accepts zero or more inputs and produces one or more outputs.
• An algorithm is independent of any programming language and machine.
• An algorithm satisfy the following criteria,
 Algorithm Properties
Input: Zero or more quantities are externally supplied.
Output: At least one quantity is produced.
Definiteness: Each instruction is clear and unambiguous.(it also must be feasible)
Finiteness: Algorithm terminates after a finite number of steps. The time for termination
should be reasonably short.
Effectiveness: Every instruction must be very basic so that it can be carried out by a person
using only pencil and paper.
A program is the expression of an algorithm in a programming language.
The study of algorithms include 4 areas:
How to devise algorithms
How to validate algorithms
How to analyze algorithms
How to test a program
 Analyze
• This phase performs performance analysis.
• Execution requires computing time and storage.
• It allows quantitative judgements.
• Task of determining time and space requirements.
• Performance in cases considered separately,
– Best case
– Worst case
– Average case
 Testing
Testing: Two phases
1. Debugging
2. Profiling (performance measurement)
Debugging:
• Process of executing programs on sample data sets to determine whether faulty results
occur and if so to correct them.
• Requirement
– A proof of correctness prove that the algorithm will correctly for all cases than a
thousand tests.
– Not easy to obtain mathematical proofs.
Profiling:
• Executing programs with sample data sets to measure the time and space requirements.
• It is helpful for optimization since it can point out logical places that require optimization.
 ALGORITHM SPECIFICATION
• An algorithm can be specified in
1. Simple English.
2. Graphical representation like flow chart. Flowcharts graphically represent the
algorithm. But it can only be used for simple algorithms or problems.
3. Pseudo code is used to represent algorithm.
4. Programming language like c++ / java.
5. Combination of above method.

Selection Sort
Selection Sort Example
 Direct Recursive and Indirect Recursive
• An algorithm is said to be recursive if the same algorithm is invoked in the body (direct
recursive).
• An algorithm A is said to be indirect recursive if it calls another algorithm which in turn
calls A.
Example 1: Factorial computation n! = n * (n-1)!
Example 2: Tower of Hanoi problem
Example 3: Permutation Generator program

Tower of Hanoi
Permutation Generator Process Flow
 Permutation Generator Function
void perm(a[], k, n)
{
if(k==n)
{
for(int i=1; i<=n; i++)
print a[i];
}
else // a[k:n] has more than one permutation
for(int i=k; i<=n; i++)
{
t=a[k];
a[k]=a[i];
a[i]=t;
perm(a,k+1,n); // all permutation of a[k+1:n]
t=a[k];
a[k]=a[i];
a[i]=t;
}
} Note: The number of character is always equal to the depth of the tree.
Permutation Generator Example
 ANALYSIS FRAMEWORK
Two kinds of efficiency
1. Time efficiency
2. Space efficiency
Measuring an Input size:
If an algorithm examines individual characters of its input, measure the size by the
number of characters.
Units for measuring running time:
Approach is to count the number of times each of the algorithms operations is
executed, it is difficult.
Identify the most important operation of the algorithm called basic operation.
T(n) is the running time of a program.
Cop is the execution time of an algorithm’s basic operation on a particular computer.
C(n) is the number of times this operation needs to be executed for this algorithm.

T(n) ͌ Cop*C(n)
 Analysis Framework
Order of Growth:
An order of growth is a set of functions whose asymptotic growth behavior is
considered equivalent. For example, 2n, 100n and n + 1 belong to the same order of growth,
which is written O(n) in Big-Oh notation and often called linear because every function in the
set grows linearly with n.
constant − Ο(1)

logarithmic − Ο(log n)

linear − Ο(n)

n log n − Ο(n log n)

2
quadratic − Ο(n )
3
cubic − Ο(n )
Ο(1)
polynomial − n
Ο(n)
exponential − 2
 PERFORMANCE ANALYSIS – SPACE COMPLEXITY
• To improve existing algorithms
• To choose among several available algorithms
Space Complexity:
Amount of memory the algorithm needs to run to completion
Performance evaluation divided into 2 major phases
1.priori estimates (performance analysis)
2.posteriori testing (performance measurement)
Space needed by the program is the sum of,
– Fixed part
– Variable part S(P)= c+Sp
 PERFORMANCE ANALYSIS – TIME COMPLEXITY
Time Complexity:
Is the amount of computer time it needs to run to completion.
Time T(p) taken by a program p is the sum of the compile time and the run time.
➢ 2 ways to determine the number of steps needed by a program to solve a particular
problem instance,

1. variable count
2. step count

Variable count:
Introduce variable count into the program.
Each time a statement in the original program is executed, count is incremented by
the step count of that statement.
Step count:
Build a table
 Variable Count
Variable Count:

float sum(float a[], int n)

{
float s=0.0;
count++;
for(int i=1; i<=n; i++)
{
count++;
s+= a[i];
count++;
}
count++;
count++;
return s;
}
 Step Count
Step Count: s/e freq total steps

float sum(float a[], int n) 0 - 0

{ 0 - -
float s= 0.0; 1 1 1
for(int i=1; i<=n; i++) 1 n+1 n+1
s+= a[i]; 1 n n
return s; 1 1 1
} 0 - 0 =2n+3
---------------------------------------------------------------------------------------------------------------------------
n=0 n>0 n=0 n>0
float rsum(float a[], int n) 0 - - 0 0
{ - - - - -
if(n<=0) 1 1 1 1 1
return(0.0); 1 1 0 1 0
else
return (rsum(a, n-1)+a[n]); 1+x 0 1 0 1+x
} 0 - - 0 0 =2, 2+x
 Recurrence Relation

x = trsum(n-1)

trsum(n)= { 2 if n=0
2+ trsum(n-1) if n>0

known as a recurrence relation.

 Matrix Addition
Matrix Addition:

S/e freq Total

void add(a, b, c, m, n)
{
1 m+1 m+1
for (i=1; i<=m; i++)
1 m(n+1) m(n+1)
for (j=1; j<=n; j++)
1 mn mn
c[i][j]=a[i][j]+b[i][j];
}
=2mn+2m+1
 ASYMPTOTIC NOTATIONS
To compare orders of growth, use three notations:
O(Big Oh)
Ω(Big Omega)
Θ (Big Theta)
Informal:
O(g(n)) is the set of all functions with a smaller or same order of growth as g(n), (to
within a constant multiple, as n goes to infinity).
nϵO(n^2)
n3ϵO(n^2)

Ω(g(n)) is the set of all functions with a larger or same order of growth as g(n), (to
within a constant multiple, as n goes to infinity).

θ(g(n)) is the set of all functions that have same order of growth as g(n), (to within a
constant multiple, as n goes to infinity).
 Big Oh
Formal:
A function f(n) is said to be in O(g(n)), denoted by f(n)ϵO(g(n)), if f(n) is bounded
above by some constant multiple of g(n) for all large n. (ie) if there exist some positive
constant C and some non negative integer n0 such that,

f(n)<=Cg(n) for all n>=n0. ( f(n) algorithm’s running time)

 Big Omega
A function f(n) is said to be in Ω(g(n)), denoted by f(n)ϵ Ω(g(n)), if f(n) is bounded
below by some positive constant multiple of g(n) for all large n. (ie) if there exist some
positive constant C and some non negative integer n0 such that,

f(n)>= Cg(n) for all n>= n0.

 Big Theta
A function f(n) is said to be in θ(g(n)), denoted by f(n)ϵ θ(g(n)), if f(n) is bounded both
above and below by some positive constant multiples of g(n) for all large n. (ie) if there exist
some positive constant C1 and C2 and some non negative integer n0 such that,

C2g(n)<= f(n)<=C1g(n) for all n>=n0.

 Example
For example, let us prove that 1/2n(n − 1) ∈ θ(n^2).

First, prove the right inequality (the upper bound):

1/2n(n − 1) = 1/2n^2 − 1/2n ≤ 1/2n^2 for all n ≥ 0.

Second, prove the left inequality (the lower bound):

1/2n(n − 1) = 1/2n^2 − 1/2n ≥ 1/4n^2 (for all n ≥ 2) = 1/4n^2

Hence, select c2 = 1/4 , c1 = 1/2 , and n0 = 2.

 Property involving asymptotic notations
To analyze the algorithm that comprise 2 consecutively executed parts.
Theorem,
If t1(n) ∈ O(g1(n)) and t2(n) ∈ O(g2(n)), then
t1(n) + t2(n) ∈ O(max{g1(n), g2(n)}).

Proof:
Take four arbitrary real numbers a1, b1, a2, b2
if a1 ≤ b1 and a2 ≤ b2, then a1 + a2 ≤2 max{b1, b2}.

Since t1(n) ∈ O(g1(n)), there exist some positive constant c1 and some non-negative integer
n1 such that,

t1(n) ≤ c1g1(n) for all n ≥ n1.

 Property involving asymptotic notations
Similarly, since t2(n) ∈ O(g2(n)),
t2(n) ≤ c2g2(n) for all n ≥ n2.

Let us denote c3 = max{c1, c2} and consider n ≥ max{n1, n2} so that use both inequalities.
adding the 2 inequivalities above,
t1(n) + t2(n) ≤ c1g1(n) + c2g2(n)
≤ c3g1(n) + c3g2(n)
= c3[g1(n) + g2(n)]
≤ c3 2max{g1(n), g2(n)}.

Hence, t1(n) + t2(n) ∈ O(max{g1(n), g2(n)}), with the constants c and n0 required
by the O definition being
2c3 = 2 max{c1, c2} and max{n1, n2}, respectively.
If the algorithm comprises 2 consecutively executed parts, the algorithms overall efficiency is
determined by the part with the larger order of growth.(ie) its least efficient part.
 Order of Growth
Using limits for comparing order of growth:
Though the formal definitions of asymptotic notations are indispensable for proving
their abstract properties, it is rarely used for comparing the orders of growth of
two specific functions.
A convenient method for doing is based on computing the limit of the ratio of two functions.

lim t (n) 0, implies that t (n) has a smaller order of growth than g(n)
_____ =
n→∞ g(n) c>0, implies that t (n) has the same order of growth as g(n)

∞, implies that t (n) has a larger order of growth than g(n)

Limit based approach is more convenient than the one based on the definitions because it
take advantage of the powerful calculus techniques developed for computing limits, such as
L’Hospital rule and Stirling’s formula.
 Formulas
L’ Hospital rule:

lim t (n) lim t’(n)

_____ = ________
n→∞ g(n) n→∞ g’(n)

Stirling’s formula:

n!≈ √2πn (n/e)^n for large values of n.

 Problems
1. Compare the orders of growth of 1/2n(n − 1) and n^2.

lim 1/2n(n-1)
________ = ½ lim n^2-n/n^2
n→∞ n^2 n->∞

= ½ lim
n→∞ (1 − 1/n) = 1/2
.

Since the limit is equal to a positive constant, the functions have the same order
of growth.
1/2n(n − 1) ∈ θ (n^2).
 Problems
2. Compare the orders of growth of log n base 2 and √n.
lim log n base 2
___________
n→∞ √n

= lim( log n base 2)’

____________
n→∞ ( √n)’

lim ( log e base 2)1/n

= ______________
n→∞ 1/2√n
lim √n
= 2 log e base 2 ____ =0. Since the limit is equal to zero, log n base 2 has a
n→∞ n
a smaller order of growth than √n. so log n base 2 ∈ O √n.
 Problems
3. Compare the orders of growth of n! and 2^n.

lim n! lim √2πn(n/e)^n

_____ = ____________
n→∞ 2^n n→∞ 2^n

lim √2πn (n)^n

= ______________
n→∞ 2^ne^n

lim
= √2πn(n/2e)^n =∞
n→∞

2^n grows very fast, n! grows still faster. n!∈ Ω(2^n)

MATHEMATICAL ANALYSIS OF NON RECURSIVE ALGORITHM
General Plan:
1. Decide on a parameter indicating an input’s size.
2. Identify the algorithm’s basic operation.
3. Check whether the number of times the basic operation is executed depends only on
the size of an input. If it also depends on some additional property, the worst-case,
average-case, and, if necessary, best-case efficiencies have to be investigated
separately.
4. Set up a sum expressing the number of times the algorithm’s basic operation is
executed.
5. Using standard formulas and rules of sum manipulation, either find a closed form
formula for the count or, at the very least, establish its order of growth.
 Finding the largest element
Finding the value of the largest element in a list of n numbers.

ALGORITHM MaxElement(A[0..n − 1])

// Determines the value of the largest element in a given array
// Input: An array A[0..n − 1] of real numbers
// Output: The value of the largest element in A
maxval ← A[0]
for i ← 1 to n − 1 do
if A[i] > maxval
maxval ← A[i]
return maxval
Number of comparisons will be same for all arrays of size n, there is no need to distinguish among the worst,
average and best case.
n-1
C(n) = Σ 1
i=1
=UB-LB+1
=(n-1-1+1) = n-1
ϵ θ(n)
 Element Uniqueness
Element uniqueness problem
check whether all the elements in a given array are distinct.

ALGORITHM UniqueElements(A[0..n − 1])

// Determines whether all the elements in a given array are distinct
// Input: An array A[0..n − 1]
// Output: Returns “true” if all the elements in A are distinct and “false” otherwise
for i ← 0 to n − 2 do
for j ← i + 1 to n − 1 do
if A[i] = = A[j ]return false
return true

Number of element comparison will depend not only on n but also on whether there are equal element in the
array.
n-2 n-1 n-2 n-2 n-2 n-2
C(n) = Σ Σ 1 = Σ [(n-1)-(i+1)+1] = Σ (n-1-i) = Σ (n-1) - Σ i = θ(n^2)
i=0 j=i+1 i=0 i=0 i=0 i=0
 Matrix Multiplication
Matrix Multiplication:

ALGORITHM MatrixMultiplication(A[0..n − 1, 0..n − 1], B[0..n − 1, 0..n − 1])

// Multiplies two square matrices of order n
// Input: Two n × n matrices A and B
// Output: Matrix C = AB

for i ← 0 to n − 1 do
for j ← 0 to n − 1 do
C[i, j ]← 0.0
for k ← 0 to n − 1 do
C[i, j ]← C[i, j ] + A[i, k] ∗ B[k, j ]
return C
The total number of multiplications M(n) is expressed by the following triple sum:
n-1 n-1 n-1 n-1 n-1 n-1
M(n) = Σ Σ Σ 1 = Σ Σ n = Σ n^2 = n^3
i=0 j=0 k=0 i=0 j=0 i=0

ϵ ϴ(n^3)
 Number of binary digits
Find the number of binary digits in the binary representation of a positive decimal integer.

ALGORITHM Binary(n)
// Input: A positive decimal integer n
// Output: The number of binary digits in n’s binary representation
count ← 1
while n > 1 do
count ← count + 1
n ← L( n/2) ˩
return count

Time complexity ϴ(log2n)

 MATHEMATICAL ANALYSIS OF RECURSIVE ALGORITHMS
General Plan for Analyzing the Time Efficiency of Recursive Algorithms:

1. Decide on a parameter indicating an input’s size.

2. Identify the algorithm’s basic operation.
3. Check whether the number of times the basic operation is executed can vary on
different inputs of the same size; if it can, the worst-case, average-case, and best-case
efficiencies must be investigated separately.
4. Set up a recurrence relation, with an appropriate initial condition, for the number
of times the basic operation is executed.
5. Solve the recurrence or at least ascertain the order of growth of its solution.
 Factorial
Compute the factorial function F(n)=n! for an arbitrary non negative integer n.
n!= 1 . ... . (n − 1) . n
= (n − 1)!. n for n ≥ 1 and 0!= 1
compute F (n) = F (n − 1) . n with the following recursive algorithm.

ALGORITHM F(n)
// Computes n! recursively
// Input: A nonnegative integer n
// Output: The value of n!
if n = 0 return 1
else return F (n − 1) ∗ n
Recurrence relation M(n) = M(n − 1) + 1 for n>0

to compute F (n−1) to multiply F (n−1) by n

M(0)=0
M(n) = M(n − 1) + 1
use substitution method
= [M(n − 2) + 1]+1 = M(n − 2) + 2 ........
= M(n − i) + i = M(n − n) + n = M(0)+n ϵ ϴ(n)
 Tower of Hanoi
n disks of different sizes and 3 pegs.
Move all the disks to the third peg using the second one as an auxiliary.
Move only one disk at a time.
 Tower of Hanoi
M(n) = M(n − 1) + 1 + M(n − 1) for n > 1. Initial condition M(1) = 1
M(n) = 2M(n − 1) + 1

solve this recurrence by the substitution method:

M(n) = 2M(n − 1) + 1
= 2[2M(n − 2) + 1] + 1 = 2^2 M(n − 2) + 2 + 1
= 2^2 [2M(n − 3) + 1] + 2 + 1 = 2^3 M(n − 3) + 2^2 + 2 + 1.

after i substitutions,
M(n) = 2^i M(n − i) + 2^(i−1) + 2^(i−2) + ... + 2 + 1
= 2^i M(n − i) + 2i − 1.

Since the initial condition is specified for n = 1, which is achieved for i = n − 1

M(n) = 2^n−1 M(n − (n − 1)) + 2^(n−1)− 1 = 2^(n−1) M(1) + 2^(n−1) − 1

= 2^(n−1) + 2^(n−1) − 1
= 2^n −1
ϵ ϴ(2^n)
 Number of binary digits
Find the number of binary digits in the binary representation of a positive decimal integer:

ALGORITHM BinRec(n)
// Input: A positive decimal integer n
// Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec L(n/2)˩ + 1
Recurrence relation A(n)= AL(n/2)˩ + 1 for n > 1
Initial condition is A(1) = 0.
0 if n<=1
F(n)=
1+f(n/2) otherwise
F(n)=1+f(n/2)
=1+1+f(n/2^2)=2+f(n/2^2)
=3+f(n/2^3)
=i+f(n/2^i) take 2^i=n,
=i+f(1) ilog2 base2=logn base2
F(n)=i i= logn base 2
=logn base2
ϵ ϴ(logn base2)
 IMPORTANT PROBLEM TYPES
• Sorting – rearrange the item of a given list in non-decreasing order.
– there is no algorithm that would be the best solution in all situations.
Two properties – Algorithm is stable if it preserves the relative order of any two equal elements in its input.
– inplace (ie) no extra memory.
• Searching – Linear search and Binary search.
• String Processing – String matching is a searching for a given word in a text.
• Graph Problems – Can be used for modeling a wide variety of real life applications, including transportation
and communication networks, project scheduling and games.
• Combinatorial Problems – ask to find the combinatorial object such as a permutation, a combination or a
subset that satisfies certain constraints and has some desired property. Eg. TSP and Graph Coloring problem.
• Geometric problems – deals with geometric objects such as points, lines and polygons.
• Numerical problems – problems that involve mathematical objects of continuous nature.
 FUNDAMENTAL DATA STRUCTURES
1. Stack
2. Queues
3. Graphs
4. Trees
5. Sets and Dictionaries
A set can be described as an unordered collection of distinct items called element of
the set.
Operations perform for a set or a multiset are searching for a given item, adding a
new item, deleting an item from the collection. A data structure that implements these
three operations is called the dictionary.
 FORMULAS
1 + 2 + … + n = n * (n + 1) / 2
1 + 22 + … + n2 = n * (n + 1) * (2n + 1) / 6
1 + a + a2 + … + an = (a(n+1) – 1) / (a – 1)
A + ar + ar2 + ar3 +……….+ arn = a(rn+1-1) / (r-1)
 FORMULAS
• Floor(x) is the largest integer less than or equal to x
• Ceil(x) is the smallest integer greater than or equal to x

loga xy=log a x + loga y

log a(x/y) = loga x - loga y
log a x^n = n log a x
log a 1=0
log x x=1
log a x=1/log x a
loga(x) = loga(b) * logb(x)
log a x= log b x / log b a
alog a x=x
if 2k=n then k=log 2 n
xlogby=ylogbx
FORMULAS

for n>=k>=0