Design and Analysis of

Algorithm
6CME
Unit 1
• Algorithm-definition, Specification- pseudo code conventions,
recursive algorithms, Performance analysis – space complexity, time
complexity, asymptotic notation, practical complexities,
performance measurement, Randomized algorithms- basics of
probability theory, identifying the repeated element, primality
testing, advantages and disadvantages
Algorithm
• Informally, an algorithm is any well-defined computational
procedure that takes some value, or set of values, as input and
produces some value, or set of values, as output.

• An algorithm is thus a sequence of computational steps that

transform the input into the output.
• Example

• To sort a sequence of numbers into increasing order.

Algorithm specification
An algorithm is defined as a finite set of instructions that, if followed,
performs a particular task. All algorithms must satisfy the following
criteria
• Input. An algorithm has zero or more inputs, taken or collected from
a specified set of objects.
• Output. An algorithm has one or more outputs having a specific
relation to the inputs.
• Definiteness. Each step must be clearly defined; Each instruction
must be clear and unambiguous.
• Finiteness. The algorithm must always finish or terminate after a
finite number of steps.
• Effectiveness. All operations to be accomplished must be sufficiently
basic that they can be done exactly and in finite length.
• Input: A sequence of n numbers (a1, a 2, …. ,a n)

• Output: A permutation (reordering) (a 1’, a2’,…,an’) of the input

sequence such that a 1’ <= a2’<= …. <=a n’.
• For example,
• given the input sequence (31; 41; 59; 26; 41; 58), a sorting algorithm
returns as output the sequence (26; 31; 41; 41; 58; 59).
• Such an input sequence is called an instance of the sorting problem.
• In general, an instance of a problem consists of the input (satisfying
whatever constraints are imposed in the problem statement) needed
to compute a solution to the p
• Insertion sort

• Insertion sort works the way many people sort a hand of playing
cards. We start with an empty left hand and the cards face down on
the table. We then remove one card at a time from the table and
insert it into the correct position in the left hand. To find the correct
position for a card, we compare it with each of the cards already in
the hand, from right to left.
Insertion sort contd..
Insertion sort contd..
• The numbers that we wish to sort are also known as the keys
• Although conceptually we are sorting a sequence, the input comes to
us in the form of an array with n elements
Pseudo code and real code
• In pseudocode, we employ whatever expressive method is most
clear and concise to specify a given algorithm.
• Sometimes, the clearest method is English, so do not be surprised if
you come across an English phrase or sentence embedded within a
section of “real” code.
• Another difference between pseudocode and real code is that
pseudocode is not typically concerned with issues of software
engineering.
• Exmaple: Issues of data abstraction, modularity, and error handling
are often ignored in order to convey the essence of the algorithm
more concisely.
Insertion sort pseudocode
Conventions used in pseudocode
• Indentation indicates block structure. Using indentation instead of
conventional indicators of block structure, such as begin and end
statements, greatly reduces clutter while preserving, or even
enhancing, clarity
• The symbol “//” indicates that the remainder of the line is a
comment.
• A multiple assignment of the form i = j = e assigns to both variables i
and j the value of expression
• Variables (such as i, j , and key) are local to the given procedure.
Analyzing algorithms
• Analyzing an algorithm has come to mean predicting the resources
that the algorithm requires.
• Occasionally, resources such as memory, communication bandwidth,
or computer hardware are of primary concern.
• but most often it is computational time that we want to measure.
• Hence algorithm analysis refers to the task of determining the
computing time and storage space required for an algorithm. It is
also known as performance analysis/ efficiency of an algorithm.
Analyzing algorithm contd..

• Given an algorithm, first task is to determine which operations are

employed and what their relative costs are
• An algorithm can be analyzed in two ways
• By checking the correctness of the algorithm
• By measuring time and space complexity of the algorithm
Analyzing algorithm contd..

• To compute the analysis of the algorithm two phases are required

• Priori analysis
• Posteriori analysis
Priori analysis
• In this method we obtain the function which bounds the algorithms
computing time
• Assume there is a statement in the middle of the program
• We need to determine the total time that statement will spend for execution
given some input data
• This requires the statements frequency count and the time taken for one
execution.
• The notation used in priori analysis is Big-oh(O) , Omega (Ω),
theta(Ɵ) and small o(o)
• Priori analysis ignores factors depended on machine and
programming language and only concentrates on determining the
frequency of execution of the statements.
Posteriori analysis
• In this method, the actual statistics of the algorithm, conjunction of
the time and space while executing is collected.
• Once the algorithm is written , it has to be tested.
• Debugging: Executing program with sample data to determine if we get
proper results.
• Profiling: executing correct program on actual dataset to measure the time
and space it takes to compute the result during execution. The actual time
taken by the algorithm to process data is called profiling
 ANALYSIS PROFILING

Analysis is the process of determining how much Profiling is the process of executing the correct program
computing time and storage an algorithm will require on datasets and measuring the time and space it takes
to compute the results

This is independent of machine programming language Dependent on machine programming language and the
compiler
Step count and operation count

• In the step-count method, we attempt to account for the time spent

in all parts of the algorithm.

• The operation-count method of estimating time complexity omits

accounting for the time spent on all but the chosen operations.
Example:
Analysis of insertion sort
Analysis of insertion sort
• The time taken by the INSERTION-SORT procedure depends on the
input.
• sorting a thousand numbers takes longer than sorting three numbers.

• Moreover, INSERTION SORT can take different amounts of time to

sort two input sequences of the same size depending on how nearly
sorted they already are.
• In general, the time taken by an algorithm grows with the size of the
input, so it is traditional to describe the running time of a program as
a function of the size of its input.
Time complexity
• Time complexity is the amount of time taken by an algorithm to run,
as a function of the length of the input.

• It measures the time taken to execute each statement of code in an

algorithm
Space complexity
• Indicates the amount of temporary storage required for running an
algorithm
• Memory needed by the algorithm to run to complete
 Asymptotic analysis
• Asymptotic analysis refers to computing the running time of any
operation in mathematical units of computation.

• i.e. when I give an input how does running time behaves

• Usually, the time required by an algorithm falls under three types −

• Best Case − Minimum time required for program execution.
• Average Case − Average time required for program execution.
• Worst Case − Maximum time required for program execution.
Rate of growth
• Depends on the algorithm
• Ignore unnecessary elements/ small values
• Approximation

n 4+100n2+50 approximated to n 4
Asymptotic notation

• Asymptotic Notation is used to describe the running time of an

algorithm - how much time an algorithm takes with a given input, n.
• There are three different notations :
• Big-oh(O)
• Omega (Ω)
• Theta(Ɵ)
Big-oh-notation
• Big-oh notation gives an upper bound on function f(n). The upper
bound indicates that the function f(n) will be the worst case that it
does not consume more than this computing time

• F(n)=Big-Oh (g(n)) f(n) <= g(n)

Omega notation
• This notation is used to find the lower bound behavior of f(n). The
lower bound describes that below this time the algorithm can not
perform better
• OR (The algorithm will take at least this much time)

• F(n) = Omega(g(n)) f(n) >= g(n)

Theta notation
• Theta notation is used when the function f(n) is bound both from
above and below by the same function g(n).
• For some functions upper bound and the lower bound will be the
same
• The notation used to represent functions that has same time
complexities for upper bound and lower bound is called theta
notation

• F (n) = Theta (g(n)) f(n) = g(n)

 Big-oh notation

g(n) is an asymptotic upper bound for f(n)

• Example:

2n2= O(n 3)

N2=O(n2)
• Example:

3n+10 O(N)
2n2 -10n+100 O(N 2 )
100 O(1)
Omega notation
• Example

• 5n2 = Ω(n)
Theta notation
• Example
• 5n2

O(n 2) and Ω(n2)

• A function, which takes a constant c1 and takes a lowerbound and
c2 and becomes a upperbound and is called tight bound
represented as theta.

Pick the highest power

Space complexity
• How much space is occupied in memory(RAM)

• How many variables ? 6

• Space complexity : 6 variables , since it is a constant space complexity is O(1)
• Time complexity? O(N+M)
A and b are getting values assigned to them

First for loop executes N times

Second for loop executes for M times

Hence time complexity = O(N+M)

Space complexity is the storage space. In this case 6

variables and none of them are dependent on the
number of inputs. Hence O(1)
Outer for loop runs N times

Inner for loop ?

Time complexity chart
Iterative and recursive algorithms
• A program is call iterative when there is a loop (or repetition).

• A program is called recursive when an entity calls itself.

Iterative and recursive algorithm

• This program finds the factorial of a

number
• Iterative program
 Iterative and recursive algorithm
#include <stdio.h>
int fact(int n)
{
if (n==0 || n==1){ • This program finds the factorial of a
return 1 ;} number
else {
return n*fact(n-1);}
• Recursive program
}

int main()
{
int n;
printf(“enter the number”);
scanf(“%d”, &n);
printf(“%d”, fact(n));
return 0;
}
Recursion
Key components of a recursive algorithm
design
Randomized algorithm
• Deterministic algorithm: include simple steps and has predefined
input and a required output. In this case running time is a function of
input.
• Randomized algorithm: random bit is introduced in the algorithm
which is designed to produced random input.
input Deterministic algorithm output

input Randomized algorithm Output

Random number
Randomized algorithm

• In addition to the input the algorithm takes random number and

makes random choices during execution
• Behaviour of the algorithm can vary in different runs even on a fixed
input
• The running time and correctness of the output is controlled by the
random choice made by the algorithm and independent of input
Given an array with n elements A[1..n] . Let half of the array contain 0 and other half contain 1. find the
index that contains a 1. [0,1,1,1,1,,0,0,0,1,0,1,0]

Repeat 300 times:

Repeat: k=randomint(n)
k=randomint(n) if A[k]==1
if A[k]==1 return k
return k return “failed”

Does not gamble with correctness. Does not gambles with runtime
Gambles with running time Gambles with correctness

MONTE CARLO
LAS VEGAS
Advantages of randomized algorithm
• The first advantage is performance; randomized algo- rithms run
faster than the best-known deterministic algorithms for many
problems.
• The second advantage is that many randomized algorithms are
simpler to describe and implement than deterministic algorithms of
comparable performance.
Disadvantages of randomized algorithm
• The disadvantage is that there isn’t a guarantee that the problem will
be solved all
• upper time limit to obtain the solution can not be guaranteed.
Example: randomized quick sort
Objective: Partition around a random element.

• Running time is independent of the input order.

• No assumptions need to be made about the input distribution.

• No specific input elicits the worst-case behavior.

• The worst case is determined only by the output of a random-number

generator.
 exercises
• Prove that 1000n^2+1000n = O(n^2)
• Prove or disprove n^3 = O(n^2)
• Prove or disprove 5n+3 = O(n)
• Prove 2n+10 = O(n)
• Prove that 2n^2 + 5n + 6 is O(n^2)