UNIT – I

Introduction - Algorithm Definition, Algorithm Specification – Pseudocode Conventions, Recursive

Algorithms, Performance Analysis- space Complexity, Time Complexity, Asymptotic Notations, Practical
Complexities and Performance Measurement.
Divide and Conquer: General Method, Binary Search, Finding Maximum and Minimum, Merge Sort,
Quick Sort, Strassen’s Matrix Multiplication.

Introduction
What is an algorithm?
• An algorithm is a finite set of instructions that, if followed, accomplishes a particular task
• All algorithms must satisfy the following criteria:
• Input. There are zero or more quantities that are externally supplied.
• Output. At least one quantity is produced.
• Definiteness. Each instruction is clear and unambiguous.
• Finiteness. If we trace out the instructions of an algorithm, then for all cases, the algorithm
terminates after a finite number of steps.
• Effectiveness. Every instruction must be basic enough to be carried out, in principle, by a
person using only pencil and paper.

Algorithm Specification
Algorithm can be described in three ways.
1. Natural language like English - we should ensure that each & every statement is definite.
2. Graphic representation called flowchart: - This method will work well when the algorithm is small&
simple.
3. Pseudo-code Method: - In this method, we should typically describe algorithms as program, which
resembles language like Pascal & algol.

Pseudocode Conventions:
1. Comments begin with // and continue until the end of line.
2. Blocks are indicated with matching braces {and}.
3. An identifier begins with a letter. The data types of variables are not explicitly declared. Compound
data types can be formed with records.
4. Assignment of values to variables is done using the assignment statement.
<Variable>:= <expression>;
5. There are two Boolean values TRUE and FALSE. Logical Operators used are AND, OR and NOT
Relational Operators <, <=,>,>=, =, != are used
6. Elements of multidimensional arrays are accessed using [ and ]
7. The following looping statements are employed.
For, while and repeat-until
While Loop:
While < condition > do
{
<statement-1>
.
.
<statement-n>
}
For Loop:
For variable: = value-1 to value-2 step step do
{
<statement-1>
.
.
.
<statement-n>
}
repeat-until:
repeat
<statement-1>
.
.
.
<statement-n>
until<condition>
8. A conditional statement has the following forms.
If <condition> then <statement>
If <condition> then <statement-1> else <statement-1>
Case statement:
Case
{
: <condition-1> : <statement-1>
.
.
.
: <condition-n> : <statement-n>
: else : <statement-n+1>
}
9. Input and output are done using the instructions read & write.
10. There is only one type of procedure: Algorithm. It consists of
heading and body. Heading takes the form,
Algorithm Name (Parameter lists)
As an example, the following algorithm finds & returns the maximum of “n‟ given numbers:
1. algorithm Max(A,n)
2. // A is an array of size n
3. {
4. Result := A[1];
5. for I:= 2 to n do
6. if A[I] > Result then
7. Result :=A[I];
8. return Result;
9. }

Recursive Algorithms
• A Recursive function is a function that is defined in terms of itself.
• Similarly, an algorithm is said to be recursive if the same algorithm is invoked in the body.
• An algorithm that calls itself is Direct Recursive.
• Algorithm “A” is said to be Indirect Recursive if it calls another algorithm which in turns calls “A”.
• The Recursive mechanism, are externally powerful, but even more importantly, many times they can
express an otherwise complex process very clearly. Or these reasons we introduce recursion here.
An example Program using Recursion
int main(void)
{
fun(3);
return 0;
}
void fun(int a)
{
printf(" %d ",a);
if(a>0)
fun(a-1);
printf(" %d ",a);
}
int main(void)
{
fun(3);
return 0;
}
OUTPUT: 3 2 1 0 0 1 2 3

Performance Analysis
• Performance analysis: a priori estimate
• Space complexity: amount of memory the algorithm needs to run to completion
• Time complexity: amount of computer time the algorithm needs to run to completion
• Performance measurement: a posteriori testing

Space Complexity
• The space required by an algorithm is equal to the sum of the following two components −
• A fixed part that is a space required to store certain data and variables, that are independent of the
size of the problem. For example, simple variables and constants used, program size, etc.
• A variable part is a space required by variables, whose size depends on the size of the problem. For
example, dynamic memory allocation, recursion stack space, etc.
• Space complexity S(P) of any algorithm P is S(P) = C + SP(I), where C is the fixed part and S(I) is the
variable part of the algorithm, which depends on instance characteristic I.

Space Complexity:
Time Complexity:
• The time T(p) taken by a program P is the sum of the compile time and the run time (execution
time)
• The compile time does not depend on the instance characteristics. Also, we may assume that a
compiled program will be run several times without recompilation. This rum time is denoted by
tp(instance characteristics).
• The number of steps any problem statement is assigned depends on the kind of statement.
• For example, comments à 0 steps.
Assignment statements is 1 steps.

[Which does not involve any calls to other algorithms]

Interactive statement such as for, while & repeat-until Control part of the statement.

We introduce a variable, count into the program statement to increment count with initial value 0.
Statement to increment count by the appropriate amount are introduced into the program.

This is done so that each time a statement in the original program is executes count is incremented by
the step count of that statement.
Algorithm:
Algorithm sum(a, n) {
s= 0.0;
count = count+1;
for I=1 to n do{
count =count+1;
s=s + a[I];
count=count+1;
}
count=count+1;
count=count+1;
return s;
}

1. If the count is zero to start with, then it will be 2n+3 on termination. So each invocation of sum
execute a total of 2n+3 steps.
2. The second method to determine the step count of an algorithm is to build a table in which we list the
total number of steps contributes by each statement.
o First determine the number of steps per execution (s/e) of the statement and the
total number of times (i.e., frequency) each statement is executed.
o By combining these two quantities, the total contribution of all statements, the
step count for the entire algorithm is obtained.

Statement Steps per Frequency Total

execution
 1. Algorithm Sum(a,n) 0 -- 0
2.{ 0 1 n+1 n 1 0
3. S=0.0; 1 - 1 n+1 n
4. for I=1 to n do 1 1
5. s=s+a[I]; 1 0
6. return s; 1
7. } 0

Total 2n+3

Complexity of Algorithms

The complexity of an algorithm M is the function f(n) which gives the running time and/or storage
space requirement of the algorithm in terms of the size ‘n’ of the input data. Mostly, the storage space
required by an algorithm is simply a multiple of the data size ‘n’.
Complexity shall refer to the running time of the algorithm.

The function f(n), gives the running time of an algorithm, depends not only on the size ‘n’ of the input
data but also on the particular data. The complexity function f(n) for certain cases are:
1. Best Case: The minimum possible value of f(n) is called the best case.
2. Average Case: The expected value of f(n).
3. Worst Case: The maximum value of f(n) for any key possible input.

Asymptotic Notation
Formal way notation to speak about functions and classify them
The following notations are commonly use notations in performance analysis and used to characterize
the complexity of an algorithm:
1. Big–OH (O) ,
2. Big–OMEGA (Ω),
3. Big–THETA (Θ)

Big – Oh Notation:

• Big - Oh notation is used to define the upper bound of an algorithm in terms of Time Complexity.
• Big - Oh notation always indicates the maximum time required by an algorithm for all input
values. That means Big - Oh notation describes the worst-case of an algorithm time complexity.
• The worst-case complexity of the algorithm is the function defined by the maximum number of
steps taken on any instance of size n.
• The function f(n) = O(g(n)) iff there exist positive constants c and n0 such that f(n) ≤ c*g(n) for
all n, n≥n0.
Big – Omega Notation:

• Big-Omega notation is used to define the lower bound of an algorithm in terms of Time
Complexity.
• That means Big-Omega notation always indicates the minimum time required by an algorithm
for all input values. That means Big-Omega notation describes the best case of an algorithm time
complexity.
• The best-case complexity of the algorithm is the function defined by the minimum number of
steps taken on any instance of size n.
• The function f(n) = Ω(g(n)) iff there exist positive constants c and n0 such that f(n) ≥ c*g(n) for
all n, n≥n0.

Big – Theta Notation:

• Big - Theta notation is used to define the average bound of an algorithm in terms of Time
Complexity.
• That means Big - Theta notation always indicates the average time required by an algorithm for
all input values. That means Big - Theta notation describes the average case of an algorithm time
complexity.
• The average-case complexity of the algorithm is the function defined by the average number of
steps taken on any instance of size n.
• The function f(n) = Ө(g(n)) iff there exist positive constants c1, c2 and n0 such that c1*g(n) ≤
f(n) ≤ c2*g(n) for all n, n≥n0.
(logn) is faster than O(n)
O(nlogn) is faster than O(n2) not as good as O(n)
O(n2) is faster than O(2n)

Divide & Conquer

The most well-known algorithm design strategy:
1. Divide the problem into two or more smaller subproblems.
2. Conquer the subproblems by solving them recursively.
3. Combine the solutions to the subproblems into the solutions for the original problem.
General Method
Divide_Conquer(problem P)
{
if Small(P) return S(P);
else {
divide P into smaller instances P1, P2, …,Pk, k≥1;
Apply Divide_Conquer to each of these subproblems;
return Combine(Divide_Conque(P1), Divide_Conque(P2),…, Divide_Conque(Pk));
}
Divide & Conquer Examples:
• Merge sort
• Quick sort
• Strassen’s Matrix Multiplication
Binary Search
• Binary Search is one of the fastest searching algorithms.
• It is used for finding the location of an element in a linear array.
• It works on the principle of divide and conquer technique.

Binary Search Algorithm can be applied only on Sorted arrays.

So, the elements must be arranged in-
• Either ascending order if the elements are numbers.
• Or dictionary order if the elements are strings.

To apply binary search on an unsorted array,

• First, sort the array using some sorting technique.
• Then, use binary search algorithm.

Binary Search Algorithm-

Consider-
• There is a linear array ‘a’ of size ‘n’.
• Binary search algorithm is being used to search an element ‘item’ in this linear array.
• If search ends in success, it sets loc to the index of the element otherwise it sets loc to -1.
• Variables beg and end keeps track of the index of the first and last element of the array or sub
array in which the element is being searched at that instant.
• Variable mid keeps track of the index of the middle element of that array or sub array in which
the element is being searched at that instant.
Then, Binary Search Algorithm is as follows-
1. Begin
2. Set beg = 0
3. Set end = n-1
4. Set mid = (beg + end) / 2
5. while ( (beg <= end) and (a[mid] ≠ item) ) do
6. if (item < a[mid]) then
7. Set end = mid - 1
8. else
9. Set beg = mid + 1
10. endif
11. Set mid = (beg + end) / 2
12. endwhile
13. if (beg > end) then
14. Set loc = -1
15. else
16. Set loc = mid
17. endif
18. End
Time Complexity:

Case Time Complexity

Best Case O(1)

Average Case O(logn)

Worst Case O(logn)

Finding Maximum and Minimum
the maximum and minimum element can be found by comparing each element and updating Max and
Min values as and when required. This approach is simple but it does (n – 1) comparisons for finding
max and the same number of comparisons for finding the min. It results in a total of 2(n – 1)
comparisons. Using a divide and conquer approach, we can reduce the number of comparisons.
Divide and conquer approach for Max. Min problem works in three stages.
▪ If a1 is the only element in the array, a1 is the maximum and minimum.
▪ If the array contains only two elements a1 and a2, then the single comparison between two
elements can decide the minimum and maximum of them.
▪ If there are more than two elements, the algorithm divides the array from the middle and
creates two subproblems. Both subproblems are treated as an independent problem and the
same recursive process is applied to them. This division continues until subproblem size
becomes one or two.
After solving two subproblems, their minimum and maximum numbers are compared to build the
solution of the large problem. This process continues in a bottom-up fashion to build the solution of a
parent problem.
Algorithm for Max-Min Problem
Algorithm DC_MAXMIN (A, low, high)
// Description : Find minimum and maximum element from array using divide and conquer approach
// Input : Array A of length n, and indices low = 0 and high = n - 1
// Output : (min, max) variables holding minimum and maximum element of array

if n == 1 then
return (A[1], A[1])
else if n == 2 then
if A[1] < A[2] then
return (A[1], A[2])
else
return (A[2], A[1])
else
mid ← (low + high) / 2
[LMin, LMax] = DC_MAXMIN (A, low, mid)
[RMin, RMax] = DC_MAXMIN (A, mid + 1, high)
if LMax > RMax then
// Combine solution
max ← LMax
else
max ← RMax
end
if LMin < RMin then
// Combine solution
min ← LMin
else
min ← RMin
end
return (min, max)
end

Complexity analysis
The conventional algorithm takes 2(n – 1) comparisons in worst, best and average case.
DC_MAXMIN does two comparisons to determine the minimum and maximum element and creates two
problems of size n/2, so the recurrence can be formulated as
T(n) = 0, if n = 1
T(n) = 1, if n = 2
T(n) = 2T(n/2) + 2, if n > 2
Let us solve this equation using interactive approach.
T(n) = 2T(n/2) + 2 … (1)
By substituting n by (n / 2) in Equation (1)
T(n/2) = 2T(n/4) + 2
⇒ T(n) = 2(2T(n/4) + 2) + 2
= 4T(n/4) + 4 + 2 … (2)
By substituting n by n/4 in Equation (1),
T(n/4) = 2T(n/8) + 2
Substitute it in Equation (1),
T(n) = 4[2T(n/8) + 2] + 4 + 2
= 8T(n/8) + 8 + 4 + 2
= 23 T(n/23) + 23 + 22 + 21
.
.
After k – 1 iterations

It can be observed that divide and conquer approach does only [(3n/2) – 2] comparisons compared to
2(n – 1) comparisons of the conventional approach.
For any random pattern, this algorithm takes the same number of comparisons.

Merge Sort
The merge sort algorithm works from “the bottom up”
• start by solving the smallest pieces of the main problem
• keep combining their results into larger solutions
• eventually the original problem will be solved
 8 3 2 9 7 1 54

8 3 2 9 7 1 5 4

8 3 2 9 71 5 4

8 3 2 9 7 1 5 4

3 8 2 9 1 7 4 5

2 3 8 9 1 4 5 7

1 2 3 4 5 7 8 9

Problem statement:
Given an array of elements, we want to arrange them in non-decreasing order.
Procedure:
Given a sequence of n elements a[1], a[2], ..……..a[n], the general idea is to imagine them split into two
sets a[1], a[2],…….a[(n/2)] and a[(n/2)+1],……a[n] each set is again individually sorted and the
resulting sorting sequences are merged to produce a single sorted sequence of n elements.

The recursive calls are given by :

Algorithm Mergesort (a, l, h)
{
If (l<h) // if it has more than one element
{
mid = [(l+h)/2];
Mergesort (a,l,mid);
Mergesort (a,mid+1,h);
Merge (a,l,mid,h);
}
}

Algorithm Merge (a ,l, mid, h){

i:=l;
j:=mid+1;
k:=l;
while (i≤mid)&&j≤h){
If (a[i]<a[j]) then{
b[k]: =a[i];
i++;
k++;
}
Else{
b[k]=a[j];
j++;
k++;
}
}
while(i≤mid){
b[k]: = a[i];
i++;
k++;
}
while (j≤h){
b[k]:=a[j];
j++;
k++;
}
for k:=l to h do
a[k]:=b[k];
}
Time complexity of merge sort:
T(n) = 2T(n/2)+cn
= 2(2T(n/4)+cn/2)+cn
= 4T(n/4)+2cn
= 4(2T(n/8)+cn/4)+2cn
= 8T(n/8)+3cn
.
.
.
.
=2kT(n/2k)+kcn = nT(1)+cn log n = an+ c nlog n = O(log n) [Best, Avg. & Worst cases]

Quick Sort
Given an array of n elements (e.g., integers): If array only contains one element, return
Else
• pick one element to use as pivot.
• Partition elements into two sub-arrays:
• Elements less than or equal to pivot
• Elements greater than pivot
• Quicksort two sub-arrays
• Return results
Example:
Consider: arr[] = {10, 80, 30, 90, 40, 50, 70}
• Indexes: 0 1 2 3 4 5 6
• low = 0, high = 6, pivot = arr[h] = 70
• Initialize index of smaller element, i = -1
• Traverse elements from j = low to high-1
• j = 0: Since arr[j] <= pivot, do i++ and swap(arr[i], arr[j])
• i=0
• arr[] = {10, 80, 30, 90, 40, 50, 70} // No change as i and j are same
• j = 1: Since arr[j] > pivot, do nothing

• j = 2 : Since arr[j] <= pivot, do i++ and swap(arr[i], arr[j])

• i=1
• arr[] = {10, 30, 80, 90, 40, 50, 70} // We swap 80 and 30
• j = 3 : Since arr[j] > pivot, do nothing // No change in i and arr[]
• j = 4 : Since arr[j] <= pivot, do i++ and swap(arr[i], arr[j])
• i=2
• arr[] = {10, 30, 40, 90, 80, 50, 70} // 80 and 40 Swapped

• j = 5 : Since arr[j] <= pivot, do i++ and swap arr[i] with arr[j]
• i=3
• arr[] = {10, 30, 40, 50, 80, 90, 70} // 90 and 50 Swapped

• We come out of loop because j is now equal to high-1.

• Finally we place pivot at correct position by swapping arr[i+1] and arr[high] (or pivot)
• arr[] = {10, 30, 40, 50, 70, 90, 80} // 80 and 70 Swapped
 • Now 70 is at its correct place. All elements smaller than 70 are before it and all elements greater
than 70 are after it.
• Since quick sort is a recursive function, we call the partition function again at left and right
partitions

• Again call function at right part and swap 80 and 90

Algorithm for Quick Sort:

Algorithm QUICKSORT(low, high)
{
if low < high then
{
j := PARTITION(a, low, high+1);
// j is the position of the partitioning element
QUICKSORT(low, j –1);
QUICKSORT(j + 1 , high);
}
}
Algorithm PARTITION(a, m, p)
{
V = a(m);
i = m;
j = p; // A (m) is the partition element
do
{
 loop i := i + 1 until a(i) ≥ v
loop j := j –1 until a(j) ≤ v
if (i < j) then SWAP (a, i, j)
} while (i >j);
a[m] := a[j];
a[j] := V; // the partition element belongs at position P
return j;
}
Algorithm SWAP(a, i, j)
{
P:=a[i];
a[i] := a[j];
a[j] := p;
}
Time Complexity
Name Best case Average Case Worst Case
Space Complexity
Bubble O(n) - O(n2) O(n)
Insertion O(n) O(n2) O(n2) O(n)
Selection O(n2) O(n2) O(n2) O(n)
Quick O(n log n) O(n log n) O(n2) O(n + log n)
Merge O(n log n) O(n log n) O(n log n) O(2n)
Heap O(n log n) O(n log n) O(n log n) O(n)

Comparison between Merge and Quick Sort:

➢ Both follows Divide and Conquer rule.
➢ Statistically both merge sort and quick sort have the same average case time i.e.,
O(n log n).
➢ Merge Sort Requires additional memory. The pros of merge sort are: it is a stable
sort, and there is no worst case (means average case and worst case time complexity
is same).
➢ Quick sort is often implemented in place thus saving the performance and
memory by not creating extra storage space.
➢ But in Quick sort, the performance falls on already sorted/almost sorted list if
the pivot is not randomized. Thus why the worst case time is O(n2).

Strassen’s Matrix Multiplication

Strassen has used some formulas for multiplying the two 2*2 dimension matrices where the number of
multiplications is seven, additions and subtractions are is eighteen, and in brute force algorithm, there
is eight number of multiplications and four addition.

When the order n of matrix reaches infinity, the utility of Strassen’s formula is shown by its asymptotic
superiority. For example, let us consider two matrices A and B of n*n dimension, where n is a power of
two. It can be observed that we can have four submatrices of order n/2 * n/2 from A, B, and their
product C where C is the resultant matrix of A and B.

Strassen’s Matrix multiplication can be performed only on square matrices where n is a power of 2.
Order of both of the matrices are n × n.
The procedure of Strassen’s matrix multiplication
Here is the procedure :
1. Divide a matrix of the order of 2*2 recursively until we get the matrix of order 2*2.
2. To carry out the multiplication of the 2*2 matrix, use the previous set of formulas.
3. Subtraction is also performed within these eight multiplications and four additions.
4. To find the final product or final matrix combine the result of two matrixes.

Formulas for Strassen’s matrix multiplication.

Following are the formulae that are to be used for matrix multiplication.
1. D1 = (a11 + a22) * (b11 + b22)
2. D2 = (a21 + a22)*b11
3. D3 = (b12 – b22)*a11
4. D4 = (b21 – b11)*a22
5. D5 = (a11 + a12)*b22
6. D6 = (a21 – a11) * (b11 + b12)
7. D7 = (a12 – a22) * (b21 + b22)

C00= d1 + d4 – d5 + d7
C01 = d3 + d5
C10 = d2 + d4
C11 = d1 + d3 – d2 – d6
Here, C00, C01, C10, and C11 are the elements of the 2*2 matrix.

Algorithm for Strassen’s matrix multiplication

The algorithm for Strassen’s matrix Multiplication is as follows:
Algorithm Strass(n, x, y, z)
begin
If n = threshold then compute
C = x * y is a conventional matrix.
Else
Partition a into four sub matrices a00, a01, a10, a11.
Partition b into four sub matrices b00, b01, b10, b11.
Strass ( n/2, a00 + a11, b00 + b11, d1)
Strass ( n/2, a10 + a11, b00, d2)
Strass ( n/2, a00, b01 – b11, d3)
Strass ( n/2, a11, b10 – b00, d4)
Strass ( n/2, a00 + a01, b11, d5)
Strass (n/2, a10 – a00, b00 + b11, d6)
Strass (n/2, a01 – a11, b10 + b11, d7)

C = d1+d4-d5+d7 d3+d5
d2+d4 d1+d3-d2-d6

end if
return (C)
end.
Implementation of Strassen’s matrix multiplication:
#include <stdio.h>
int main(){
int a[2][2],b[2][2],c[2][2],i,j;
int m1,m2,m3,m4,m5,m6,m7;
// Here we are scanning and printing the first matrix
printf("Enter the 4 elements of first matrix: ");
for(i=0;i<2;i++)
for(j=0;j<2;j++)
scanf("%d",&a[i][j]);
// Here we are scanning and printing the second matrix
printf("Enter the 4 elements of second matrix: ");
for(i=0;i<2;i++)
for(j=0;j<2;j++)
scanf("%d",&b[i][j]);
printf("\nThe first matrix is\n");
for(i=0;i<2;i++)
{
printf("\n");
for(j=0;j<2;j++)
printf("%d\t",a[i][j]);
}
printf("\nThe second matrix is\n");
for(i=0;i<2;i++){
printf("\n");
for(j=0;j<2;j++)
printf("%d\t",b[i][j]);
}
// Here we are applying the above mentioned formulae
m1= (a[0][0] + a[1][1])*(b[0][0]+b[1][1]);
m2= (a[1][0]+a[1][1])*b[0][0];
m3= a[0][0]*(b[0][1]-b[1][1]);
m4= a[1][1]*(b[1][0]-b[0][0]);
m5= (a[0][0]+a[0][1])*b[1][1];
m6= (a[1][0]-a[0][0])*(b[0][0]+b[0][1]);
m7= (a[0][1]-a[1][1])*(b[1][0]+b[1][1]);
c[0][0]=m1+m4-m5+m7;
c[0][1]=m3+m5;
c[1][0]=m2+m4;
c[1][1]=m1-m2+m3+m6;
// As we got the value of the elements, we now print them
printf("\n After performing multiplication \n");
for(i=0;i<2;i++){
printf("\n");
for(j=0;j<2;j++)
printf("%d\t",c[i][j]);
}
return 0;
}

O(nlog27) or O(n2.81)

***