DESIGN AND ANALYSIS OF ALGORITHMS

Module I – Chapter - II

Mr. K Bhargav
Asst. Professor
Dept. of CSE
 DIVIDE AND CONQUER

• In divide and conquer approach, Given a function to compute on n inputs the divide-and-
conquer strategy suggests splitting the inputs k distinct subsets , 1<k<=n , yielding k sub
problems.
• These sub problems must be solved, and then a method must be found to combine
solutions into a solution of the whole.
• Generally, divide-and-conquer algorithms have three parts −
• Divide the problem into a number of sub-problems that are smaller instances of the same
problem.
• Conquer the sub-problems by solving them recursively. If they are small enough, solve the
sub-problems as base cases.
• Combine the solutions to the sub-problems into the solution for the original problem.

O1
08/21/2021 DAA - Unit - I
 02
08/21/2021 DAA - Unit - I
 General method(Control Abstraction):
Algorithm DandC(p)
{
if( small (p)) return s(p);
else
{
divide(p) into smaller problems p1,p2,….pk,k>=1
Apply DC to each of the sub problems;
return combine(DC(p1),DC(p2),……..,DC(pk));
}
}
03
08/21/2021 DAA - Unit - I
 Recurrence Relation
Computing time of DAndC is described by recurrence relation.

g(n) if n small
T (n)  
T(n1)  T(n2)    T(nk)  f(n) otherwise

The Complexity of many divide-and-conquer algorithms is given by recurrence of the form.

T(1) if n  1
T (n)  
aT(n/b)  f(n) if n  1

Where a and b are known constants. We assume that T(1) is Known and n is power of b (i.e., n = bk )

04
08/21/2021 DAA - Unit - I
 Methods to solve Recurrence relation

• Substitution Method
• Master Method

05
08/21/2021 DAA - Unit - I
Substitution Method
• One of the methods for solving any such recurrence relation is called the substitution
method.
• In this method repeatedly makes substitutions for each occurrence of the function T in
the right-side until all such occurrences disappear.
Example1:
T(1)=2
T(n)= 2.T(n/2)+n
sol: T(1)=2 a=2 b=2 f(n)= n
T(n)=2.T(n/2)+n T(n/2)=2.T(n/(2*2))+n/2

=2.[2.T(n/(2*2))+n/2]+n
= 22.T(N/22 )+n+n
= 22.T(N/22 )+2n
06
 .
.
.
Kth = 2kT(n/2k)+kn n= 2k
=2kT(2k/2k)+kn
Log2n =log22k
=2k T(1)+kn
= 2k .2+kn Log2n = k
n
=2log2 .2+n.log2n
T(n)=n.2+n logn

07
08/21/2021 DAA - Unit - I
 Example2:

 0 n0
T ( n)  
c  T (n  1) n  0
• T(n) = c + T(n-1)
= c + [c + T(n-2)]
= 2.c + T(n-2)
= 2.c + [c + T(n-3)]
= 3.c + T(n-3)
.
.
nth = n.c + T(n-n) = n.c + T(0) =n.c+0
T(n) = n.c O8
08/21/2021 DAA - Unit - I
 Example 3:
 c n 1
 n
T (n)  2T
   c n 1
  2 

• T(n) = 2T(n/2) + c
=2[2T(n/(2*2)) + c] + c
=22T(n/22) + 2c + c
=22[2T(n/(22*2)) + c] + 3c
=23T(n/23) + 4c + 3c
=23T(n/23) + 7c
.
.
=2kT(n/2k) + (2k - 1)c

09
08/21/2021 DAA - Unit - I
 =2kT(n/2k) + (2k - 1)c
=2kT(2k/2k) + (2k - 1)c
=2kT(1) + (2k - 1)c
=2k .c + (2k - 1)c
=c(2k + 2k -1)
=c(2. 2k -1 )
=c (2.2log2n -1)
=c.(2n-1)
T(n)= (2n-1).c
10
08/21/2021 DAA - Unit - I
 Binary Search

MIN-MAX

Merge Sort

Quick-Sort

Selection

Strassen’s Matrix Multiplication

11
08/21/2021 DAA - Unit - I
 Linear Search
Linear search is a very simple search algorithm. In this type of search, a
sequential search is made over all items one by one. Every item is checked and if
a match is found then that particular item location is returned, otherwise the
search continues till the end of the data collection.
5 77 8 33 9 2
x=9
a[1] a[2] a[3] a[4] a[5] a[6]
i=1 i=2 i=3 i=4 i=5

12
08/21/2021 DAA - Unit - I
 Algorithm( a, n, x)
{
int i;
for i:=1 to n do
{
if(a[i]==x)
return i;
}
}
TC= Best case O(1)
AVG Case O(n/2)
Worst Case O(n)
13
08/21/2021 DAA - Unit - I
 Binary Search
• Given value and sorted array a[], find index i such that a[i] = value.

• List of elements in the array in non-decreasing order.

Example:

2 55 7 9 12 17 32 38 44

Target element = 5

14
08/21/2021 DAA - Unit - I
 mid= (lb+ub)/2
lb=1
a[1] 2

a[2] 5
mid
mid= (1+9)/2=5
a[3] 7

9 If(a[mid]=X)
a[4]
mid-1 return mid;
12 mid
a[5]
17 mid+1 If(a[mid]>X)
a[6] return Binsea(a, lb, mid-1)
32
a[7]
If(a[mid]<X)
38 return Binsea(a, mid+1, ub)
a[8]

a[9] 44 mid= (1+4)/2=2

ub=9
15
08/21/2021 DAA - Unit - I
 AlgorithmBinsrch( a, lb, ub, X)
// Given an array a[lb:ub] of elements in non decreasing
//if X is present return i such that X=a[i] ; else return 0
{
if (lb=ub) then //if Small(P)
{
if(X=a[lb]) then return lb;
else return 0;
}
else
{
mid=[(lb+ub)/2];
if(X=a[mid]) then return mid;
else if(X<a[mid]) then
return Binsrch(a, lb, mid-1, X);
else return Binsrch(a, mid+1, ub, X);
}
}
16
08/21/2021 DAA - Unit - I
 Time Complexity
 1 n 1

T ( n)  1.T  n   1 n 1
 

 2
• T(n) =T(n/2) + 1
=[T(n/(2*2)) + 1] + 1 T(n/2)=T(n/(2*2))+1
=T(n/22) + 2
=[T(n/(22*2)) + 1] + 2
=T(n/23) + 3
...
kth =T(n/2k) + k
= T(2k/2k) + k
=T(1)+k
= 1+k n= 2k
=1+log2n
T(n) =O(logn) Log2n =log22k
Best Case=O(1) Log2n = k

17
08/21/2021 DAA - Unit - I
 MIN-MAX Algorithm

• Find the maximum and minimum elements in a set of n elements.

Example:
25 532 7 44 52 9 88 64 33 7171

18
08/21/2021 DAA - Unit - I
 i=1 min=7 min=25
25
a[1] min=7
32 mid mid=(i+j)/2
a[2]
max=32
7 min=7
a[3] mid
max=32 mid=(1+10)/2=5
max=7
44 min=44
a[4]
mid=(1+5)/2=3
52
a[5]
mid max=52 max=52 mid=(6+10)/2=8
min=9
9
a[6] mid=(1+3)/2=2
min=9 min=9
a[7] 88
mid
max=88 mid=(6+8)/2=7
a[8] 64 min=64
mid max=88
max=64
a[9] 33 min=33

a[10] 71
max=88
max=71
j=10
19
08/21/2021 DAA - Unit - I
 Algorithm MaxMin( i, j, max, min)
{
if (i=j) then max:=min:=a[i]; //if Small(P)
else if(i=j -1) then // two elements
{
if(a[i]<a[j]) then
{
max:=a[j]; min:=a[i];
}
else
{
max:=a[i]; min:=a[j];
}
}
else
{
mid=[(i+j)/2];

20
08/21/2021 DAA - Unit - I
 // Solve the sub problems

MaxMin(i, mid, max, min);

MaxMin(mid+1, j, max1, min1);

// Combine the solutions

if(max<max1) then max:=max1;

if(min>min1) then min:=min1;

21
08/21/2021 DAA - Unit - I
 Time Complexity

Recurrence relation
 0 n 1

 1 n2
T (n)  
2.T  n   2 n2

 2
 •a=2 b=2 f(n)=2 n=2k
T(n/2)=2.T(n/(2*2))+2
• T(n) =2.T(n/2) + 2
=2.[2.T(n/(2*2)) + 2] + 2
=22 .T(n/22) + 22 + 2
=23 .T(n/23) + 23 +22 + 2
...
(k-1)th =2k-1T(n/2k-1) + 2k-1 +……..+23 +22 + 2
= 2k-1T(2k/2k-1) + 2k-1 +……..+23 +22 + 2
= 2k-1T((2k/2k ) *2) +
=2k-1T(2) + 22
08/21/2021 DAA - Unit - I
 
=2k-1T(2) +
= 2k-1T(2)+2k-2  =2n-2

=2k-1 .1+2k-2
= 2k /2 +2k-2 n= 2k

=2k (1/2 +1) -2 Log2n =log22

k

=2log2n (3/2)-2 Log2n = k

= nlog22 (3/2)-2
= 3/2(n)-2
T(n)=O(n)

23
 Merge Sort Algorithm

• It works on the principle of Divide and Conquer.

• Merge sort repeatedly breaks down a list into sub lists until each sub list
consists of a single element and merging those sub lists in a manner that
results into a sorted list.
Example:
22 531 12 72 44 33 19 56 66
71

24
08/21/2021 DAA - Unit - I
 a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9]
22 531 12 72 44 33 19 56 66

22 31 12 72 44 33 19 56 66

22 31 12 72 44 33 19 56 66 mid=(low+high)/2

22 31 12 72 44 33 19 56 66 mid=(1+9)/2=5

mid=(1+5)/2=3
22 31
44 72 19 33 56 66
mid=(1+3)/2=2
22 31 mid=(1+2)/2=1

19 33 56 66 mid=(4+5)/2=4
12 22 31
mid=(6+9)/2=7
12 22 31 44 72 mid=(6+7)/2=6
mid=(8+9)/2=8
12 519 2222 31 33 44 56 66 72

25
08/21/2021 DAA - Unit - I
 h=low j=mid+1 i=low
12 19 2
12
22 i 19
h j 33
i 22
31 j 56
h i 31
44 66 i 33
h j
72 i 44
h high
i 56
mid i 66
i 72

26
08/21/2021 DAA - Unit - I
Algorithm MergeSort( low, high)
{
if (low<high) then
{
mid=[(low+high)/2];
MergeSort(low,mid);
MergeSort(mid+1,high);
//Combine Solution
MergeSort(low,mid,high);
}
}
Algorithm Merge(low,mid,high)
//a[low:high] global array containing two sorted arrays
// a[low:mid] and a[mid+1:high]. The goal is to merge these arrays a[low:high]
{
h:=low; i:=low; j:=mid+1;
while((h ≤ mid)) and (j ≤ high)) do
{
if(a[h] ≤ a[j]) then
{
b[i]=a[h]; h+1;
}
else
{
b[i]=a[j]; j+1;
}
i=i+1;
}
if(h>mid) then
for k:=j to high do
{
b[i]:=a[k]; i:=i+1;
}
else
for k:=h to mid do
{
b[i]:=a[k]; i:=i+1;
}
}
 Time Complexity
 a n 1

T (n)  2.T  n   cn n  1
 

 2
• a=2 b=2 f(n)=cn n=2k
• T(n) =2.T(n/2) + cn
=2.[2.T(n/(2*2)) + cn/2] + cn T(n/2)=2.T(n/(2*2)) + cn/2
=22 .T(n/22) + cn+ cn
=22 .T(n/22) + 2cn
=23 .T(n/23) + 3cn
...
(k)th =2kT(n/2k) + kcn
= 2kT(2k/2k) + kcn
= 2kT(1) + kcn
= 2k (a) + kcn
n= 2k
=an+kcn
=an+ cn (log n) log2n =log22k
=O(n logn)
log2n = k
 Quick-Sort Algorithm
• Quick sort is a highly efficient sorting algorithm and is based on partitioning
of array of data into smaller arrays.
• It picks an element as pivot and partitions the given array.

• Example:
33 517 35 6 64 7 37 45 12 48
 i=1 i=2 i=3 j=9 j=10 j=10+1
a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10]
33
33 17 35 6 64 7 37 45 12 48

pivot

Left to Right search

i=3 i=4 i=5 j=6 j=7 j=8 j=9 for greater than
33 17 12 6 64 7 37 45 35 48 pivot element

Right to Left search

pivot for smaller than
pivot element
i=5
j=5 j=6 i=6
33 17 12 6 7 64 37 45 35 48 If(i < j) then
swap(a[i],a[j])

pivot j-1 j+1 high

i=1 If(i > j) then
7 17 12 6 33 64 37 45 35 48 swap(pivot,a[j])
33
Algorithm QuickSort( a,low, high)
{
int i, j, temp, pivot, flag=0;
if(low<high)
{
pivot=a[low];
i=low;
j=high+1;
while(flag==0)
{
i=i+1;
while(a[i] < pivot)
{ i:=i+1; }
j:=j-1;
while(a[j] > pivot)
{j:=j-1;}
 if(i < j)
{
temp:=a[i];
a[i]:=a[j];
a[j]:=temp;
}
else
flag:=1;
}
temp:=a[low];
a[low]:=a[j];
a[j]:=temp;
quicksort(a,low,j-1);
quicksort(a, j+1, high);
}
}
 Time Complexity
 0 n 1

T (n)  2.T  n   n n  1
 

 2
• a=2 b=2 f(n)=n n=2k
• T(n) =2.T(n/2) + n
=2.[2.T(n/(2*2)) + n/2] + n T(n/2)=2.T(n/(2*2)) + n/2
=22 .T(n/22) + n+ n
=22 .T(n/22) + 2n
=23 .T(n/23) + 3n
...
kth =2kT(n/2k) + kn
= 2kT(2k/2k) + kn
= 2kT(1) + kn n= 2k
= 2k (0) + kn
=kn log2n =log22k
=n. log2n
log2n = k
T(n)= O(n logn)
 Worst-case Time Complexity
If the input of the Quicksort is a sorted array.
5 7 12 17 20 27
55 7 12 17 20 27

 0 n 1
T ( n)  
T  n  1  n n 1
T(n) =T(n-1) + n
=[T(n-2)) + (n-1)] + n
T(n-1)=T(n-2) + n-1
=T(n-2) + (n-1)+ n
=T(n-3) + (n-2)+(n-1)+n
...
nth =T(n-n)+2+ …………… + (n-2)+(n-1)+n
=T(0)+2+ …………… + (n-2)+(n-1)+n
=0+2…………… + (n-2)+(n-1)+n
=2+ …………… + (n-2)+(n-1)+n +(1-1)
=1+ 2+ …………… + (n-2)+(n-1)+n -1=n(n+1)/2-1
T(n)= O(n2)
 Selection
• Determine Kth smallest element in a list or an array.
• Partition algorithm.

Algorithm Select(a, n, k)
{
low:=1; up:=n+1;
repeat
{
j:= Partition(a, low, up);
if(k=j) then return a[j];
else if(k<j) then up:=j-1;
else low:=j+1;
} until(false);
}
• Time Complexity Analysis: Worst case : O(n2)
AVG Case : O(n)
 Strassen’s Matrix Multiplication

A divide and conquer approach

• Divide each nxn matrix into four matrices of size (n/2)x(n/2):

• Computing all of requires 8 multiplications and 4 additions.
 b n2
• Therefore, the total running time is T ( n)  8.T  n / 2   an2 n  2

Conventional method , this solves to   n  .
3
•

C11  A11 B11  A12 B21

 A11 A12   B11 B12   C11 C12 
 .     C12  A11 B12  A12 B22
 A21 A22   B21 B22   C21 C22  C21  A21 B11  A22 B21
C
A B C22  A21 B12  A22 B22
 Strassen’s algorithm

• Define seven matrices of size (n/2)x(n/2) :

P= (A11+A22).(B11+B22)
Q= (A21+A22).B11
 A11 A12   B11 B12   C11 C12 
 .     R= A11.(B12-B22)
A A B B
 21  22   21  22   21 C C 22 
A B
C
S= A22.(B21-B11)
T= (A11+A12).B22
U= (A21-A22).(B11+B12)
V= (A12-A22).(B21+B22)
• The four (n/2)x(n/2) matrices Cij can be defined in terms of P,Q,R,S,T,U,V:
C11= P+S-T+V
C12= R+T
C21= Q+S
C22= P+R-Q+U
 C11= P+S-T+V
C12= R+T Running time? Each matrix Mi requires additions
C21= Q+S and subtractions but only one multiplication:
C22= P+R-Q+U 
T ( n)  
b n2 which solves to
  
 O n 2.807  O n 2.81 
7.T  n / 2   an2 n  2

• a=7 b=2 f(n)=an2 n=2k

• T(n) =7.T(n/2) + an2
=7.[7.T(n/(2*2)) + a(n/2)2] + n
=72 .T(n/22) + 7/4. an2 +an2
=73 .T(n/23) + (7/4)2. an2 +7/4.an2 + an2
=73 .T(n/23) + [(7/4)2+7/4 + 1] an2
...
kth =7kT(n/2k) + [(7/4)k-1 +…………..+(7/4)2+7/4 + 1] an2

= 2kT(2k/2k) + an2.(7/4)k
= 7k .T(2k/2k) + an2.(7/4)k n= 2k
= 7k .T(1)+ an2.(7/4)k
log2n =log22k
n n
= b.7log2 + an2.(7/4) log2 log2n = k

7 7/4
= b.nlog2 + an2.(n) log2

7
= b.nlog2 + a[nlog4+nlog7 -nlog4 ]

 O n 2.81 
7 7
=b.nlog2 +a.nlog2
7
= O(nlog2 )
= O(n2.81 )