SE/SS * ZG 519

Data Structures and Algorithms Design

Divide and Conquer
Rev-1.0

BITS Pilani Prof Vineet Garg

Work Integrated Learning Program Bangalore Professional Development Center
Session Outline

• What are Divide and Conquer Algorithms?

• Few Classical Examples of Divide and Conquer
Algorithms:
– Defective Chessboard
– Binary Search
– Merge Sort
– Quick Sort
• Recursive formulation and Time Complexity of the
algorithms.

2
BITS Pilani, Pilani Campus
What is Divide and Conquer?

Divide and Conquer Algorithms decompose a problem

into several smaller independent instances, so that
solutions of the smaller instances, when combined,
provide the solution of the original bigger instance.

3
BITS Pilani, Pilani Campus
 Defective Chess Board
What is it?

• In our context, chessboard is an n x n square grid, where n is a power of 2.

• A defective chessboard is a chessboard that has only one defective square
out of n x n.
• Defect –different color than others squares.

Normal Square

Defective Square

1x1 2x2 4x4 8x8

4
BITS Pilani, Pilani Campus
Defective Chess Board
A Triomino

• A triomino is an L shaped object that can cover three

normal squares of a chessboard perfectly.
• If a square is defective, a triomino should not cover
that.
• A triomino has four orientations as shown below:

5
BITS Pilani, Pilani Campus
Defective Chess Board
Some Mathematics

• Total Squares in the chessboard = n x n = n2

• Defective Square = 1
• Normal Squares = (n2-1)
• Count of squares in a triomino = 3
• Triominoes required to cover normal squares (x) = (n2-1)/3.
x=5

x =1
x=0

1x1 2x2 4x4

6
BITS Pilani, Pilani Campus
Defective Chess Board
Problem Statement and Solution Approach

• Problem Statement: A chessboard with only one

defective square, needs to be covered completely
with triominoes, such that:
– Defective square remains uncovered.
– No normal square should be left uncovered.
– Triominoes should not fall outside the board.

• Will L shape triominoes suffice? How to go about

it?
• Solution Approach: A 8x8
chessboard is divided into 4
equal parts.
• One of these parts has the
defective square.
• Other three parts are made to
have one pseudo defective
square each placing a triomino
at their common corners.
• Approach is followed recursively
for each of the 4 x 4 parts. 7
BITS Pilani, Pilani Campus
Let us take one 4 x 4 Chessboard from
the previous 8 x 8 Chessboard

Similarly other 4x4 squares can also be covered!

BITS Pilani, Pilani Campus

Defective Chess Board
Recurrence Equation Formulation

• Let us say the chessboard size is 2k x 2k, and it takes T(k)

time to cover this chessboard.
– k=0 it is a 1x1 chessboard,
– k=1 it is a 2x2 chessboard,
– k=2 it is a 4x4 chessboard,
– k=3 it is a 8x8 chessboard, and so on….

• T (k) = d, if k = 0, where d is some constant

4.T(k-1)+c, otherwise, where c is constant overhead
(4 x time to cover 2k-1 x 2k-1 chessboard)

9
BITS Pilani, Pilani Campus
Defective Chess Board
Time Complexity – Through Recurrence Equation

T(k) = 4T(k-1) + c
= 4[4T(k-2) + c] + c
= 42 T(k-2) + 4c + c
= 42[4T(k-3) + c] + 4c + c
= 43 T(k-3) + 42c + 4c + c
=…
= 4k T(0) + 4k-1c + 4k-2c + ... + 42c + 4c + c for kth term
= 4k d + 4k-1c + 4k-2c + ... + 42c + 4c + c T(0) = d
= O(4k) = O (2kx2k)
= O (number of squares)
10
BITS Pilani, Pilani Campus
Binary Search
Divide and Conquer Approach

• A divide and conquer approach to search a key in a sorted list.

0 1 2 3 4 5 6 7 8
list = 10 20 30 40 50 60 70 80 90
low = 0 mid=(low + high)/2 = (0+8)/2 = 4 high= 8
1. In the beginning elements at low, mid and high indices are known in the sorted list.
2. If the key is to be searched = 50.
o Since key = list[mid], search terminates here.
3. If the key is to be searched = 70 (> 50 the mid element)
o New low = mid+1 = 4+1 = 5
o New high = high = 8 3 independent
o New mid = (low + high)/2 = (5+8)/2 = 6 cases.
4. If the key is to be searched = 30 (< 50 the mid element)
o New low = low = 0
o New high = mid-1 = 4-1 = 3
o New mid = (low + high)/2 = (0+3)/2 = 1
5. The steps 1, 2, and 3 are repeated as long as low <= high or the key is found.

11
BITS Pilani, Pilani Campus
 Binary Search
Examples
0 1
10 20
low = 0
• Find key = 80 from the list. •
• Iteration-1 • Iteration-1
– low = 0, high = 8
– mid = 4 and key > list[mid]
– So, new low = 5,high = 8
• Iteration-2
– low = 5, high = 8
– mid = 6 and key > list[mid]
– So, new low = 7,high = 8
• Iteration-3
– low = 7, high = 8
– mid = 7 and key = list[mid]
– Search terminates here.
2 3 4 5 6 7
30 40 50 60 70
mid=(low + high)/2 = (0+8)/2 = 4
Find key = 30 from the list.
– low = 0, high = 8
– mid = 4 and key < list[mid]
– So, new low = 0,high = 3
• Iteration-2
– low = 0, high = 3
– mid = 1 and key > list[mid]
– So, new low = 2,high = 3
• Iteration-3
– low = 2, high = 3
– mid = 2 and key = list[mid]
– Search terminates here.
8
80 90
high= 8
• Find key = 95 from the list.
• Iteration-1
– low = 0, high = 8
– mid = 4 and key > list[mid]
– So, new low = 5,high = 8
• Iteration-2
– low = 5, high = 8
–
–
mid = 6 and key > list[mid]
So, new low = 7,high = 8
• Iteration-3
– low = 7, high = 8
– mid = 7 and key > list[mid]
– So, new low = 8,high = 8
• Iteration-4
– low = 8, high = 8
– mid = 8 and key > list[mid]
– So, new low = 9,high = 8
– Since low > high now, search
terminates here. Key is not found!
12
BITS Pilani, Pilani Campus
Binary Search
Recurrence Equation & Time Complexity

T (n) = 1, if n = 1 -------------------------------------------------- (1)

1+ T(n/2), otherwise ------------------------------------(2)
(time to compare the mid element and search upper or lower half )

T(n) = 1 + T(n/2)
T(n/2) = 1 + T(n/4)
T(n/4) = 1 + T(n/8)
T(n/8) = 1 + T(n/16) Illustration:
Taking only last 4 expressions and adding on both the sides:
. T(8) + T(4) + T(2) + T(1) = 1+1+1+1 + T4 + T(2) + T(1), or
T(8) = 1 + T(4) T(8) = 1 + 1 + 1 + 1, or
T(8) = 1+ 3, or
T(4) = 1 + T(2) T(8) = 1 + log28
T(2) = 1 + T(1) Generally we can observe that:
T(1) =1 T(n) = 1 + log2n
Adding both sides,
T (n) = 1 + log2n
T(n) = O(log2n)
13
BITS Pilani, Pilani Campus
Exercise

• Write a C/C++/Java program for Binary Search to find

an integer key from a sorted list of integers using:
– Recursive Divide and Conquer Algorithm.
– Iterative Divide and Conquer Algorithm.

• In the Binary Search example, the list is sorted in

non-decreasing order. If the list is non-increasing
order, how the “low-high-mid” logic will change?

14
BITS Pilani, Pilani Campus
Merge Sort
Divide and Conquer Approach

• Initially an unsorted list is available. It has to be sorted.

• Partition the given list of n elements into two parts using array
indices. It is possible that two parts are not of equal size.
• Keep partitioning until it is no more possible.
• Merge the parts sorting them.

15
BITS Pilani, Pilani Campus
Merge Sort
Illustration
0 1 2 3 4 5 6
low = 0 38 27 43 3 9 82 10 high= 6

0 1 2 3 4 5 6
38 27 43 3 9 82 10
0 1 2 3 4 5 6
38 27 43 3 9 82 10
0 1 2 3 4 5 6
38 27 43 3 9 82 10
0 1 2 3 4 5 6
27 38 3 43 9 82 10
0 1 2 3 4 5 6
3 27 38 43 9 10 82
All of these are 0 1 2 3 4 5 6
Element movements in
the same array. 3 9 10 27 38 43 82
BITS Pilani, Pilani Campus
Merge Sort
Pseudo Code

// A list of elements – list

// Index of the first element = low
// Index of the last element = high
Algorithm mergeSort (list, low, high)
{
if (low >= high) return;
mid = (low + high)/2; How does merge
merges in a sorted
mergeSort (list, low, mid); order? Let us see that
mergeSort (list, mid+1, high); in the next slides.
merge(list, low, mid, high);
}
17
BITS Pilani, Pilani Campus
Merge Sort
How Merge Works continues...
0 1 2 3 4 5 6
3 27 38 43 9 10 82
0 1 2 3 4 5 6
merge 3 9 10 27 38 43 82

0 1 2 3 4 5 6
list 3 27 38 43 9 10 82
i= low mid j= mid+1 high

0 1 2 3 4 5 6
tempList 3
k

Copy the smaller of list[low] and list[mid+1]in tempList and

increase i or j and k 18
BITS Pilani, Pilani Campus
Merge Sort
How Merge Works continues...

0 1 2 3 4 5 6
list 3 27 38 43 9 10 82
low i mid j = mid+1 high

0 1 2 3 4 5 6
tempList 3 9
k

Copy the smaller of list[low] and list[mid+1]in tempList and

increase i or j and k

19
BITS Pilani, Pilani Campus
Merge Sort
How Merge Works continues...

0 1 2 3 4 5 6
list 3 27 38 43 9 10 82
low i mid mid+1 j high

0 1 2 3 4 5 6
tempList 3 9 10
k

Copy the smaller of list[low] and list[mid+1]in tempList and

increase i or j and k

20
BITS Pilani, Pilani Campus
Merge Sort
How Merge Works continues...

0 1 2 3 4 5 6
list 3 27 38 43 9 10 82
low i mid mid+1 j= high

0 1 2 3 4 5 6
tempList 3 9 10 27
k

Copy the smaller of list[low] and list[mid+1]in tempList and

increase i or j and k

21
BITS Pilani, Pilani Campus
Merge Sort
How Merge Works continues...

0 1 2 3 4 5 6
list 3 27 38 43 9 10 82
low i mid mid+1 j= high

0 1 2 3 4 5 6
tempList 3 9 10 27 38
k

Copy the smaller of list[low] and list[mid+1]in tempList and

increase i or j and k

22
BITS Pilani, Pilani Campus
Merge Sort
How Merge Works continues...

0 1 2 3 4 5 6
list 3 27 38 43 9 10 82
low i = mid mid+1 j= high

0 1 2 3 4 5 6
tempList 3 9 10 27 38 43
k

Copy the smaller of list[low] and list[mid+1]in tempList and

increase i or j and k - i will not increase beyond mid.

23
BITS Pilani, Pilani Campus
Merge Sort
How Merge Works concluded.

0 1 2 3 4 5 6
list 3 27 38 43 9 10 82
low i = mid mid+1 j= high

0 1 2 3 4 5 6
tempList 3 9 10 27 38 43 82
k

Copy the last remaining elements that cannot be compared

24
BITS Pilani, Pilani Campus
Merge Sort
Merge - Pseudo Code
Algorithm merge(list, low, mid, high)
{
i = low; j = mid+1; k = low;
tempList is a local variable list for this function;
while (i <= mid AND j <= high) {
if (list[i] < list[j]) {
tempList[k] = list[i]; i = i+1; k = k+1;
}
else {
tempList[k] = list[j]; j = j+1; k = k+1;
}
}
// Copy remaining elements of two parts – if there are any
while (i <= mid) {
tempList[k] = list[i]; k= k+1; i = i+1;
}
while (j <= high) {
tempList[k] = list[j]; k= k+1; j =j+1;
}
// Copy all elements from tempList to list – because tempList does not exist outside this function (local)
for (i = low to high) {
list[i] = tempList[i];
}
} 25
BITS Pilani, Pilani Campus
Exercise

• Why mergeSort is required? merge itself seems to

be doing the whole sorting.
– Try merge on the original list below. Do you get the sorted
list in the end?
0 1 2 3 4 5 6
38 27 43 3 9 82 10

• No. So, how the mergeSort logic is working?

26
BITS Pilani, Pilani Campus
Merge Sort
Recurrence Equation & Time Complexity

T (n) = 1, if n = 1 (no sorting required)------------------------------ (1)

2.T(n/2) + n, otherwise ---------------------------------------- (2)
(two parts and then merging effort of n elements)
Dividing both sides by n in equation (2): T( n ) T( n / 2 ) n
 
T (n)/n = T (n/2) / (n/2) + 1 n n/2 n
T (n/2)/n/2 = T (n/4) / (n/4) + 1
T (n/4)/n/4 = T (n/8) / (n/8) + 1
.........................
T (8)/8 = T (4) / 4 +1
T (4)/4 = T (2) / 2 +1
T (2)/2 = T(1) +1

Adding both sides,

T (n)/n = T (1) + log2n
T (n)/n = 1 + log2n
T (n) = n + n log2n
T (n) = O (n log2n)
27
BITS Pilani, Pilani Campus
 Quick Sort
Solution Approach

// A list of elements – list

// Index of the first element = low
// Index of the last element = high
Pivot - the central point, pin, or shaft on
which a mechanism turns or oscillates.
Algorithm quickSort (list, low, high)
{
select a random element in the list (the pivot)
partition the remaining elements in two segments left and right;
no element in the left segment is larger than the pivot;
no element in the right segment is smaller than the pivot;
// You might need to bring the pivot somewhere in the middle if not there. See illustration.

// Terminal condition will be when only one element left in the partition
// Please note – Quick Sort Algorithm does not suggest how to select the pivot.

quickSort (list, low, pivotIndex-1); // recursive call for the left segment
quickSort (list, pivotIndex+1, high); // recursive call for the right segment
// pivotIndex = the array index of the pivot

}
28
BITS Pilani, Pilani Campus
 Partitioning Example
Quick Sort

pivot = 44 0 1 2 3 4 5 6 7 8
low 44 75 23 43 55 12 64 77 33 high

up down

0 1 2 3 4 5 6 7 8
low 44 75 23 43 55 12 64 77 33 high

up down
Value on the left segment that Value on the right segment
is larger than the pivot. that is smaller than the pivot.

0 1 2 3 4 5 6 7 8
low 44 33 23 43 55 12 64 77 75 high

Values are swapped

up down 29
BITS Pilani, Pilani Campus
 Example Continues..
pivot = 44
0 1 2 3 4 5 6 7 8
low 44 33 23 43 55 12 64 77 75 high

up down

0 1 2 3 4 5 6 7 8
low 44 33 23 43 55 12 64 77 75 high

up down

0 1 2 3 4 5 6 7 8
low 44 33 23 43 55 12 64 77 75 high

up down
30
BITS Pilani, Pilani Campus
Example Continues..

pivot = 44 0 1 2 3 4 5 6 7 8
low 44 33 23 43 55 12 64 77 75 high

Value on the left segment that Value on the right segment

is larger than the pivot. up down that is smaller than the pivot.

0 1 2 3 4 5 6 7 8
low 44 33 23 43 12 55 64 77 75 high

up down

Values are swapped

31
BITS Pilani, Pilani Campus
Example Concluded.

pivot = 44
0 1 2 3 4 5 6 7 8
low 44 33 23 43 12 55 64 77 75 high

down up No element in the left

up and down markers have rolled over. segment is larger than the
pivot.
Swap down and pivot key values No element in the right
segment is smaller than the
pivot.

0 1 2 3 4 5 6 7 8
low 12 33 23 43 44 55 64 77 75 high

Recursive calls to left left segment right segment

and right segments
around pivot with the
same logic to get the
final sorted array. 32
BITS Pilani, Pilani Campus
Quick Sort: Pivot is in the middle
Recurrence Equation & Time Complexity

T (n) = 1, if n = 1 (no sorting required)------------------------------ (1)

2.T(n/2) + n, otherwise ---------------------------------------- (2)
(partition effort for n elements and repetition for two parts)

Dividing both sides by n in equation (2):

T (n)/n = T (n/2) / (n/2) + 1
T (n/2)/n/2 = T (n/4) / (n/4) + 1
T (n/4)/n/4 = T (n/8) / (n/8) + 1
......
T (2)/2 = T(1) +1
Same as Merge Sort? Why?
Adding both sides,
T (n)/n = T (1) + log2n
T (n)/n = 1 + log2n from equation (1)
T (n) = n + n log2n
T (n) = O (n log2n)
33
BITS Pilani, Pilani Campus
Exercise

• What is median-of-three pivot quick sort?

• Quick sort 9, 7, 5, 11, 12, 2, 14, 10, 3, 6 in ascending
order taking 6 as the first pivot.
• The worst case for quick sort is when after partitioning
one of the segments is always empty. The recursive
formulation is given by:
T (n) = 1, if n = 1 (no sorting required)
T(0) + T(n-1) + n, otherwise
(an empty segment, second segment with n-1 elements and time to partition)

Solve this recurrence equation and prove that t(n) = O(n2).

34
BITS Pilani, Pilani Campus
Interesting Reading

• Quick Sort on Cavium Multicore Processors.

• Comparison among sorting algorithms.

35
BITS Pilani, Pilani Campus
 Appendix
Quick Sort Partitioning Examples

36
BITS Pilani, Pilani Campus
Partitioning Example
Quick Sort

69 is selected as pivot and

brought to extreme left
for simpler operations
before partitioning.

Final partitioned array around 69

37
BITS Pilani, Pilani Campus
Partitioning Example
Quick Sort

38
BITS Pilani, Pilani Campus
Partitioning Example
Quick Sort

54 is randomly selected as a pivot.

39
BITS Pilani, Pilani Campus
Thank You

40
BITS Pilani, Pilani Campus