ADVANCED

ALGORITHMS &
DATA STRUCTURES
Lecture-03
Recursive Bubble Sort, Insertion Sort, Recursive Insertion
Sort, Merge Sort, Iterative Merge Sort, Quick Sort, Iterative
Quick Sort, Radix Sort, Bucket Sort

Lecturer: Karimzhan Nurlan Berlibekuly

nkarimzhan@gmail.com
Recursive Bubble Sort
Bubble Sort is the simplest sorting algorithm that
works by repeatedly swapping the adjacent elements
if they are in wrong order.
Recursive Bubble Sort
Following is iterative Bubble sort algorithm :

// Iterative Bubble Sort

bubbleSort(arr[], n)
{
for (i = 0; i < n-1; i++)

// Last i elements are already in place

for (j = 0; j arr[j+1])
swap(arr[j], arr[j+1]);
}
Recursive Bubble Sort
How to implement it recursively?
Recursive Bubble Sort has no
performance/implementation advantages, but can
be a good question to check one’s understanding of
Bubble Sort and recursion.

If we take a closer look at Bubble Sort algorithm, we

can notice that in first pass, we move largest element
to end (Assuming sorting in increasing order). In
second pass, we move second largest element to
second last position and so on.
Recursive Bubble Sort
Recursion Idea.
1. Base Case: If array size is 1, return.
2. Do One Pass of normal Bubble Sort. This pass
fixes last element of current subarray.
3. Recur for all elements except last of current
subarray.
 Recursive Bubble Sort
void bubbleSort(int arr[], int n) /* Function to print an array */
{ void printArray(int arr[], int n)
// Base case {
if (n == 1) for (int i=0; i < n; i++)
return; printf("%d ", arr[i]);
printf("\n");
// One pass of bubble sort. After }
// this pass, the largest element // Driver program to test above functions
// is moved (or bubbled) to end. int main()
for (int i=0; i<n-1; i++) {
if (arr[i] > arr[i+1]) int arr[] = {64, 34, 25, 12, 22, 11, 90};
swap(arr[i], arr[i+1]); int n = sizeof(arr)/sizeof(arr[0]);
bubbleSort(arr, n);
// Largest element is fixed, printf("Sorted array : \n");
// recur for remaining array printArray(arr, n);
bubbleSort(arr, n-1); return 0;
} }
Insertion Sort
Insertion sort is a simple sorting algorithm that works
the way we sort playing cards in our hands.
Algorithm
// Sort an arr[] of size n
insertionSort(arr, n)
Loop from i = 1 to n-1.
……a) Pick element arr[i]
and insert it into
sorted sequence arr[0…i-1]
Insertion Sort
Insertion Sort
• Another Example:
12, 11, 13, 5, 6
• Let us loop for i = 1 (second element of the array) to 4 (last element of the
array)
• i = 1. Since 11 is smaller than 12, move 12 and insert 11 before 12
11, 12, 13, 5, 6
• i = 2. 13 will remain at its position as all elements in A[0..I-1] are smaller than
13
11, 12, 13, 5, 6
• i = 3. 5 will move to the beginning and all other elements from 11 to 13 will
move one position ahead of their current position.
5, 11, 12, 13, 6
• i = 4. 6 will move to position after 5, and elements from 11 to 13 will move one
position ahead of their current position.
5, 6, 11, 12, 13
 Insertion Sort
/* Function to sort an array using insertion sort*/
void insertionSort(int arr[], int n) // A utility function to print an array of size n
{ void printArray(int arr[], int n)
int i, key, j; {
for (i = 1; i < n; i++) int i;
{ for (i = 0; i < n; i++)
key = arr[i]; cout << arr[i] << " ";
j = i - 1; cout << endl;
/* Move elements of arr[0..i-1], that are }
greater than key, to one position ahead /* Driver code */
of their current position */ int main()
while (j >= 0 && arr[j] > key) { int arr[] = { 12, 11, 13, 5, 6 };
{ int n = sizeof(arr) / sizeof(arr[0]);
arr[j + 1] = arr[j];
j = j - 1; insertionSort(arr, n);
} printArray(arr, n);
arr[j + 1] = key;
} return 0; }
}
 Insertion Sort
• Time Complexity: O(n*2)
• Auxiliary Space: O(1)
• Boundary Cases: Insertion sort takes maximum time to
sort if elements are sorted in reverse order. And it takes
minimum time (Order of n) when elements are already
sorted.
• Algorithmic Paradigm: Incremental Approach
• Sorting In Place: Yes
• Stable: Yes
• Online: Yes
• Uses: Insertion sort is used when number of elements is
small. It can also be useful when input array is almost
sorted, only few elements are misplaced in complete big
array.
Recursive Insertion Sort
Insertion sort is a simple sorting algorithm that works
the way we sort playing cards in our hands.
Below is an iterative algorithm for insertion sort
Algorithm
// Sort an arr[] of size n
insertionSort(arr, n)
Loop from i = 1 to n-1.
a) Pick element arr[i] and insert
it into sorted sequence arr[0..i-1]
Recursive Insertion Sort
How to implement it recursively?
Recursive Insertion Sort has no
performance/implementation advantages, but can
be a good question to check one’s understanding of
Insertion Sort and recursion.

If we take a closer look at Insertion Sort algorithm,

we keep processed elements sorted and insert new
elements one by one in the inserted array.
Recursive Insertion Sort
Recursion Idea.
1. Base Case: If array size is 1 or smaller, return.
2. Recursively sort first n-1 elements.
3. Insert last element at its correct position in sorted
array.
 Recursive Insertion Sort

// Recursive function to sort an array // in sorted array.

using
int last = arr[n-1];
// insertion sort int j = n-2;
void insertionSortRecursive(int arr[], int n)
{ /* Move elements of arr[0..i-1], that are
// Base case greater than key, to one position
if (n <= 1) ahead
return; of their current position */
while (j >= 0 && arr[j] > last)
// Sort first n-1 elements {
insertionSortRecursive( arr, n-1 ); arr[j+1] = arr[j];
j--;
// Insert last element at its correct }
position arr[j+1] = last; }
Merge Sort (Recursive)
 Merge Sort
Like QuickSort, Merge Sort is a 
Divide and Conquer algorithm. It divides input
array in two halves, calls itself for the two
halves and then merges the two sorted
halves. 

The merge() function is used for merging two

halves. The merge(arr, l, m, r) is key process
that assumes that arr[l..m] and arr[m+1..r] are
sorted and merges the two sorted sub-arrays
 Merge Sort
MergeSort(arr[], l, r)
If r > l
1. Find the middle point to divide the array into two
halves:
middle m = (l+r)/2
2. Call mergeSort for first half:
Call mergeSort(arr, l, m)
3. Call mergeSort for second half:
Call mergeSort(arr, m+1, r)
4. Merge the two halves sorted in step 2 and 3:
Call merge(arr, l, m, r)
Merge Sort
Merge Sort
// Merges two subarrays of arr[]. if (L[i] <= R[j]) arr[k] = R[j];
// First subarray is arr[l..m] { j++;
// Second subarray is arr[m+1..r] arr[k] = L[i]; k++;
void merge(int arr[], int l, int m, int r) i++; }
{ } }
int i, j, k; else
int n1 = m - l + 1; { /* l is for left index and r is right index
int n2 = r - m; arr[k] = R[j]; of the
j++; sub-array of arr to be sorted */
/* create temp arrays */ } void mergeSort(int arr[], int l, int r)
int L[n1], R[n2]; k++; {
} if (l < r)
/* Copy data to temp arrays L[] and {
R[] */ /* Copy the remaining elements of L[], // Same as (l+r)/2, but avoids
for (i = 0; i < n1; i++) if there overflow for
L[i] = arr[l + i]; are any */ // large l and h
for (j = 0; j < n2; j++) while (i < n1) int m = l+(r-l)/2;
R[j] = arr[m + 1+ j]; {
arr[k] = L[i]; // Sort first and second halves
/* Merge the temp arrays back into i++; mergeSort(arr, l, m);
arr[l..r]*/ k++; mergeSort(arr, m+1, r);
i = 0; // Initial index of first subarray }
j = 0; // Initial index of second merge(arr, l, m, r);
subarray /* Copy the remaining elements of }
k = l; // Initial index of merged R[], if there }
subarray are any */
while (j < n2)
{
while (i < n1 && j < n2)
{
Merge Sort
Time Complexity: Sorting arrays on different machines. Merge
Sort is a recursive algorithm and time complexity can be
expressed as following recurrence relation.
T(n) = 2T(n/2) + Ɵ(n)

The above recurrence can be solved either using Recurrence

Tree method or Master method. It falls in case II of Master
Method and solution of the recurrence is Ɵ(nLogn).

Time complexity of Merge Sort is Ɵ(nLogn) in all 3 cases (worst,

average and best) as merge sort always divides the array into
two halves and take linear time to merge two halves.
Merge Sort
Auxiliary Space: O(n)
Algorithmic Paradigm: Divide and Conquer
Sorting In Place: No in a typical implementation
Stable: Yes
/* Iterative mergesort function to sort arr[0...n-1] */ Merge Sort (Iterative)
void mergeSort(int arr[], int n)
{
int curr_size; // For current size of subarrays to be merged
// curr_size varies from 1 to n/2
int left_start; // For picking starting index of left subarray
// to be merged

// Merge subarrays in bottom up manner. First merge subarrays of

// size 1 to create sorted subarrays of size 2, then merge subarrays
// of size 2 to create sorted subarrays of size 4, and so on.
for (curr_size=1; curr_size<=n-1; curr_size = 2*curr_size)
{
// Pick starting point of different subarrays of current size
for (left_start=0; left_start<n-1; left_start += 2*curr_size)
{
// Find ending point of left subarray. mid+1 is starting
// point of right
int mid = Math.min(left_start + curr_size - 1, n-1);

int right_end = min(left_start + 2*curr_size - 1, n-1);

// Merge Subarrays arr[left_start...mid] & arr[mid+1...right_end]

merge(arr, left_start, mid, right_end);
}
}
}
 Merge Sort (Iterative) {
/* Function to merge the two haves arr[l..m] if (L[i] <= R[j])
and arr[m+1..r] of array arr[] */ {
void merge(int arr[], int l, int m, int r) arr[k] = L[i];
{ i++;
int i, j, k; }
int n1 = m - l + 1; else
int n2 = r - m; {
arr[k] = R[j];
/* create temp arrays */ j++;
int L[n1], R[n2]; }
k++;
/* Copy data to temp arrays L[] and R[] */ }
for (i = 0; i < n1; i++)
L[i] = arr[l + i]; /* Copy the remaining elements of L[], if
for (j = 0; j < n2; j++) there are any */
R[j] = arr[m + 1+ j]; while (i < n1)
{
/* Merge the temp arrays back into arr[k] = L[i];
arr[l..r]*/ i++;
i = 0; k++;
j = 0; }
k = l;
while (i < n1 && j < n2)
Merge Sort (Iterative) /* Function to print an array */
void printArray(int A[], int size)
/* Copy the remaining elements of R[], if there {
are any */ int i;
while (j < n2) for (i=0; i < size; i++)
{ printf("%d ", A[i]);
arr[k] = R[j]; printf("\n");
j++; }
k++;
} /* Driver program to test above functions */
} int main()
{
int arr[] = {12, 11, 13, 5, 6, 7};
Time complexity of above iterative int n = sizeof(arr)/sizeof(arr[0]);
function is same as recursive, i.e.,
Θ(nLogn). printf("Given array is \n");
printArray(arr, n);

mergeSort(arr, n);

printf("\nSorted array is \n");

printArray(arr, n);
return 0;
}
Quick Sort (Recursive)
Like Merge Sort, QuickSort is a Divide and Conquer
algorithm. It picks an element as pivot and
partitions the given array around the picked pivot.
 Quick Sort
There are many different versions of quickSort that
pick pivot in different ways.

1. Always pick first element as pivot.

2. Always pick last element as pivot (implemented below)
3. Pick a random element as pivot.
4. Pick median as pivot.
 Quick Sort
The key process in quickSort is partition().
Target of partitions is, given an array and an
element x of array as pivot, put x at its correct
position in sorted array and put all smaller
elements (smaller than x) before x, and put all
greater elements (greater than x) after x. All
this should be done in linear time
 Pseudo Code for recursive QuickSort function :

The key process in quickSort is partition().

Target of partitions is, given an array and an
element x of array as pivot, put x at its correct
position in sorted array and put all smaller
elements (smaller than x) before x, and put all
greater elements (greater than x) after x. All
this should be done in linear time
 Pseudo Code for recursive QuickSort function :

/* low --> Starting index, high --> Ending index */

quickSort(arr[], low, high)
{
if (low < high)
{
/* pi is partitioning index, arr[pi] is now
at right place */
pi = partition(arr, low, high);

quickSort(arr, low, pi - 1); // Before pi

quickSort(arr, pi + 1, high); // After pi
}
}
Quick Sort
 Quick Sort - Partition Algorithm
There can be many ways to do partition, // pivot (Element to be placed at right position
following pseudo code adopts the method pivot = arr[high];
given in CLRS book. The logic is simple, we
start from the leftmost element and keep i = (low - 1) // Index of smaller element
track of index of smaller (or equal to)
elements as i. While traversing, if we find a for (j = low; j <= high- 1; j++)
smaller element, we swap current element {
with arr[i]. Otherwise we ignore current // If current element is smaller than the
element. pivot
if (arr[j] < pivot)
/* This function takes last element as pivot, {
places i++; // increment index of smaller
the pivot element at its correct position in element
sorted swap arr[i] and arr[j]
array, and places all smaller (smaller than }
pivot) }
to left of pivot and all greater elements to swap arr[i + 1] and arr[high])
right return (i + 1)
of pivot */ }
partition (arr[], low, high)
{
 Quick Sort. Illustration of partition() :
arr[] = {10, 80, 30, 90, 40, 50, 70}
Indexes: 0 1 2 3 4 5 6
low = 0, high = 6, pivot = arr[h] = 70
Initialize index of smaller element, i = -1
Traverse elements from j = low to high-1
j = 0 : Since arr[j] <= pivot, do i++ and
swap(arr[i], arr[j])
i=0
arr[] = {10, 80, 30, 90, 40, 50, 70} // No change
as i and j
// are same
j = 1 : Since arr[j] > pivot, do nothing
// No change in i and arr[]
j = 2 : Since arr[j] <= pivot, do i++ and
swap(arr[i], arr[j])
i=1
arr[] = {10, 30, 80, 90, 40, 50, 70} // We swap
80 and 30
j = 3 : Since arr[j] > pivot, do nothing
// No change in i and arr[]
j = 4 : Since arr[j] <= pivot, do i++ and
swap(arr[i], arr[j])
i=2
arr[] = {10, 30, 40, 90, 80, 50, 70} // 80 and 40
Swapped
j = 5 : Since arr[j] <= pivot, do i++ and swap
arr[i] with arr[j]
i=3
arr[] = {10, 30, 40, 50, 80, 90, 70} // 90 and 50
Swapped
We come out of loop because j is now equal to
high-1. Finally we place pivot at correct
position by swapping
arr[i+1] and arr[high] (or pivot)
arr[] = {10, 30, 40, 50, 70, 90, 80} // 80 and 70
Swapped
Now 70 is at its correct place. All elements
smaller than 70 are before it and all elements
greater than 70 are after it.
Quick Sort
int partition (int arr[], int low, int high) void quickSort(int arr[], int low, int high)
{ {
int pivot = arr[high]; // pivot if (low < high)
int i = (low - 1); // Index of smaller element {
/* pi is partitioning index, arr[p] is now
for (int j = low; j <= high - 1; j++) at right place */
{ int pi = partition(arr, low, high);
// If current element is smaller than the
pivot // Separately sort elements before
if (arr[j] < pivot) // partition and after partition
{ quickSort(arr, low, pi - 1);
i++; // increment index of smaller quickSort(arr, pi + 1, high);
element }
swap(&arr[i], &arr[j]); }
}
}
swap(&arr[i + 1], &arr[high]);
return (i + 1);
}
Quick Sort (Iterative)
/* A[] --> Array to be sorted, // Set pivot element at its correct position
l --> Starting index, // in sorted array
h --> Ending index */ int p = partition(arr, l, h);
void quickSortIterative(int arr[], int l, int h)
{ // If there are elements on left side of
// Create an auxiliary stack pivot,
int stack[h - l + 1]; // then push left side to stack
if (p - 1 > l) {
// initialize top of stack stack[++top] = l;
int top = -1; stack[++top] = p - 1;
}
// push initial values of l and h to stack
stack[++top] = l; // If there are elements on right side of
stack[++top] = h; pivot,
// then push right side to stack
// Keep popping from stack while is not if (p + 1 < h) {
empty stack[++top] = p + 1;
while (top >= 0) { stack[++top] = h;
// Pop h and l }
h = stack[top--]; }
l = stack[top--]; }
Radix Sort
The 
lower bound for Comparison based sorting algorithm
 (Merge Sort, Heap Sort, Quick-Sort .. etc) is Ω(nLogn),
i.e., they cannot do better than nLogn.
Counting sort is a linear time sorting algorithm that sort
in O(n+k) time when elements are in range from 1 to k.
Radix Sort
What if the elements are in range from 1 to n2?
We can’t use counting sort because counting sort will
take O(n2) which is worse than comparison based sorting
algorithms. Can we sort such an array in linear time?
Radix Sort is the answer. The idea of Radix Sort is to do
digit by digit sort starting from least significant digit to
most significant digit. Radix sort uses counting sort as a
subroutine to sort.
 Radix Sort

The Radix Sort Algorithm

1) Do following for each digit i where i varies from least significant digit to the most
significant digit.
………….a) Sort input array using counting sort (or any stable sort) according to the i’th
digit.
Example:
Original, unsorted list:
170, 45, 75, 90, 802, 24, 2, 66
Sorting by least significant digit (1s place) gives: [*Notice that we keep 802 before 2,
because 802 occurred before 2 in the original list, and similarly for pairs 170 & 90 and
45 & 75.]

170, 90, 802, 2, 24, 45, 75, 66

Sorting by next digit (10s place) gives: [*Notice that 802 again comes before 2 as 802
comes before 2 in the previous list.]

802, 2, 24, 45, 66, 170, 75, 90

Sorting by most significant digit (100s place) gives:

2, 24, 45, 66, 75, 90, 170, 802

 Radix Sort

// The main function to that sorts arr[] of size n using

// Radix Sort
void radixsort(int arr[], int n)
{
// Find the maximum number to know number of digits
int m = getMax(arr, n);

// Do counting sort for every digit. Note that instead

// of passing digit number, exp is passed. exp is 10^i
// where i is current digit number
for (int exp = 1; m/exp > 0; exp *= 10)
countSort(arr, n, exp);
}
 Radix Sort // Change count[i] so that count[i] now
contains actual
// position of this digit in output[]
// A utility function to get maximum value in arr[]
for (i = 1; i < 10; i++)
int getMax(int arr[], int n)
count[i] += count[i - 1];
{
int mx = arr[0];
// Build the output array
for (int i = 1; i < n; i++)
for (i = n - 1; i >= 0; i--)
if (arr[i] > mx)
{
mx = arr[i];
output[count[ (arr[i]/exp)%10 ] - 1] =
return mx;
arr[i];
}
count[ (arr[i]/exp)%10 ]--;
}
// A function to do counting sort of arr[] according to
// the digit represented by exp.
// Copy the output array to arr[], so that arr
void countSort(int arr[], int n, int exp)
now
{
// contains sorted numbers according to
int output[n]; // output array
current digit
int i, count[10] = {0};
for (i = 0; i < n; i++)
arr[i] = output[i];
// Store count of occurrences in count[]
}
for (i = 0; i < n; i++)
count[ (arr[i]/exp)%10 ]++;
Bucket Sort
Bucket sort is mainly useful when input is uniformly distributed
over a range. For example, consider the following problem.
Sort a large set of floating point numbers which are in range from
0.0 to 1.0 and are uniformly distributed across the range. How do
we sort the numbers efficiently?
Bucket Sort
bucketSort(arr[], n)
1) Create n empty buckets (Or lists).
2) Do following for every array element arr[i].
.......a) Insert arr[i] into bucket[n*array[i]]
3) Sort individual buckets using insertion sort.
4) Concatenate all sorted buckets.
Bucket Sort
 // Function to sort arr[] of size n using bucket
Bucket Sort sort
void bucketSort(float arr[], int n)
{
// 1) Create n empty buckets
vector<float> b[n];

// 2) Put array elements in different buckets

for (int i=0; i<n; i++)
{
int bi = n*arr[i]; // Index in bucket
b[bi].push_back(arr[i]);
}

// 3) Sort individual buckets

for (int i=0; i<n; i++)
sort(b[i].begin(), b[i].end());

// 4) Concatenate all buckets into arr[]

int index = 0;
for (int i = 0; i < n; i++)
for (int j = 0; j < b[i].size(); j++)
arr[index++] = b[i][j];
}
 The End

t.me/ads_iitu