Sorting Algorithms

Bubble Sort
Merge Sort
Quick Sort
Randomized Quick Sort
Overview
 Bubble Sort

 Divide and Conquer

 Merge Sort

 Quick Sort

 Randomization
2
Sorting

 Sorting takes an unordered collection and makes it an

ordered one.

1 2 3 4 5 6

77 42 35 12 101 5

1 2 3 4 5 6
5 12 35 42 77 101

3
"Bubbling Up" the Largest
Element
 Traverse a collection of elements
 Move from the front to the end
 “Bubble” the largest value to the end using
pair-wise comparisons and swapping

1 2 3 4 5 6

77 42 35 12 101 5

4
"Bubbling Up" the Largest
Element
 Traverse a collection of elements
 Move from the front to the end
 “Bubble” the largest value to the end using
pair-wise comparisons and swapping

1 2 3 4 5 6
42Swap77 12 101
77 42 35 5

5
"Bubbling Up" the Largest
Element
 Traverse a collection of elements
 Move from the front to the end
 “Bubble” the largest value to the end using
pair-wise comparisons and swapping

1 2 3 4 5 6

42 35Swap35
77 77 12 101 5

6
"Bubbling Up" the Largest
Element
 Traverse a collection of elements
 Move from the front to the end
 “Bubble” the largest value to the end using
pair-wise comparisons and swapping

1 2 3 4 5 6

42 35 12Swap12
77 77 101 5

7
"Bubbling Up" the Largest
Element
 Traverse a collection of elements
 Move from the front to the end
 “Bubble” the largest value to the end using
pair-wise comparisons and swapping

1 2 3 4 5 6

42 35 12 77 101 5

No need to swap

8
"Bubbling Up" the Largest
Element
 Traverse a collection of elements
 Move from the front to the end
 “Bubble” the largest value to the end using
pair-wise comparisons and swapping

1 2 3 4 5 6

42 35 12 77 5 Swap101
101 5

9
"Bubbling Up" the Largest
Element
 Traverse a collection of elements
 Move from the front to the end
 “Bubble” the largest value to the end using
pair-wise comparisons and swapping

1 2 3 4 5 6

42 35 12 77 5 101

Largest value correctly placed

10
The “Bubble Up” Algorithm
index <- 1
last_compare_at <- n – 1

loop
exitif(index > last_compare_at)
if(A[index] > A[index + 1]) then
Swap(A[index], A[index + 1])
endif
index <- index + 1
endloop 11
 No, Swap isn’t built in.

Procedure Swap(a, b isoftype in/out Num)

t isoftype Num
t <- a
a <- b
b <- t
endprocedure // Swap

12
Items of Interest
 Notice that only the largest value is
correctly placed
 All other values are still out of order
 So we need to repeat this process

1 2 3 4 5 6

42 35 12 77 5 101

Largest value correctly placed

13
Repeat “Bubble Up” How Many
Times?
 If we have N elements…

 And if each time we bubble an element, we

place it in its correct location…

 Then we repeat the “bubble up” process N – 1

times.

 This guarantees we’ll correctly place all N

elements.

14
“Bubbling” All the Elements

1 2 3 4 5 6
42 35 12 77 5 101
1 2 3 4 5 6
35 12 42 5 77 101
1 2 3 4 5 6
N-1

12 35 5 42 77 101
1 2 3 4 5 6
12 5 35 42 77 101
1 2 3 4 5 6
5 12 35 42 77 101
15
Reducing the Number of
Comparisons
1 2 3 4 5 6
77 42 35 12 101 5
1 2 3 4 5 6
42 35 12 77 5 101
1 2 3 4 5 6
35 12 42 5 77 101
1 2 3 4 5 6
12 35 5 42 77 101
1 2 3 4 5 6
12 5 35 42 77 101
16
Reducing the Number of
Comparisons
 On the Nth “bubble up”, we only need to
do MAX-N comparisons.

 For example:
 This is the 4th “bubble up”

 MAX is 6

 Thus we have 2 comparisons to do

1 2 3 4 5 6
12 35 5 42 77 101

17
Putting It All Together
N is … // Size of Array

Arr_Type definesa Array[1..N] of Num

Procedure Swap(n1, n2 isoftype Num)

temp isoftype Num
temp <- n1
n1 <- n2
n2 <- temp
endprocedure // Swap
18
The Truth

 NOBODY EVER uses Bubble Sort (except for Ezra)

 NOBODY

 NOT EVER

 Because it is EXTREMELY INEFFICIENT

19
This is how it goes…

20
Comparison (semi-log y axis)

21
Let’s forget about Bubble Sort

22
 Divide and Conquer
1. Base Case, solve the problem directly
if it is small enough

1. Divide the problem into two or more

similar and smaller subproblems

1. Recursively solve the subproblems

1. Combine solutions to the subproblems

23
Divide and Conquer - Sort
Problem:
 Input: A[left..right] – unsorted array of integers

 Output: A[left..right] – sorted in non-decreasing

order

24
 Divide and Conquer - Sort
1. Base case
at most one element (left ≥ right), return
2. Divide A into two subarrays: FirstPart, SecondPart
Two Subproblems:
sort the FirstPart
sort the SecondPart

3. Recursively
sort FirstPart
sort SecondPart

4. Combine sorted FirstPart and sorted SecondPart

25
This reminds me of…
That’s easy. Mariah
Carey.

As a singing star, Ms.

Carey has perfected
the “wax-on” wave
motion--a clockwise
sweep of her hand
used to emphasize
lyrics.

The Maria
“Wax-on” Angle:
(,t) The Siren of Subquadratic Sorts 26
 How To Remember Merge Sort?
(,t)

Just as Mariah recursively moves

her hands into smaller circles, so
too does merge sort recursively We need two such recursions,
split an array into smaller one for each half of the split
27
segments. array.
Overview
 Divide and Conquer

 Merge Sort

 Quick Sort

 Randomization

28
 Merge Sort: Idea

Divide into
A FirstPart SecondPart
two halves
Recursively
sort
FirstPart SecondPart

Merge

A is sorted!

29
Merge Sort: Algorithm
Merge-Sort (A, left, right)
if left ≥ right return
else
middle ← b(left+right)/2
Merge-Sort(A, left, middle) Recursive Call

Merge-Sort(A, middle+1, right)

Merge(A, left, middle, right)

30
Merge-Sort: Merge
Sorted

A:

merge
Sorted Sorted
FirstPart SecondPart

A:

A[left] A[middle] A[right] 31

Merge-Sort: Merge Example

A: 2
5 3
5 7 28
15 8 30
1 4
6 5 14
10 6

L: R:
3 5 15 28 6 10 14 22

Temporary Arrays
32
Merge-Sort: Merge Example

A:
3
1 5 15 28 30 6 10 14

k=0

L: R:
3
2 15
3 28
7 30
8 6
1 10
4 14
5 22
6

i=0 j=0
33
Merge-Sort: Merge Example

A:
1 2
5 15 28 30 6 10 14

k=1

L: R:
3
2 5
3 15
7 28
8 6
1 10
4 14
5 22
6

i=0 j=1
34
Merge-Sort: Merge Example

A:
1 2 3 28 30
15 6 10 14

k=2

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=1 j=1
35
Merge-Sort: Merge Example

A:
1 2 3 4 6 10 14

k=3

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=2 j=1
36
Merge-Sort: Merge Example

A:
1 2 3 4 5 6 10 14

k=4

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=2 j=2
37
Merge-Sort: Merge Example

A:
1 2 3 4 5 6 10 14

k=5

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=2 j=3
38
Merge-Sort: Merge Example

A:
1 2 3 4 5 6 7 14

k=6

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=2 j=4
39
Merge-Sort: Merge Example

A:
1 2 3 4 5 6 7 8
14

k=7

L: R:
3
2 5
3 15
7 28
8 6
1 10
4 14
5 22
6

i=3 j=4
40
Merge-Sort: Merge Example

A:
1 2 3 4 5 6 7 8

k=8

L: R:
3
2 5
3 15
7 28
8 6
1 10
4 14
5 22
6

i=4 j=4
41
Merge(A, left, middle, right)
1. n1 ← middle – left + 1
2. n2 ← right – middle
3. create array L[n1], R[n2]
4. for i ← 0 to n1-1 do L[i] ← A[left +i]
5. for j ← 0 to n2-1 do R[j] ← A[middle+j]
6. k ← i ← j ← 0
7. while i < n1 & j < n2
8. if L[i] < R[j]
9. A[k++] ← L[i++]
10. else
11. A[k++] ← R[j++]
12. while i < n1
13. A[k++] ← L[i++]
14. while j < n2
15. A[k++] ← R[j++] n = n1+n2

Space: n
42
Time : cn for some constant c
Merge-Sort(A, 0, 7)
Divide
6 2 8 3 7 5
A: 6 2 8 4 3 7 5 1
4 1

43
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 3) , divide
3 7 5
A:
1

6 2 88
2
4 4

44
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 1) , divide
3 7 5
A:
1

8
4

6
6 2
2

45
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 0) , base case
3 7 5
A:
1

8
4

46
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 0), return
3 7 5
A:
1

8
4

6 2

47
Merge-Sort(A, 0, 7)
Merge-Sort(A, 1, 1) , base case
3 7 5
A:
1

8
4

48
Merge-Sort(A, 0, 7)
Merge-Sort(A, 1, 1), return
3 7 5
A:
1

8
4

6 2

49
Merge-Sort(A, 0, 7)
Merge(A, 0, 0, 1)
3 7 5
A:
1

8
4

2
6

50
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 1), return
3 7 5
A:
1

2 8
6 4

51
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 3) , divide
3 7 5
A:
1

2
6

8
8 4
4

52
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 2), base case
3 7 5
A:
1

2
6

53
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 2), return
3 7 5
A:
1

2
6

8 4

54
Merge-Sort(A, 0, 7)
Merge-Sort(A, 3, 3), base case
A:

2
6

55
Merge-Sort(A, 0, 7)
Merge-Sort(A, 3, 3), return
3 7 5
A:
1

2
6

8 4

56
Merge-Sort(A, 0, 7)
Merge(A, 2, 2, 3)
3 7 5
A:
1

2
6

4 8

57
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 3), return
3 7 5
A:
1

2
4 8
6

58
Merge-Sort(A, 0, 7)
Merge(A, 0, 1, 3)
3 7 5
A:
1

2 4 6
8

59
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 3), return
2 4 6 3 7 5
A:
8 1

60
Merge-Sort(A, 0, 7)
Merge-Sort(A, 4, 7)
A: 2 4 6
8

3 7 5
1

61
Merge-Sort(A, 0, 7)
Merge (A, 4, 5, 7)
A: 2 4 6
8

1 3 5
7

62
Merge-Sort(A, 0, 7)
Merge-Sort(A, 4, 7), return
A: 2 4 6 1 3 5
8 7

63
Merge-Sort(A, 0, 7)
Merge-Sort(A,
Merge(A, 0, 3, 0,
7)7), done!
A: 1 2 3 4 5 6 7
8

64
Merge-Sort Analysis
n cn

n/2 n/2 2 × cn/2 = cn

log n levels
n/4 n/4 n/4 n/4 4 × cn/4 = cn

n/2 × 2c = cn
2 2 2

Total: cn log n

• Total running time: (nlogn)

• Total Space:  (n)
65
Merge-Sort Summary
Approach: divide and conquer
Time
 Most of the work is in the merging
 Total time: (n log n)
Space:
 (n), more space than other sorts.

66
Overview
 Divide and Conquer

 Merge Sort

 Quick Sort

67
Quick Sort
 Divide:
 Pick any element p as the pivot, e.g, the first
element
 Partition the remaining elements into
FirstPart, which contains all elements < p
SecondPart, which contains all elements ≥ p

 Recursively sort the FirstPart and SecondPart

 Combine: no work is necessary since sorting

is done in place
68
Quick Sort
A: p
pivot Partition
FirstPart SecondPart

x<p p p≤x
Recursive call
Sorted Sorted
FirstPart SecondPart

x<p p p≤x

Sorted 69
Quick Sort
Quick-Sort(A, left, right)
if left ≥ right return
else
middle ← Partition(A, left, right)
Quick-Sort(A, left, middle–1 )
Quick-Sort(A, middle+1, right)
end if

70
 Partition
A: p

A: p x<p p≤x

A:
p x<p p p≤x

71
Partition Example

A: 4 8 6 3 5 1 7 2

72
Partition Example

i=0

A: 4 8 6 3 5 1 7 2

j=1

73
Partition Example

i=0

A: 4 8 6 3 5 1 7 2

j=1

74
Partition Example

i=0

A: 4 8 6 3 5 1 7 2

j=2

75
Partition Example

i=0i=1

A: 4 8
3 6 3
8 5 1 7 2

j=3

76
Partition Example

i=1

A: 4 3 6 8 5 1 7 2

j=4

77
Partition Example

i=1

A: 4 3 6 8 5 1 7 2

j=5

78
Partition Example

i=2

A: 4 3 1
6 8 5 6
1 7 2

j=5

79
Partition Example

i=2

A: 4 3 1 8 5 6 7 2

j=6

80
Partition Example

i=2i=3

A: 4 3 1 2
8 5 6 7 8
2

j=7

81
Partition Example

i=3

A: 4 3 1 2 5 6 7 8

j=8

82
Partition Example

i=3

A: 24 3 1 4
2 5 6 7 8

83
Partition Example

pivot in
correct position

A: 2 3 1 4 5 6 7 8

x<4 4≤x

84
Partition(A, left, right)
1. x ← A[left]
2. i ← left
3. for j ← left+1 to right
4. if A[j] < x then
5. i ← i + 1
6. swap(A[i], A[j])
7. end if
8. end for j
9. swap(A[i], A[left])
10. return i
n = right – left +1
Time: cn for some constant c
85
Space: constant
Quick-Sort(A, 0, 7)
Partition
5 6 7
A: 2
4 3
8 1
6 34 5 1 7 2
8

86
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 2) , partition
5 6 7
A: 4
8

2 2
3
1 3
1

87
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 0) , base
returncase
5 6 7
4
8

2 3
1

88
Quick-Sort(A, 0, 7)
Quick-Sort(A, 1, 1) , base case
5 6 7
4
8

2
1

89
Quick-Sort(A, 0, 7)
Quick-Sort(A,2,
Quick-Sort(A, 0,2),
2),return
return
2 5 6 7
1 3 4
8

2
1 3

90
Quick-Sort(A, 0, 7)
Quick-Sort(A,2,
Quick-Sort(A, 4,2),
7) ,return
partition
2
1 3 4

5 6 7
5
8 8

91
Quick-Sort(A, 0, 7)
Quick-Sort(A, 5, 7) , partition
2
1 3 4

6 7
6
8 8

92
Quick-Sort(A, 0, 7)
Quick-Sort(A, 6, 7) , partition
2
1 3 4

7
7 8
8

93
Quick-Sort(A, 0, 7)
returncase
Quick-Sort(A, 7, 7) , base
2
1 3 4

7 8

94
Quick-Sort(A, 0, 7)
Quick-Sort(A, 6, 7) , return
2
1 3 4

6 7 8

95
Quick-Sort(A, 0, 7)
Quick-Sort(A, 5, 7) , return
2
1 3 4

5 6 7 8

96
Quick-Sort(A, 0, 7)
Quick-Sort(A, 4, 7) , return
2
1 3 4 5 6 7 8

97
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 7) , done!
2
1 3 4 5 6 7 8

98
Quick-Sort: Best Case
 Even Partition
n cn

n/2 n/2 2 × cn/2 = cn

log n levels
n/4 n/4 n/4 n/4 4 × c/4 = cn

n/3 × 3c = cn
3 3 3

Total time: (nlogn)

99
Quick-Sort: Worst Case
 Unbalanced Partition
n cn

n-1 c(n-1)

n-2 c(n-2)

3c
3
Happens only if
 input is sortd
2 2c
 input is reversely sorted
Total time: (n2)

100
Randomized Quick Sort

101
 Quick-Sort: an Average Case
 Suppose the split is 1/10 : 9/10
n cn

0.1n 0.9n cn

log10n 0.01n 0.09n 0.09n 0.81n

cn

log10/9n
2
≤cn

2 ≤cn
Total time: (nlogn) 102
Quick-Sort Summary
 Time
 Most of the work done in partitioning.
 Average case takes (n log(n)) time.
 Worst case takes (n2) time
Space
 Sorts in-place, i.e., does not require additional
space

103
Summary
 Divide and Conquer
 Merge-Sort
 Most of the work done in Merging
 (n log(n)) time
 (n) space

 Quick-Sort
 Most of the work done in partitioning
 Average case takes (n log(n)) time
 Worst case takes (n2) time
 (1) space
104
Quiz 6 (Show output)
public static void main(String[ ] args) {
int[ ] array = {4,18,16,3,5,21,7};
recursiveQuickSort(array, 0, array.length-1);
System.out.println("The following array should be sorted: ");
printList(array);
System.exit(0);
}

public static void recursiveQuickSort(int[ ] list, int first, int last) {

if(first < last)
{
int p = partition(list, first, last);
printList(list);
recursiveQuickSort(list, first, p-1);
recursiveQuickSort(list, p+1, last);
}
}

Assume that printlist(list) prints the list separated by commas.

105