Randomized Algorithms

Quick Sort

Md. Bakhtiar Hasan

Lecturer
Department of Computer Science and Engineering
Islamic University of Technology
Motivation

Insertion Sort, Selection Sort

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)
• Doesn’t work for objects

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)
• Doesn’t work for objects
Merge Sort

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)
• Doesn’t work for objects
Merge Sort
• Worst-case complexity: O(n log(n))

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)
• Doesn’t work for objects
Merge Sort
• Worst-case complexity: O(n log(n))
• Not in-place

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)
• Doesn’t work for objects
Merge Sort
• Worst-case complexity: O(n log(n))
• Not in-place→ Requires O(n) auxiliary space in the merge step

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)
• Doesn’t work for objects
Merge Sort
• Worst-case complexity: O(n log(n))
• Not in-place→ Requires O(n) auxiliary space in the merge step
Quick Sort

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)
• Doesn’t work for objects
Merge Sort
• Worst-case complexity: O(n log(n))
• Not in-place→ Requires O(n) auxiliary space in the merge step
Quick Sort
• Divide and Conquer

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)
• Doesn’t work for objects
Merge Sort
• Worst-case complexity: O(n log(n))
• Not in-place→ Requires O(n) auxiliary space in the merge step
Quick Sort
• Divide and Conquer
• In-place

MBH (CSE, IUT) Randomized Algorithms 2/8

Motivation

Insertion Sort, Selection Sort

• Worst-case complexity: O(n2 )
Radix Sort
• Worst-case complexity: O(n)
• Doesn’t work for objects
Merge Sort
• Worst-case complexity: O(n log(n))
• Not in-place→ Requires O(n) auxiliary space in the merge step
Quick Sort
• Divide and Conquer
• In-place
• O(n log(n)) runtime

MBH (CSE, IUT) Randomized Algorithms 2/8

Quick Sort

Divide

MBH (CSE, IUT) Randomized Algorithms 3/8

Quick Sort

Divide
• Pick a pivot element x in A

MBH (CSE, IUT) Randomized Algorithms 3/8

Quick Sort

Divide
• Pick a pivot element x in A
• Partition the array into subarrays

MBH (CSE, IUT) Randomized Algorithms 3/8

Quick Sort

Divide
• Pick a pivot element x in A
• Partition the array into subarrays

MBH (CSE, IUT) Randomized Algorithms 3/8

Quick Sort

Divide
• Pick a pivot element x in A
• Partition the array into subarrays

• Recurse on L and G

MBH (CSE, IUT) Randomized Algorithms 3/8

Quick Sort

Divide
• Pick a pivot element x in A
• Partition the array into subarrays

• Recurse on L and G
Combine: Trivial

MBH (CSE, IUT) Randomized Algorithms 3/8

Basic Quick Sort

Pivot x = A[1] or A[n]

MBH (CSE, IUT) Randomized Algorithms 4/8

Basic Quick Sort

Pivot x = A[1] or A[n]

• Partition: O(n) time

MBH (CSE, IUT) Randomized Algorithms 4/8

Basic Quick Sort

Pivot x = A[1] or A[n]

• Partition: O(n) time
Worst Case Time?

MBH (CSE, IUT) Randomized Algorithms 4/8

Basic Quick Sort

Pivot x = A[1] or A[n]

• Partition: O(n) time
Worst Case Time?
• Reverse Sorted Array

MBH (CSE, IUT) Randomized Algorithms 4/8

Basic Quick Sort

Pivot x = A[1] or A[n]

• Partition: O(n) time
Worst Case Time?
• Reverse Sorted Array
• T (n) = T (0) + T (n − 1) + O(n)

MBH (CSE, IUT) Randomized Algorithms 4/8

Basic Quick Sort

Pivot x = A[1] or A[n]

• Partition: O(n) time
Worst Case Time?
• Reverse Sorted Array
• T (n) = T (0) + T (n − 1) + O(n)
• T (n) = O(n2 )

MBH (CSE, IUT) Randomized Algorithms 4/8

Basic Quick Sort

Pivot x = A[1] or A[n]

• Partition: O(n) time
Worst Case Time?
• Reverse Sorted Array
• T (n) = T (0) + T (n − 1) + O(n)
• T (n) = O(n2 )
Works well on random inputs

MBH (CSE, IUT) Randomized Algorithms 4/8

Basic Quick Sort

Pivot x = A[1] or A[n]

• Partition: O(n) time
Worst Case Time?
• Reverse Sorted Array
• T (n) = T (0) + T (n − 1) + O(n)
• T (n) = O(n2 )
Works well on random inputs
• Shuffle the input in O(n) time before sorting

MBH (CSE, IUT) Randomized Algorithms 4/8

Variants

Select pivot x intelligently

MBH (CSE, IUT) Randomized Algorithms 5/8

Variants

Select pivot x intelligently

• Choose median as pivot

MBH (CSE, IUT) Randomized Algorithms 5/8

Variants

Select pivot x intelligently

• Choose median as pivot
I Guaranteed balanced partition

MBH (CSE, IUT) Randomized Algorithms 5/8

Variants

Select pivot x intelligently

• Choose median as pivot
I Guaranteed balanced partition
I Can be done in O(n) time

MBH (CSE, IUT) Randomized Algorithms 5/8

Variants

Select pivot x intelligently

• Choose median as pivot
I Guaranteed balanced partition
I Can be done in O(n) time
• T (n) = 2T n


2 + O(n) + O(n)

MBH (CSE, IUT) Randomized Algorithms 5/8

Variants

Select pivot x intelligently

• Choose median as pivot
I Guaranteed balanced partition
I Can be done in O(n) time
• T (n) = 2T n2 + O(n) + O(n)


• T (n) = O(n log(n))

MBH (CSE, IUT) Randomized Algorithms 5/8

Variants

Select pivot x intelligently

• Choose median as pivot
I Guaranteed balanced partition
I Can be done in O(n) time
• T (n) = 2T n2 + O(n) + O(n)


• T (n) = O(n log(n))
Select pivot x randomly

MBH (CSE, IUT) Randomized Algorithms 5/8

Variants

Select pivot x intelligently

• Choose median as pivot
I Guaranteed balanced partition
I Can be done in O(n) time
• T (n) = 2T n2 + O(n) + O(n)


• T (n) = O(n log(n))
Select pivot x randomly
• Expected runtime: O(n log(n))

MBH (CSE, IUT) Randomized Algorithms 5/8

Paranoid Quick Sort

Try to get a balanced partition

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps
Choose pivot x randomly

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps
Choose pivot x randomly
Perform the partition

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps
Choose pivot x randomly
Perform the partition
Repeat until |L| ≤ 43 |A| and |G | ≤ 43 |A|

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps
Choose pivot x randomly
Perform the partition
Repeat until |L| ≤ 43 |A| and |G | ≤ 43 |A|
Recurse

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps
Choose pivot x randomly
Perform the partition
Repeat until |L| ≤ 43 |A| and |G | ≤ 43 |A|
Recurse
Analysis

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps
Choose pivot x randomly
Perform the partition
Repeat until |L| ≤ 43 |A| and |G | ≤ 43 |A|
Recurse
Analysis

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps
Choose pivot x randomly
Perform the partition
Repeat until |L| ≤ 43 |A| and |G | ≤ 43 |A|
Recurse
Analysis

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps
Choose pivot x randomly
Perform the partition
Repeat until |L| ≤ 43 |A| and |G | ≤ 43 |A|
Recurse
Analysis

1
Probability of picking a good pivot: 2

MBH (CSE, IUT) Randomized Algorithms 6/8

Paranoid Quick Sort

Try to get a balanced partition

• Probabilistically
• Might take a few more steps
Choose pivot x randomly
Perform the partition
Repeat until |L| ≤ 43 |A| and |G | ≤ 43 |A|
Recurse
Analysis

1
Probability of picking a good pivot: 2
1
Expected repeats: p =2
MBH (CSE, IUT) Randomized Algorithms 6/8
Paranoid Quick Sort

Runtime Analysis

MBH (CSE, IUT) Randomized Algorithms 7/8

Paranoid Quick Sort

Runtime Analysis
n 3n



T (n) = T 4 +T 4 + Expected repeats × cn

MBH (CSE, IUT) Randomized Algorithms 7/8

Paranoid Quick Sort

Runtime Analysis
n 3n



T (n) = T 4
+T 4
+ Expected repeats × cn
n 3n
T (n) = T 4 +T 4 + 2cn

MBH (CSE, IUT) Randomized Algorithms 7/8

Paranoid Quick Sort

Runtime Analysis
n 3n



T (n) = T 4
+T 4
+ Expected repeats × cn
n 3n
T (n) = T 4 +T 4 + 2cn

2cn

2cn 6cn
4 4

2cn 6cn 6cn 18cn

16 16 16 16
.. .. .. .. .. .. ..
. . . . . . .

MBH (CSE, IUT) Randomized Algorithms 7/8

Paranoid Quick Sort

Runtime Analysis
n 3n



T (n) = T 4
+T 4
+ Expected repeats × cn
n 3n
T (n) = T 4 +T 4 + 2cn

2cn

2cn 6cn
4 4

2cn 6cn 6cn 18cn

16 16 16 16
.. .. .. .. .. .. ..
. . . . . . .

< 2cn

MBH (CSE, IUT) Randomized Algorithms 7/8

Paranoid Quick Sort

Runtime Analysis
n 3n



T (n) = T 4
+T 4
+ Expected repeats × cn
n 3n
T (n) = T 4 +T 4 + 2cn

2cn

2cn 6cn
4 4
O(log4 (2cn))
2cn 6cn 6cn 18cn
16 16 16 16
.. .. .. .. .. .. ..
. . . . . . .

< 2cn

MBH (CSE, IUT) Randomized Algorithms 7/8

Paranoid Quick Sort

Runtime Analysis
n 3n



T (n) = T 4
+T 4
+ Expected repeats × cn
n 3n
T (n) = T 4 +T 4 + 2cn

2cn

2cn 6cn
4 4
O(log4 (2cn)) O(log 4 (2cn))
3
2cn 6cn 6cn 18cn
16 16 16 16
.. .. .. .. .. .. ..
. . . . . . .

< 2cn

MBH (CSE, IUT) Randomized Algorithms 7/8

Paranoid Quick Sort

Runtime Analysis
n 3n



T (n) = T 4
+T 4
+ Expected repeats × cn
n 3n
T (n) = T 4 +T 4 + 2cn

2cn 2cn

2cn 6cn
4 4 2cn
O(log4 (2cn)) O(log 4 (2cn))
3
2cn 6cn 6cn 18cn
16 16 16 16 2cn
.. .. .. .. .. .. ..
. . . . . . .

< 2cn

MBH (CSE, IUT) Randomized Algorithms 7/8

Paranoid Quick Sort

Runtime Analysis
n 3n



T (n) = T 4
+T 4
+ Expected repeats × cn
n 3n
T (n) = T 4 +T 4 + 2cn

2cn 2cn

2cn 6cn
4 4 2cn
O(log4 (2cn)) O(log 4 (2cn))
3
2cn 6cn 6cn 18cn
16 16 16 16 2cn
.. .. .. .. .. .. ..
. . . . . . .

< 2cn

Total: 2cn × log 4 (2cn) = O(n log2 (n))

3

MBH (CSE, IUT) Randomized Algorithms 7/8

Reading

CLRS 7.0-7.4
Dasgupta 2.4
Erickson 1.5, 1.8

MBH (CSE, IUT) Randomized Algorithms 8/8