Download as pdf or txt
Download as pdf or txt
You are on page 1of 78

The Sorting Problem

• Input:
▪ A sequence of n numbers a1, a2, . . . , an

• Output:
▪ A permutation (reordering) a1’, a2’, . . . , an’ of the

input sequence such that a1’ ≤ a2’ ≤ · · · ≤ an’

IT3103, Dept of IT,SIT, MUJ


1
Insertion Sort

• Idea: like sorting a hand of playing cards


▪ Start with an empty left hand and the cards facing
down on the table.
▪ Remove one card at a time from the table, and
insert it into the correct position in the left hand
o compare it with each of the cards already in the hand,
from right to left
▪ The cards held in the left hand are sorted
o these cards were originally the top cards of the pile on
the table
IT3103, Dept of IT,SIT, MUJ
2
Insertion Sort

To insert 12, we need to


make room for it by moving
first 36 and then 24.

IT3103, Dept of IT,SIT, MUJ


3
Insertion Sort

IT3103, Dept of IT,SIT, MUJ


4
Insertion Sort

IT3103, Dept of IT,SIT, MUJ


5
Insertion Sort
input array

5 2 4 6 1 3

at each iteration, the array is divided in two sub-arrays:


left sub-array right sub-array

sorted unsorted

IT3103, Dept of IT,SIT, MUJ


6
Insertion Sort

IT3103, Dept of IT,SIT, MUJ


7
INSERTION-SORT
Alg.: INSERTION-SORT(A) 1 2 3 4 5 6 7 8

for j ← 2 to n a1 a2 a3 a4 a5 a6 a7 a8
do key ← A[ j ]
ke y
// Insert A[ j ] into the sorted sequence A[1 . . j -1]
i←j- 1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i– 1
A[i + 1] ← key
• Insertion sort – sorts the elements in place

IT3103, Dept of IT,SIT, MUJ


8
Loop Invariant for Insertion Sort
Alg.: INSERTION-SORT(A)
for j ← 2 to n
do key ← A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]

i←j- 1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
Invariant: at the start of the for loop the elements in A[1 . . j-1]
are in sorted order
IT3103, Dept of IT,SIT, MUJ
9
Loop Invariant for Insertion Sort
• Initialization:
▪ Just before the first iteration, j =
2:
the subarray A[1 . . j-1] = A[1],
(the element originally in A[1]) –
is sorted

IT3103, Dept of IT,SIT, MUJ


10
Loop Invariant for Insertion Sort
• Maintenance:
▪ the while inner loop moves A[j -1], A[j -2], A[j -
3], and so on, by one position to the right until the
proper position for key (which has the value that
started out in A[j]) is found
▪ At that point, the value of key is placed into this
position.

IT3103, Dept of IT,SIT, MUJ


11
Loop Invariant for Insertion Sort
• Termination:
▪ The outer for loop ends when j = n + 1  j-1 = n
▪ Replace n with j-1 in the loop invariant:
o the subarray A[1 . . n] consists of the elements originally
in A[1 . . n], but in sorted order j-1 j

• The entire array is sorted!


Invariant: at the start of the for loop the elements in A[1 . . j-1]
are in sorted order
IT3103, Dept of IT,SIT, MUJ
12
An Example: Insertion Sort
30 10 40 20 i =  j =  key = 
A[j] =  A[j+1] = 
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
30 10 40 20 i=2 j=1 key = 10
A[j] = 30 A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
30 30 40 20 i=2 j=1 key = 10
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
30 30 40 20 i=2 j=1 key = 10
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
30 30 40 20 i=2 j=0 key = 10
A[j] =  A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
30 30 40 20 i=2 j=0 key = 10
A[j] =  A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=2 j=0 key = 10
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=3 j=0 key = 10
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=3 j=0 key = 40
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=3 j=0 key = 40
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=3 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=3 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=3 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=4 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 20 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 40 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 40 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 40 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 40 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 30 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 30 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 30 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 20 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


An Example: Insertion Sort
10 20 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
} Done!

IT3103, Dept of IT,SIT, MUJ


Insertion Sort
InsertionSort(A, n) { What is the precondition
for i = 2 to n { for this loop?
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

IT3103, Dept of IT,SIT, MUJ


Insertion Sort
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
} How many times will
} this loop execute?

IT3103, Dept of IT,SIT, MUJ


Analysis of Insertion Sort
INSERTION-SORT(A) cost times
for j ← 2 to n c1 n
do key ← A[ j ] c2 n-1
Insert A[ j ] into the sorted sequence A[1 . . j -1] 0 n-1
i←j-1 c4 n-1
while i > 0 and A[i] > key c5  n

c6 
n
do A[i + 1] ← A[i] j=2
(t j −1)
c7 
n
i←i–1 j=2
(t j −1)
A[i + 1] ← key c8 n-1
tj: # of times the while statement is executed at iteration j
T (n) = c1n + c 2 (n −1) + c 4 (n −1) + c 5  t j + c6 (t j −1)+c 7 (t j −1)+ c8 (n −1)
n n n

j=2 j=2 j=2


IT3103, Dept of IT,SIT, MUJ
44
Best Case Analysis
• The array is already sorted “while i > 0 and A[i] > key”

▪ A[i] ≤ key upon the first time the while loop test is run
(when i = j -1)
▪ tj = 1
• T(n) = c1n + c2(n -1) + c4(n -1) + c5(n -1) + c8(n-1) =
(c1 + c2 + c4 + c5 + c8)n + (c2 + c4 + c5 + c8)

= an + b = (n)

T (n) = c1n + c2 (n −1) + c4 (n −1) + c5  t j + c6  (tj −1)+ c 7 (t j −1)+ c8 (n −1)


n n n

j=2 j=2 j=2


IT3103, Dept of IT,SIT, MUJ
45
Worst Case Analysis
• The array is in reverse sorted order “while i > 0 and A[i] > key”
▪ Always A[i] > key in while loop test
▪ Have to compare key with all elements to the left of the j-th position
 compare with j-1 elements  tj = j

n(n +1) n(n +1)


n
=  j =
n n
using  j=
2
− we have:
j=1

n(n + 1)  n(n −1)


T (n) = c1n + c 2 (n − 1) + c 4 (n − 1) + c  −

= an2 + bn + c a quadratic function of n

• T(n) = (n2) order of growth in n2


T (n) = c1n + c2 (n −1) + c4 (n −1) + c5  t j + c6  (tj −1)+c 7  (tj −1)+ c8 (n −1)
n n n

IT3103, Dept of IT,SIT, MUJ


j=2 j=2
4j6=2
Insertion Sort - Summary

• Advantages
▪ Good running time for “almost sorted” arrays (n)
• Disadvantages
▪ (n2) running time in worst and average case
▪  n2/2 comparisons and exchanges

IT3103, Dept of IT,SIT, MUJ


47
Bubble Sort
• Idea:
▪ Repeatedly pass through the array
▪ Swaps adjacent elements that are out of order
i
1 2 3 n

8 4 6 9 2 3 1
j

• Easier to implement, but slower than Insertion


sort

IT3103, Dept of IT,SIT, MUJ


48
Example
8 4 6 9 2 3 1 1 8 4 6 9 2 3
i=1 j i=2 j

8 4 6 9 2 1 3 1 2 8 4 6 9 3
i=1 j i=3 j

8 4 6 9 1 2 3 1 2 3 8 4 6 9
i=1 j i=4 j

8 4 6 1 9 2 3 1 2 3 4 8 6 9
i=1 j i=5 j

8 4 1 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=6 j

8 1 4 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=7
j
1 8 4 6 9 2 3
i =IT3103,
CS11501,Dept
jDeptofofIT,SIT,
CSE,SCIT,
MUJ MUJ
49
Bubble Sort
Alg.: BUBBLESORT(A)
for i  1 to length[A]
do for j  length[A] downto i + 1
do if A[j] < A[j -1]
i then exchange A[j]  A[j-1]
8 4 6 9 2 3 1
i=1 j

IT3103,
CS1501,Dept
DeptofofIT,SIT, MUJ MUJ
CSE,SCIT,
50
Bubble-Sort Running Time
Alg.: BUBBLESORT(A)
for i  1 to length[A] c1
do for j  length[A] downto i + 1 c2
do if A[j] < A[j -1] c3
then exchange A[j]  A[j-1] c4
n

 (n
n n
T(n) = c1(n+1) + c2  (n − i c3  (n − i) + c4
i=1 i=1 i=
n
= (n) + (c2 + c2 + c4)  (n − i)
i=1
n n
+1)
n
n 2
n
where  (n − i) =  n −  i = n −
2 n(n = −
i=1 i=1 i=1 2 2 2
Thus,T(n) = (n2)
IT3103, Dept of IT,SIT, MUJ
51
Selection Sort
• Idea:
▪ Find the smallest element in the array
▪ Exchange it with the element in the first position
▪ Find the second smallest element and exchange it
with the element in the second position
▪ Continue until the array is sorted
• Disadvantage:
▪ Running time depends only slightly on the amount
of order in the file

IT3103, Dept of IT,SIT, MUJ


52
Example
8 4 6 9 2 3 1 1 2 3 4 9 6 8

1 4 6 9 2 3 8 1 2 3 4 6 9 8

1 2 6 9 4 3 8 1 2 3 4 6 8 9

1 2 3 9 4 6 8 1 2 3 4 6 8 9

IT3103, Dept of IT,SIT, MUJ


53
Selection Sort
Alg.: SELECTION-SORT(A)
8 4 6 9 2 3 1
n ← length[A]
for j ← 1 to n - 1
do smallest ← j
for i ← j + 1 to n
do if A[i] < A[smallest]
then smallest ← i
exchange A[j] ↔ A[smallest]

IT3103, Dept of IT,SIT, MUJ


54
Analysis of Selection Sort
cost times
Alg.: SELECTION-SORT(A)
c1 1
n ← length[A]
c2 n
for j ← 1 to n - 1
do smallest ← j c3 n-1

for i ← j + 1 to n c4 n−1
j=
(n −

do if A[i] < A[smallest] c5  n−1


(n

then smallest ← i c6 
n−1
j=1
(n − j)

exchange A[j] ↔ A[smallest] c7 n-1


n−1 n−1 n−1
T(n) = c1 + c2 n + c3 (n −1) + c 4  (n − j +1) +c 5  (n − j) +c 6  (n − j) + c7(n −1) = (n 2)
j=1 j=1 j=1
IT3103, Dept
CS1501, DeptofofIT,SIT, MUJ MUJ
CSE,SCIT,
55
Sorting in O(N) time

CS1501
Design & Analysis of Algorithm

56
IT3103, Dept of IT,SIT, MUJ
How Fast Can We Sort?

• Selection Sort, Bubble Sort, Insertion Sort: O(n2)

• Heap Sort, Merge sort: O(nlgn)

• Quicksort: O(nlgn) - average

• What is common to all these algorithms?


– Make comparisons between input elements

ai < aj, ai ≤ aj, ai = aj, ai ≥ aj, or ai > aj

IT3103, Dept of IT,SIT, MUJ


Lower-Bound for Sorting

• Theorem: To sort n elements, comparison sorts must


make (nlgn) comparisons in the worst case.

IT3103, Dept of IT,SIT, MUJ


Can we do better?

• Linear sorting algorithms


– Counting Sort

– Radix Sort

– Bucket sort

• Make certain assumptions about the data

• Linear sorts are NOT “comparison sorts”


IT3103, Dept of IT,SIT, MUJ
Counting Sort
• Assumptions:
– n integers which are in the range [0 ... r]
– r is in the order of n, that is, r=O(n)
• Idea:
– For each element x, find the number of elements  x
– Place x into its correct position in the output array

output array

IT3103, Dept of IT,SIT, MUJ


Step 1

(i.e., frequencies)

(r=6)

IT3103, Dept of IT,SIT, MUJ


Step 2

C (frequencies) Cnew (cumulative sums)

IT3103, Dept of IT,SIT, MUJ


Algorithm

• Start from the last element of A

• Place A[i] at its correct place in the output array

• Decrease C[A[i]] by one


1 2 3 4 5 6 7 8

A 2 5 3 0 2 3 0 3
0 1 2 3 4 5

Cnew 2 2 4 7 7 8
IT3103, Dept of IT,SIT, MUJ
Example
1 2 3 4 5 6 7 8 0 1 2 3 4 5

A 2 5 3 0 2 3 0 3 Cnew 2 2 4 7 7 8

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

B 3 B 0 3
0 1 2 3 4 5
0 1 2 3 4 5

Cnew 2 2 4 6 7 8 Cnew 1 2 4 6 7 8

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

B 0 3 3 B 0 2 3 3
0 1 2 3 4 5
0 1 2 3 4 5

Cnew 1 2 4 5 7 8 Cnew 1 2 3 5 7 8

IT3103, Dept of IT,SIT, MUJ


Example (cont.)
1 2 3 4 5 6 7 8

A 2 5 3 0 2 3 0 3

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

B 0 0 2 3 3 B 0 0 2 3 3 3 5
0 1 2 3 4 5 0 1 2 3 4 5

C 0 2 3 5 7 8 C 0 2 3 4 7 7

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

B 0 0 2 3 3 3 B 0 0 2 2 3 3 3 5
0 1 2 3 4 5

C 0 2 3 4 7 8

IT3103, Dept of IT,SIT, MUJ


COUNTING-SORT
1 j n

A
Alg.: COUNTING-SORT(A, B, n, k) 0 k

1. for i ← 0 to r C
2. do C[ i ] ← 0 1 n

B
3. for j ← 1 to n
4. do C[A[ j ]] ← C[A[ j ]] + 1
5. C[i] contains the number of elements equal to i
6. for i ← 1 to r
7. do C[ i ] ← C[ i ] + C[i -1]
8. C[i] contains the number of elements ≤ i
9. for j ← n downto 1
10. do B[C[A[ j ]]] ← A[ j ]
11. C[A[ j ]] ← C[A[ j ]] - 1
IT3103, Dept of IT,SIT, MUJ
Analysis of Counting Sort
Alg.: COUNTING-SORT(A, B, n, k)
1. for i ← 0 to r
O(r)
2. do C[ i ] ← 0
3. for j ← 1 to n
O(n)
4. do C[A[ j ]] ← C[A[ j ]] + 1
5. C[i] contains the number of elements equal to i
6. for i ← 1 to r
O(r)
7. do C[ i ] ← C[ i ] + C[i -1]
8. C[i] contains the number of elements ≤ i
9. for j ← n downto 1
10. do B[C[A[ j ]]] ← A[ j ] O(n)

11. C[A[ j ]] ← C[A[ j ]] - 1


Overall time: O(n + r)
IT3103, Dept of IT,SIT, MUJ
Analysis of Counting Sort
• Overall time: O(n + r)

• In practice we use COUNTING sort when r = O(n)

 running time is O(n)

IT3103, Dept of IT,SIT, MUJ


Radix Sort
• Represents keys as d-digit numbers in some
base-k
key = x1x2...xd where 0≤xi≤k-1

• Example: key=15

key10 = 15, d=2, k=10 where 0≤xi≤9

key2 = 1111, d=4, k=2 where 0≤xi≤1

IT3103, Dept of IT,SIT, MUJ


Radix Sort
• Assumptions
d=O(1) and k =O(n)
• Sorting looks at one column at a time
– For a d digit number, sort the least significant
digit first
– Continue sorting on the next least significant
digit, until all digits have been sorted
– Requires only d passes through the list

IT3103, Dept of IT,SIT, MUJ


RADIX-SORT
Alg.: RADIX-SORT(A, d)
for i ← 1 to d
do use a stable sort to sort array A on digit i
(stable sort: preserves order of identical elements)

IT3103, Dept of IT,SIT, MUJ


Analysis of Radix Sort

• Given n numbers of d digits each, where each

digit may take up to k possible values, RADIX-

SORT correctly sorts the numbers in O(d(n+k))

– One pass of sorting per digit takes O(n+k) assuming

that we use counting sort

– There are d passes (for each digit)

IT3103, Dept of IT,SIT, MUJ


Analysis of Radix Sort

• Given n numbers of d digits each, where each

digit may take up to k possible values, RADIX-

SORT correctly sorts the numbers in O(d(n+k))

– Assuming d=O(1) and k=O(n), running time isO(n)

IT3103, Dept of IT,SIT, MUJ


Bucket Sort
• Assumption:
– the input is generated by a random process that distributes
elements uniformly over [0, 1)
• Idea:
– Divide [0, 1) into k equal-sized buckets (k=Θ(n))
– Distribute the n input values into the buckets
– Sort each bucket (e.g., using quicksort)
– Go through the buckets in order, listing elements in each one

• Input: A[1 . . n], where 0 ≤ A[i] < 1 for all i

• Output: elements A[i] sorted

IT3103, Dept of IT,SIT, MUJ


Example - Bucket Sort
A 1 .78 B 0 /

2 .17 1 .17 .12 /

3 .39 2 .26 .21 .23 /


4 .26 3 .39 /

5 .72 4 / Distribute
6 .94 5 / Into buckets
7 .21 6 .68 /

8 .12 7 .78 .72 /

9 .23 8 /
10 .68 9 .94 /

IT3103, Dept of IT,SIT, MUJ


Example - Bucket Sort

0 /

1 .12 .17 /

2 .21 .23 .26 /

3 .39 /
Sort within each
4 /
bucket
5 /

6 .68 /

7 .72 .78 /

8 /

9 .94 /
IT3103, Dept of IT,SIT, MUJ
Example - Bucket Sort
.12 .17 .21 .23 .26 .39 .68 .72 .78 .94 /

0 /

1 .12 .17 /

2 .21 .23 .26 /

3 .39 /

4 /

5 / Concatenate the lists from


6 .68 / 0 to n – 1 together, in order

7 .72 .78 /

8 /

9 .94 /
IT3103, Dept of IT,SIT, MUJ
Analysis of Bucket Sort
Alg.: BUCKET-SORT(A, n)

for i ← 1 to n
O(n)
do insert A[i] into list B[nA[i]]

for i ← 0 to k - 1
k O(n/k log(n/k))
do sort list B[i] with quicksort sort =O(nlog(n/k)

concatenate lists B[0], B[1], . . . , B[n -1]


together in order O(k)

return the concatenated lists


O(n) (if k=Θ(n))
78
IT3103, Dept of IT,SIT, MUJ

You might also like