Professional Documents
Culture Documents
Sorts
Sorts
Sal F. Gambino
CS-342
C++ CS 342
Data Structures
&
Running time, Sorts 1
Running time analysis:
•consists of reasoning about
SFG
an algorithm speed Sal F. Gambino
CS-342
What to analyze:
•Factors to consider:
•compiler-------------------->not considered
•computer used----------->not considered
•algorithm
•input of the algorithm
typically the size of the input is the
main consideration
2
Running time analysis
SFG
Sal F. Gambino
What to analyze:Some general rules
CS-342
•LOOPS:
The running time of a loop is at most the
running time of the statements inside the
loop times the number of iterations
3
Running time analysis
SFG
What to analyze:Some general rules
Sal F. Gambino
•IF/ELSE: CS-342
IF (condition)
S1
ELSE
S2 6
Running time analysis
SFG
Big O notation
Sal F. Gambino
O(n2) O( n log n )
O(n)
time
O( log n )
# of elements
7
Running time analysis
SFG
Sal F. Gambino
Best, Average, and Worst case:
CS-342
for any particular value of n, the
number of required operations can
differ depending on the user’s input
Worst case:
•counting the maximum number of operations
Average case:
•determines the average numbers of operations
required for a given input
Best case:
•determines the fewest number of operations
required for a given n 8
Sorts SFG
Sal F. Gambino
CS-342
Quadratic: n lg n:
•Selection sort •Merge sort
•Insertion sort •Quick sort
•Heap sort
9
Quadratic Sorts: Selection sort
SFG
Sal F. Gambino
O(n2) CS-342
O(n2) CS-342
11
Quadratic Sorts: Selection sort SFG
for ( i =n-1; i > 0; i--) O(n2) Sal F. Gambino
{
CS-342
index_of_largest = 0; Find the
largest = data[ 0 ] Largest
for (j= 1; j <= i; ++j) element
{
if (data[ j ] > largest) O(n)
{ lartgest = data[ j]; O(n)
And
index_of_largest = j;
Swap It
}
with
}
ith
swap (data[i], data[index_of_largest]);
element
} 12
Quadratic Sorts: Insertion sort O(n2)
SFG
Sal F. Gambino
13
Quadratic Sorts: Insertion sort O(n2) SFG
for ( i =1; i > n; i++)
Sal F. Gambino
save the current element
{
CS-342
Current = data[i ];
Find the location j where
for (j= 0; j < i; j++) it should be inserted
{ if (data[ j ] >= Current)
Shift all elements
break;
between j and i
}
one place to the right
for ( int k = i-1; k >= j; k- -)
data[k+1] = data[k];
Inserts the current
element
data[j] = Current;
where it belongs:
} 14
Summary of Quadratic Sorts:
SFG
Sal F. Gambino
O(n2) CS-342
16
Merge SORT O(n lg n ) SFG
The Merge sort closely follows the Sal F. Gambino
Divide and Conquer sorting algorithm CS-342
MergeSort(A,p,r)
if p < r
0 1 2 3 4 5
then q = (p+r)/2
MergeSort(A,p,q) A
MergeSort(A,q+1,r) p q q+1 r
Merge(A,p,q,r) 17
38 16 27 39 12 27 SFG
38 16 27 39 12 27 Sal F. Gambino
lg n CS-342
38 16 27 39 12 27 Recursive
calls
38 16 39 12 to
Done!! MergeSort
16 38 12 39
lg n-1
Merge
16 27 38 12 27 39
steps
12 16 27 27 38 39 Temporary
18
array
Running time:
•the worst-case, average-case, and best-case
SFG
running time are all O(n lg n ) Sal F. Gambino
•makes O(n) calls to merge function CS-342
•merge sort is faster than O(n2) Selection sort
and Insertion sort
•it has a disadvantage of needing to allocate
a second array to use in the merging process
n n log2 n n2
2 2 4
Merge SORT
8 24 64
32 160 1024
128 896 16,384 O(n lg n )
512 4608 262,144
2048 22,528 4,194,304
19
QUICKSORT O(n lg n ) SFG
Sal F. Gambino
CS-342
20
QUICKSORT O(n lg n ) SFG
Sal F. Gambino
CS-342
The quicksort works by partitioning a list into
two smaller lists
and
recursively calling the quicksort to partition
each sub list
until
each sublist
contains a single item.
21
QUICKSORT O(n lg n ) SFG
Sal F. Gambino
CS-342
22
SFG
QUICKSORT O(n lg n ) Sal F. Gambino
CS-342
24
QUICKSORT O(n lg n )
SFG
Sal F. Gambino
Picking the pivot value CS-342
26
QUICKSORT SFG
Sal F. Gambino
•Median of 3: CS-342
8 1 4 9 6 3 5 2 7 0
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
A[ (left +right) / 2] = 6
A[ left ] = 8 A[ right ] = 0
A[ (0 + 9) /2 ]
A[ 0 ] A[ 9 ]
A[ 4 ]
0, 6, 8 6 or A[ 4 ] is the MEDIAN
27
QUICKSORT O(n lg n ) SFG
Sal F. Gambino
Quicksort Algorithm: CS-342
ENDIF
28
Quick SORT partitions SFG
1 Sal F. Gambino
CS-342
Pivot = 40
40 20 10 80 60 50 7 30 100 90 70
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
29
Quick SORT partitions
SFG
2 Sal F. Gambino
CS-342
Swap A[3],A[7]
Pivot = 40
40 20 10 30
80 60 50 7 30
80 100 90 70
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
80
temp
30
Quick SORT partitions
SFG
3 Sal F. Gambino
CS-342
Pivot = 40
40 20 10 30 60
60 50 77 80 100 90 70
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
31
Quick SORT partitions
SFG
4 Sal F. Gambino
CS-342
Swap A[4],A[6]
Pivot = 40
40 20 10 30 60
7 50 607 80 100 90 70
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
60
temp
32
Quick SORT partitions
SFG
5 Sal F. Gambino
CS-342
Pivot = 40
40 20 10 30 7 50 60 80 100 90 70
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
33
Quick SORT partitions
SFG
6 Sal F. Gambino
CS-342
Pivot = 40
40 20 10 30 7 50 60 80 100 90 70
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
34
Quick SORT partitions
SFG
7 Sal F. Gambino
CS-342
Pivot = 40
40 20 10 30 7 50 60 80 100 90 70
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
35
Quick SORT partitions
SFG
8 Sal F. Gambino
CS-342
Pivot = 40
Swap A[0],A[4]
7
40 20 10 30 40
7 50 60 80 100 90 70
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
40
temp
36
Quick SORT partitions
SFG
9 Sal F. Gambino
CS-342
7
40 20 10 30 40
7 50 60 80 100 90 70
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
LP1 RP1
QSort left QSort right
37
Quick SORT partitions SFG
Sal F. Gambino
7
40 20 10 30 CS-342
7 20 10 30
O( log n )
10 20 30
10 30
7 10 20 30 40
O( n ) 38
50 60 80 100 90 70 SFG
Sal F. Gambino
50 60 80 100 90 70 CS-342
60 80 100 90 70
70 80 100 90
Quick SORT O( log n )
partitions 70 90 100
90
7 10 20 30 40 50 60 70 80 90 100
O( n ) 39
Analysis:Quick Sort 5 6 7 8 9 SFG
Worst case O(n2) _ _ 7 _ _ Sal F. Gambino
_ _ _ 8 _ CS-342
Worst-case is O(n2)
_ _ _ _ 9
•first partition & left side
5 6 7 8 9
is EMPTY
•the partition requires n-1 comparisons
•next recursive call it will require n-2 comparisons
because n-1 elements are passed to it
•next n - 3 and n - 2 are passed and so on…..
•Usually the worst case situation is not typical
40
5 3 6 7 4 SFG
Analysis:Quick Sort 4 3 5 7 6
Sal F. Gambino
Average case 3 4 5 _ _ CS-342
O(n lg n) 3 4 5 7 6
3 4 5 6 7
•The data is random 3 4 5 6 7
•the partition requires fewer recursive calls
•each call to Quick Sort will require m
comparisons and at most m exchanges
•where m is the number of elements in the
sub-arrays
•therefore it can be used with large files 41
Heap SFG
Sal F. Gambino
CS-342
42
Heap examples SFG
Sal F. Gambino
26
CS-342
/ \
21 19
/ \ / \ 7
4 6 2 14 / \
/ \ 3 2
1 3
43
The following examples are NOT Heaps
SFG
Sal F. Gambino
26 CS-342
/ \
20 19
3 20
/ \ / \
15 12 3 5
/
\ / \
2
9 4 1
/
1
44
Heap Sort Algorithm SFG
Sal F. Gambino
2. unsorted = n
3. while ( unsorted > 1 )
{
• reduce unsorted side by 1 --unsorted
• swap data[0] with data[unsorted]
• the unsorted side is now a heap with the
root out of place
• Do a reheapification Downward to turn
the unsorted side back into a HEAP
}
45
Heap Sort Algorithm SFG
Sal F. Gambino
35 15 77 60 22 41 80 CS-342
[0] [1] [2] [3] [4] [5] [6]
60 22 41 80
35 15 77 60 22 41 80
0 1 2 3 4 5 66
47
Convert array to HEAP
SFG
If either child is greater than the parent Sal F. Gambino
then CS-342
35
15 80
80 Par = 2
LC = 2*2 + 1 = 5
60 22 41 77
77 RC = 2*2 + 2 = 6
35 15 80
77 60 22 41 77
80
0 1 2 3 4 5 66
48
Convert array to HEAP
SFG
Proceed backwards from the current Sal F. Gambino
step
35
15 1 80 Par = 1
LC = 2*1 + 1 = 3
60 22 41 77 RC = 2*1 + 2 = 4
35 15 80 60 22 41 77
0 1 2 3 4 5 6
49
Convert array to HEAP
SFG
Proceed backwards from the current Sal F. Gambino
step
35
60
60 80 Par = 1
LC = 2*1 + 1 = 3
15 22 41 77 RC = 2*1 + 2 = 4
35 60
60 80 15
15 22 41 77
0 1 2 3 4 5 6
50
Convert array to HEAP
SFG
Proceed backwards from the current Sal F. Gambino
step
0 35
60 80 Par = 0
LC = 2*0 + 1 = 1
15 22 41 77 RC = 2*0 + 2 = 2
35 60 80 15 22 41 77
00 11 2 3 4 5 6
51
Convert array to HEAP
SFG
Proceed backwards from the current Sal F. Gambino
step
80
80
60 35
35 Par = 0
LC = 2*0 + 1 = 1
15 22 41 77 RC = 2*0 + 2 = 2
80 60 35
35 15 22 41 77
00 11 2 3 4 5 6
52
Convert array to HEAP
SFG
Sal F. Gambino
The last change caused to propagate CS-342
down the tree
80
60 2 35 Par = 2
LC = 2*2 + 1 = 5
15 22 41 77 RC = 2*2 + 2 = 6
80 60 35 15 22 41 77
0 1 2 3 4 5 66
53
Convert array to HEAP
SFG
Sal F. Gambino
Finally we have a HEAP CS-342
80
60 77
77 Par = 2
LC = 2*2 + 1 = 5
15 22 41 35 RC = 2*2 + 2 = 6
80 60 77
77 15 22 41 35
0 1 2 3 4 5 66
54
[0] [1] [2] [3] [4] [5] [6]
SFG
80 60 77 15 22 41 35 Sal F. Gambino
CS-342
80 35
60 77 60 77
15 22 41 35 15 22 41 80
SWAP data[0]
with 35 60 77 15 22 41 80
[unsorted]=6 [0] [1] [2] [3] [4] [5] [6]
55
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
35 60 77 15 22 41 80 Sal F. Gambino
CS-342
35 77
60 77 60 35
15 22 41 80 15 22 41 80
Downward
77 60 35 15 22 41 80
ReHeapification
[0] [1] [2] [3] [4] [5] [6]
56
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
77 60 35 15 22 41 80 Sal F. Gambino
CS-342
77 41
60 35 60 35
15 22 41 80 15 22 77 80
SWAP data[0]
with 41 60 35 15 22 77 80
[unsorted] = 5 [0] [1] [2] [3] [4] [5] [6]
57
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
41 60 35 15 22 77 80 Sal F. Gambino
CS-342
41 60
60 35 41 35
15 22 77 80 15 22 77 80
Downward
60 41 35 15 22 77 80
ReHeapification
[0] [1] [2] [3] [4] [5] [6]
58
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
60 41 35 15 22 77 80 Sal F. Gambino
CS-342
60 22
41 35 41 35
15 22 77 80 15 60 77 80
SWAP data[0]
with 22 41 35 15 60 77 80
[unsorted] = 4 [0] [1] [2] [3] [4] [5] [6]
59
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
22 41 35 15 60 77 80 Sal F. Gambino
CS-342
22 41
41 35 22 35
15 60 77 80 15 60 77 80
Downward
41 22 35 15 60 77 80
ReHeapification
[0] [1] [2] [3] [4] [5] [6]
60
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
41 22 35 15 60 77 80 Sal F. Gambino
CS-342
41 15
22 35 22 35
15 60 77 80 41 60 77 80
SWAP data[0]
with 15 22 35 41 60 77 80
[unsorted] = 3 [0] [1] [2] [3] [4] [5] [6]
61
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
15 22 35 41 60 77 80 Sal F. Gambino
CS-342
15 35
22 35 22 15
41 60 77 80 41 60 77 80
Downward
35 22 15 41 60 77 80
ReHeapification
[0] [1] [2] [3] [4] [5] [6]
62
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
35 22 15 41 60 77 80 Sal F. Gambino
CS-342
35 15
22 15 22 35
41 60 77 80 41 60 77 80
SWAP data[0]
with 15 22 35 41 60 77 80
[unsorted] = 2 [0] [1] [2] [3] [4] [5] [6]
63
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
15 22 35 41 60 77 80 Sal F. Gambino
CS-342
15 22
22 35 15 35
41 60 77 80 41 60 77 80
Downward
22 15 35 41 60 77 80
ReHeapification
[0] [1] [2] [3] [4] [5] [6]
64
[0] [1] [2] [3] [4] [5] [6]
Delete Max
from heap SFG
22 15 35 41 60 77 80 Sal F. Gambino
CS-342
22 15
15 35 22 35
41 60 77 80 41 60 77 80
SWAP data[0]
with 15 22 35 41 60 77 80
[unsorted] = 1 [0] [1] [2] [3] [4] [5] [6]
65
Array is now Sorted
SFG
Sal F. Gambino
CS-342
15
Done!
22 35
41 60 77 80
15 22 35 41 60 77 80
66
Heap Sort analysis
SFG
Building the initial heap:
•worst-case O(n log Sal
n) F. Gambino
CS-342
O(n) # of elements
O(log n) max # of operations to heapify
•Best-case O(n)
Pulling the items from the heap:
•remove top and re-heapify O(n log n)
O(n) # of elements
O(log n) # of operations to heapify
Total time for HeapSort
•Build the heap O(n log n)
•Pulling items out O(n log n)
•Worst-case & Average-case O(n log n) 67
Summary: SFG
Sal F. Gambino
CS-342
68
Summary: SFG
Sal F. Gambino
CS-342
69
Summary: SFG
•Mergesort: Sal F. Gambino
•Heapsort:
•Worst-case & Average-case O(n log n)
•it works by:
•creating a heap from all the array elements
•then
•pulling the largest element out of the heap
•then the next largest,
•and so on …..
71
Summary: SFG
Sal F. Gambino
•HeapSort vs Selection sort
CS-342
•Selection sort locates the largest value and
places it in the final array position
•then, locates the next largest value and
places it in the next to last array position,
……....
•HeapSort uses a similar strategy locating
the largest value and then the next
largest, …...
•However the HeapSort uses a much more
efficient algorithm to locate the array
values to be sorted 72
SFG
Summary table:
Sal F. Gambino
CS-342
73
SFG
Sal F. Gambino
CS-342
Another
partitioning
algorithm
74
Quick SORT 1st partition SFG
Sal F. Gambino
3 9 6 2 5 8 1 7 4 Q(1,9) CS-342
piv = 3
36 9 63 2 5 8 1 7 4 order
62 9 3 62 5 8 1 7 4 split
62 91 33 62 5 8 91 7 4
End of first
2 1 3 6 5 8 9 7 4
partition
75
Quick SORT 2nd & 3rd partitions SFG
Sal F. Gambino
2 1 3 6 5 8 9 7 4 Q(1,2) CS-342
piv = 2
S1 S2
End of 2nd
21 12 3 6 5 8 9 7 4 partitions
Q(1,1)
21 12 3 6 5 8 9 7 4 piv = 1
1 12 3 6 5 8 9 7 4 End of 3rd
partitions
76
Quick SORT 4th partition SFG
Sal F. Gambino
1 12 3 6 5 8 9 7 4 Q(4,9)
CS-342
piv = 6
1 12 3 68 5 86 9 7 4 order
1 12 3 84 5 86 9 7 48 split
End of 4th
1 12 3 4 5 86 9 7 8
partitions
S1 S2
77
Quick SORT ... partitions SFG
Sal F. Gambino
1 12 3 4 5 86 9 7 8 CS-342
S1 S2
1 12 3 4 5 86 9 7 8 Q(4,5)
piv = 4
1 12 3 4 5 86 9 7 8
Q(5,5)
1 12 3 4 5 86 9 7 8 piv = 5
78
Quick SORT ... partitions SFG
Sal F. Gambino
1 12 3 4 5 86 9 7 8 Q(7,9) CS-342
piv = 9
1 12 3 4 5 86 98 7 89
1 12 3 4 5 86 87 78 89 Q(7,8)
piv = 8
1 12 3 4 5 86 87 78 89 Q(7,7)
piv = 7
Finally!!
79