Linear Time (Single-Loop) Sorts

Saeed ur Rehman Turk

 Linear Time
(Without nested loop or recursive call)
• Conceptually, ordering problem must have a
solution in linear time.
– i.e N elements should be sorted in N steps.
– Humans sort small collection of N things in just N steps.
– No Universal Computer Algorithm has been found so far.
• Some sorting techniques are devised.
• Although innovative, they are not generally feasible
due to hardware or storage limits.
 Counting Sort
Linear time (single loop) Sort
Counting sort is somewhat similar to ranking.
(Ranking means finding position of a random element at
where it would be placed in sorted array).
Ranking of an element is found by counting smaller or
equal elements than it. (without Sorting).
Then element can be placed on its rank. If all elements are
ranked, they can be placed at their ranks in output array.
(It was done in Quicksort, but in very efficient loop)
. However Counting-Sort is actually non-comparison-
based sort.
• It uses value of element to estimate its rank.
 Intermediate storage
• Needs Very large intermediate array.
– (intermediate array size is equal to range of input
elements. i.e. if we are to sort 20 elements ranging
from 40 to 50000, we have to allocate intermediate
array of 50000-40 =49961 elements.
• Although there is no nested loop, but processing
loop of such large array may increase than nested
loop cost.
– Single loop cost of above array requires 49961
iterations which is very larger than 202 = 400 iterations
on any other nested loop sort.
 4 Steps of Counting Sort
(input random array A, output sorted Array B, temporary Array C

• Use an Array C as Counter array and initialize it to all zeros.

– For j=1 to K Cj=0
• Size of C should be the range of elements of A.
• (Counting sort can sort integers array only).
• Scan A from left to right.
– For i=1 to N let X = Ai
– For each element X( =Ai), go to C[X] and increment it. i.e treat each element X as index into C.
– C[Aj]++
• Transform C into Cumulative array by
• For i=2 to K
– Ci = Ci+Ci-1
• Now C contains ranks of each element of A
• But C has some misinformation too (??)
• So we just read an element of A, find its rank from C and position it in output array B on its
rank.
• For (i=N downto 1) //backward loop to ensure stable sort property(??)
– rank=C[Ai];//rank of Ai
– B[rank]=Ai;//place Ai in B at its rank position
– C[Ai]- -; // decrement rank for next repeated element
A input 3 5 6 13 8 8 3 2 10 6 9 5 3 6 8 10

For i= 1 to K where K is range of elements

Ci=0
index 1 2 3 4 5 6 7 8 9 10 11 12 13
Counter
Array C count 0 0 0 0 0 0 0 0 0 0 0 0 0

Scan A from left to right.

For i=1 to N let X = Ai
(For each element X( =Ai), go to C[X] and increment it. i.e treat each
element X as index into C)
C[X]++

index 1 2 3 4 5 6 7 8 9 10 11 12 13
Counter count 0 0, 0, 0 0, 0, 0 0, 0, 0, 0 0 0,
Array C 1 1, 1, 1, 1, 1 1, 1
with 2, 2 2, 2, 2,
history 3 3 3 3
 Index 1 2 3 4 5 6 7 8 9 10 11 12 13
Counter
count 0 0, 0, 0 0, 0, 0 0, 0, 0, 0 0 0,
Array C 1, 1, 1, 1, 1, 1, 1, 1,
with 2, 2 2, 2, 2
history 3 3 3
A input 3 5 6 13 8 8 3 2 10 6 9 5 3 6 8 10

Counter array to Cumulative Array

Counter
Array to Index 1 2 3 4 5 6 7 8 9 10 11 12 13
cumulative count 0 1 3 0 2 3 0 3 1 2 0 0 1,
array
Transform C into Cumulative array by
For i=2 to K
Ci = Ci+Ci-1
Now C contains ranks of each element of A
Counter Index 1 2 3 4 5 6 7 8 9 10 11 12 13
Array to count 0 1 3 0 2 3 0 3 1 2 0 0 1
cumulative cumula 0 1 4 4 6 9 9 12 13 15 15 15 16
array tive
 Positioning according to rank=B output
A input 3 5 6 13 8 8 3 2 10 6 9 5 3 6 8 10

Index 1 2 3 4 5 6 7 8 9 10 11 12 13
mulative
ray count 0 1 3 0 2 3 0 3 1, 2 0 0 1,
Rank 0 1, 4, 4 6, 9, 9 12, 13, 15, 15 15 16,
0 3, 5, 8, 11, 12 14, 15
2, 4 7, 10, 13
1 6 9

For (i=N downto 1) //backward loop to ensure stable sort property(??)

rank=C[Ai];//rank of Ai
B[rank]=Ai;//place Ai in B at its rank position
C[Ai]- -; // decrement rank for next repeated element
Index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
B output 2 3 3 3 5 5 6 6 6 8 8 8 9 10 10 16
• As we can see, there is no nested loop or
recursion. So it seems linear time sort.
• But actually T(N) does not depend only on N
(Number of elements), but also range of
elements (Difference between maximum and
minimum element) Which may be very larger. So
time cost depends on K= (Max(A) – Min(A))
which may become very larger than N2
• T(K)=Theta(K) roughly for C loops 1…k
• T(N) = Theta (N) for Array A loops 1…N
A input 3 5 6 13 8 8 3 2 10 6 9 5 3 6 8 10

index 1 2 3 4 5 6 7 8 9 10 11 12 13
Counter
Array C count 0 0 0 0 0 0 0 0 0 0 0 0 0

Counter Index 1 2 3 4 5 6 7 8 9 10 11 12 13
Array C count 0 1 3 0 2 3 0 3 1 2 0 0 1
Counter Index 1 2 3 4 5 6 7 8 9 10 11 12 13
Array to
cumulativ rank 0 1 4 4 6 9 9 12 13 15 15 15 16
e array

Counter Array Index 1 2 3 4 5 6 7 8 9 10 11 12 13

to cumulative Rank - - 0 0 1 4 4 6 9 9 12 13 15 15 15
array

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
B Output
2 3 3 3 5 5 6 6 6 8 8 8 9 10 10 16
 Time complexity of Counting Sort
• Algo has 4 loops on three arrays of Size A[N], B[N]
and C[K] where N is number of elements and K is
range of elements.
• Obviously, if N>K i.e. Numbers are from a small
range are repeated also,
• T(N) = Theta(N)
• However, if K>N i.e. size of input is small but range
is larger. (e.g. 100 integers from range 1-1000000)
• Then T(N) = Theta(K) which may not be feasible.
• Roughly T(N) =Theta (Max(N, K))
• Or T(N)= Max(Theta(N), Theta(K))
• On the other hand, tie complexity of a good
divide-and-conquer is Theta(NLogN)
• Typically countingsort is feasible only when
K<NLogN, otherwise any Divide-and-conquer
sort is feasible.
 Radix Sort
• Radix Sort was useful in sorting items using hardware
(old mechanical counters) and old digital IC’s.
• It sorts multidigit integers on the basis of their digit
positions.
• All elements are sorted on the basis of their
rightmost(least significant) digit in first step.
– Any stable sort technique can be used to sort single digit.
• One digit is sorted in one loop, then next digit and
then next digit is sorted, until leftmost(most
significate) digit is sorted.
• Eventually whole array is sorted.
• As number of digit is very small (2-6 etc) and digit
range is also small (0 to 9), such sorting becomes
handy.
• Radix sort uses any other sort to sort a single
digit, and depends upon time required to run that
sort. The other sort may or may not be a single
loop/nestedloop/divide&conquer sort.
– Radix Sort repeats itself for each digit Least-significant
to most significant, so its not in true sense linear sort.
 Simulation of Radix Sort
Input First Pass 2nd Pass Last Pass
Algo
 Time complexity of Radix-Sort
• Radix sort uses repeated sorts in a loop.
• Loop depends upon number of digits of input.
• Any stable sort can be used for internal sort of single
digits.
– As digit range is just 10, counting-sort can be used for single
digit sort. (Why?)
– So Time complexity of one iteration is Theta(N) for Counting
Sort (Why?)
• For a 3 digit number array, Time complexity become
around 3 x Theta(N) = Theta(N)
• For an array of D-digits numbers Time complexity is still
D x Theta(N) =Theta(N)
 Bucket Sort
An old concept before Divide-and-Conquer.

• Bucketsort uses smaller arrays/linklists (Buckets) for

input division.
• Each bucket contains a small range of numbers
value.
• Then sorts buckets internally using any sort algo.
• Time complexity reduces due to local sorting within
buckets.
• Still any sorting algo is needed but it is assumed
that its complexity will reduce due to small inputs.
 Buckesort from Cormen Book
• Algo on next slide assumes that input array A values have range [0.00 0.99]
i.e 2- digit decimal numbers only. With no integer part
– e.g 0.43, .07, 0.11, 0.94, etc
• Let n =10 i.e. 10 buckets are created. And numbered bucket0, bucket1,
bucket2. . . bucket9
• Each bucket will accept number range in leftmost (most significant) digit.
• 0.43 goes to bucket4, 0.07 goes to bucket0, 0.11 goes to bucket 1, etc
– Bucket 0 accepts 0.0D, Bucket 1 accepts 0.1D , bucket 2 accepts 0.2D…..
Bucket 9 accepts 0.9D numbers where D is any digit from 0..9.
• (Any other numbers range and buckets format etc can also be utilized)
 Deciding bucket number
• Bucket number= n x A[i] (taking integer part only) will determine
which bucket number will accept A[i].
• 10 x 0.43= 4.3. taking integer part =4 only. Decides that bucket4 will
accept 0.43.
• A bucket can be a small array or linked list.
• Out algo assumes each bucket is a linked list.
• All Ai are inserted (insertlast) into linkedlist of that bucket.
• Then All linked lists are sorted using Insertion Sort
– Insertion Sort and Selectionsort are fastest algorithms for
Linkedlist sorting (Why?)
• Now All linkedlists, staring from Bucket[0] are sorted and can be
printed or concatenated/copied back.
Bucket number= n x A[i] (taking integer part only)
 Algo
• BUCKET-SORT(A)
• 1 let B[0.. n-1] be a new array of linked lists
• 2 N = A.length
• 3 for i = 0 to n -1
• 4 make B[i] an empty list
• 5 for i = 1 to N
• 6 insert A[i] into list B[nxA[i]]
• 7 for i = 0 to n -1
• 8 sort list B[i ] with insertion sort
• 9 concatenate the lists B[0], B[1], . . . B[N-1] together in
order
 Algo Explained
• A is an input array of decimal numbers in interval [0 1]. B is an
array of linked lists such that it contains first pointers of each
linked list with following declaration.
• LinkedList B[N];//array of Linkedlist objects declared. (our
previous experience of linkedlist make it empty at construction
time by first=NULL)
• Select an element as Ai (2-digit decimal number only), compute
its bucket number by multiplying it with 10 & taking integer
part only.,.
• (one digit on left of decimal point will decide bucket number).
• InsertLast Ai into that buck number’s linked list.
– (Alternately insertinorder in linkedlist)
• Sort all linkedlists and join outputs into one array.
• Algo on next slide assumes that input array A values
have range [0 1] i.e 3- digit decimal numbers only.
– e.g 0.432, 0764, 0.111, 0.984, etc
• Let n=10 i.e. 10 buckets are created.
• Each bucket will accept number range in leftmost
(most significant) digit.
– Bucket 0 accepts 0.0DD, Bucket 1 accepts 0.1DD ….. Bukect
9 accepts 0.9DD numbers where D is any digit from 0..9.
• Any other numbers range and buckets format etc can
also be utilized.
 Time complexity of BucketSort
• As bucketsort utilizes Insertion-sort to sort
individual buckets internally, its time complexity is
still O(Insertion-sort).
– i.e. n x O(B2) where B may be number of elements in
each bucket. B is typically N/10 for 10 buckets.
– B = N/n approximately.
• N= Total number of elements, n=number of buckets
• But assuming that Insertion sort runs faster on
smaller inputs, Bucket sort reduces to O(N) under
special conditions.