Selec�on Sort – Simple Sor�ng Algorithm

For every itera�on i, we will search for the smallest number star�ng from i+1 and put it at index i.
Time complexity: Best, average, and worst: 𝑶𝑶(𝒏𝒏𝟐𝟐 )

MARC S. SALAMEH 1
Bubble Sort – Simple Sor�ng Algorithm
At every itera�on start at the end of the array, compare the last two index the smaller swap it, and move to the next
index, con�nue un�l arriving to the beginning of the array, the process is repeated un�l the whole array is sorted.
Time complexity: Best, average, and worst: 𝑶𝑶(𝒏𝒏𝟐𝟐 )

MARC S. SALAMEH 2
Inser�on Sort – Simple Sor�ng Algorithm
Start at the beginning of the array and then move forward, at each itera�on we will consider that the ith part is
ordered in the array, take the element a�er it, and compare it with the ordered array, then shi� it to its correct
posi�on.
Time complexity: Average and worst: 𝑶𝑶(𝒏𝒏𝟐𝟐 ), best: 𝑶𝑶(𝒏𝒏) if sorted array, no shi� made.

MARC S. SALAMEH 3
Shell Sort – Advanced Sor�ng Algorithm
Extension of inser�on sort with beter TC, consists of dividing the array into h subarray, we sort each subarray
using modified inser�on algorithm, and at the last step we use the original inser�on algorithm to the semi sorted
array.
Shell proposes the sequence of increments h calculated as: ℎ1 = 1, ℎ𝑖𝑖 + 1 = 3 ∗ ℎ𝑖𝑖+1 (calculated while ℎ𝑖𝑖 ≤ 𝑛𝑛).
The sor�ng is done from the greatest h to the smallest h (ending with h1=1) we use the sequence to reverse:
ℎ1 = ℎ𝑘𝑘 /3, ℎ2 = ℎ𝑘𝑘−1 /3, … 𝑢𝑢𝑢𝑢 𝑡𝑡𝑡𝑡 ℎ𝑗𝑗 = 1.
Other sequence can be used: ℎ𝑖𝑖 = 2𝑖𝑖−1 with ℎ1 = 1
Time complexity:
𝟑𝟑
Worst case: 𝑶𝑶(𝒏𝒏𝟐𝟐 = 𝒏𝒏𝟏𝟏.𝟓𝟓 )
Best case: 𝑶𝑶(𝒏𝒏)
𝟓𝟓
Average case: 𝑶𝑶(𝒏𝒏 = 𝒏𝒏𝟏𝟏.𝟐𝟐𝟐𝟐 )
𝟒𝟒

MARC S. SALAMEH 4
HeapSort – Advanced Sor�ng Algorithm
One of the best sor�ng algorithms,
Build a max heap or min heap to sort, at each itera�on remove the maximum or minimum element and insert it
inside the array, we get a sorted array at the end.
Time complexity: Best, average, and worst: 𝑶𝑶(𝒏𝒏 𝐥𝐥𝐥𝐥𝐥𝐥 𝒏𝒏)

MARC S. SALAMEH 5
MergeSort – Advanced Sor�ng Algorithm
We divide the array into two sub arrays recursively, and we
sort each part alone then merge the two results.
Each sub array is divided into more subarrays un�l having 1
element, then we apply the merge sort algorithm:
We copy the le� subarray in order and right subarray in
reverse order then we do the following:
1- create two virtual pointers on the two half arrays copied
obtained, one of them will move at each itera�on,
2- compare the elements, the smaller one will be inserted if
ASC or the greater if DESC, move its pointer,
3- repeat the process un�l sor�ng all the two sub arrays.
Time complexity: Best, average, and worst: 𝑶𝑶(𝒏𝒏 𝐥𝐥𝐥𝐥𝐥𝐥 𝒏𝒏)

MARC S. SALAMEH 6
 QuickSort – Advanced Sor�ng Algorithm
We call the par��on func�on recursively which chooses a pivot having:
- a le� part in which all the elements have a value less than or equal to
the pivot and
- a right part in which all elements have a value greater than or equal to
the pivot.
We obtain at the end a sorted array.
A�er applying the par��on algorithm, we should obtain at the end the
following sorted array:

During the applica�on of the par��on algorithm (during the process):

Time complexity: Best and average: 𝑶𝑶(𝒏𝒏 𝐥𝐥𝐥𝐥𝐥𝐥 𝒏𝒏) , worst: 𝑶𝑶(𝒏𝒏𝟐𝟐 ) the pivot is always smallest or greatest element.

MARC S. SALAMEH 7
Coun�ng Sort – Linear Sor�ng Algorithm
Non comparison algorithm, runs over specific type and range of data, they are not general and cannot run for any
type of data. It is a stable sor�ng algorithm because it retains the ini�al order of the equal elements. We create an
array B that will be used to show the result of the sorted array coming originally from an unsorted array A and we
create another array C that will be used with the sor�ng.
Time complexity: 𝑶𝑶(𝒏𝒏 + 𝒌𝒌) linear order so 𝑶𝑶(𝒏𝒏), 𝒌𝒌 ≤ 𝒏𝒏 to preserve the linearity.

MARC S. SALAMEH 8
Bucket Sort – Linear Sor�ng Algorithm
Non comparison algorithm, run over specific type and range of data, they are not general and cannot run for any
type of data. It is a stable sor�ng algorithm because it retains the ini�al order of the equal elements.
Uniform distribu�on, all values in the array are between [0,1[ , use inser�on sort to sort the items.
Create a new array B same size as A and will create linked lists inside the array for same index values.
Formula to insert in B = ⌊𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣 × 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠⌋
Finally combine the values of B in a final sorted array.
Time complexity: Best, average: 𝑶𝑶(𝒏𝒏 + 𝒌𝒌) , worst: 𝑶𝑶(𝒏𝒏𝟐𝟐 ) if all buckets were at the same index.

Bucket sort algorithm:

MARC S. SALAMEH 9
Radix Sort – Linear Sor�ng Algorithm
Non comparison algorithm, runs over specific type and range of data, they are not general and cannot run for any
type of data. It is a stable sor�ng algorithm because it retains the ini�al order of the equal elements.
We sort the numbers from their LSB to their MSB. We move the whole number while sor�ng.
Time complexity: 𝑶𝑶 (𝒅𝒅 (𝒌𝒌 + 𝒏𝒏)) if the coun�ng Sort algorithm is used, 𝒌𝒌 + 𝒏𝒏 for coun�ng sort and 𝒅𝒅 repeated
�mes for the number of digits.
Algorithm:
For i = 1 to d
Use a stable sor�ng algorithm to sort array A over digit i.

MARC S. SALAMEH 10
Time Complexity brief:
The complexity of a sor�ng algorithm is the number of comparisons performed by this algorithm.
The size of a sor�ng problem is the number of elements to sort.

Algorithm Best Case Average Case Worst Case

Simple Sor�ng Algorithm
Selec�on Sort 𝑛𝑛2 𝑛𝑛2 𝑛𝑛2
Bubble Sort 𝑛𝑛2 𝑛𝑛2 𝑛𝑛2
Inser�on Sort 𝑛𝑛 𝑛𝑛2 𝑛𝑛2
Advanced Sor�ng Algorithm
Shell Sort 𝑛𝑛 𝑛𝑛5/4 =1.25 𝑛𝑛3/2 =1.5
QuickSort 𝑛𝑛 log 𝑛𝑛 𝑛𝑛 log 𝑛𝑛 𝑛𝑛2
MergeSort 𝑛𝑛 log 𝑛𝑛 𝑛𝑛 log 𝑛𝑛 𝑛𝑛 log 𝑛𝑛
HeapSort 𝑛𝑛 log 𝑛𝑛 𝑛𝑛 log 𝑛𝑛 𝑛𝑛 log 𝑛𝑛
Linear Sor�ng Algorithm
Coun�ng Sort 𝑛𝑛 + 𝑘𝑘 𝑛𝑛 + 𝑘𝑘 𝑛𝑛 + 𝑘𝑘
Bucket Sort 𝑛𝑛 + 𝑘𝑘 𝑛𝑛 + 𝑘𝑘 𝑛𝑛2
Radix Sort 𝑑𝑑(𝑘𝑘 + 𝑛𝑛) 𝑑𝑑(𝑘𝑘 + 𝑛𝑛) 𝑑𝑑(𝑘𝑘 + 𝑛𝑛)
Linear Sor�ng Algorithm: Number of comparisons = 0

Stale sor�ng algorithm brief:

Stable sor�ng algorithm preserve the order number of the elements inside the array:
The first one that appears in the array will be sorted first and etc.

Stable VS Unstable algorithm brief:

Stable Unstable
Inser�on Sort Selec�on Sort
Bubble Sort QuickSort
Merge Sort HeapSort
Coun�ng Sort Shell sort
Bucket Sort
Radix Sort

MARC S. SALAMEH 11