Data Structure & Algorithms

CS-201
Lecture 05-Sorting Algorithms

Dr. Muhammad Mobeen Movania

 Outline
• What is sorting
• Sorting classification
• Sorting Algorithms and Efficiencies
• Types of Sorting
– Selection
• Selection Sort
• Heap Sort
– Insertion
• Insertion Sort
• Shell Sort
– Exchange
• Bubble Sort
• Quick Sort
• Merge Sort
• Non-comparison sort
– Radix Sort
 What is sorting
• Process with which data are arranged according to their values
• Most common process in CS
• Makes identification and finding of data easy
• Generic classification
– Comparison sort
– Non-comparison sort
• Two basic categories of sorting algorithms
– Internal sorts (data is stored in primary memory)
– External sorts (when data cannot fit into primary memory, it is moved
to secondary memory for processing
• Sort order identifies the sequence of sorted data
– Two types: Ascending and Descending Order
• Sort efficiency is the measure of relative efficiency of a sort
• Sort pass is the traversal of data during sorting
• Sort stability indicates that data with equal keys maintain their
relative input order in the output
Sort Classification
 Selection Sort
• The most intuitive of all sorts
• Given a list of data to be sorted, we simply
select the smallest item and place it in a
sorted list
• These steps are then repeated until we have
sorted all of the data.
• Two types of selection sorts
– the straight selection sort and
– the heap sort (we will do it once we finish heaps)
Selection Sort Concept
Selection Sort Example
Selection Sort Algorithm
 Selection Sort Efficiency
• The basic selection sort algorithm contains
two nested loops

• Hence, the worst case complexity is O(n2)

 Insertion Sort
• Works similar to how cards are arranged by
card players
• In each pass, the first element from the
unsorted list is inserted into the appropriate
place in the sorted list
• For n elements, n-1 passes are required
Insertion Sort Concept
Insertion Sort Example
Insertion Sort Algorithm
 Insertion Sort Efficiency
• The basic insertion sort requires two loops

• Hence, the worst case complexity is O(n2)

 Shell Sort
• Improved version of insertion sort
• Diminishing partitions are used to sort the data
• A list of N elements is divided into K segments
• K is called the increment
• Each segment has N/K elements
• Segment size is usually N/2 half of the total
number of elements.
– For higher segments, segment size is half of the
previous segment size
– No hard and fast rule to follow
Shell Sort Segments
 Shell Sort Example

http://interactivepython.org/courselib/static/pythonds/SortSearch/TheShellSort.html
 Shell Sort Example

http://interactivepython.org/courselib/static/pythonds/SortSearch/TheShellSort.html
Shell Sort Algorithm

http://mathbits.com/MathBits/CompSci/Arrays/Shell.htm
 Shell Sort Efficiency
• Requires nested loops that do not run n times but are
based on increment size
• Outermost loop divides the increment by 2
– Complexity O(log2n)
• The first inner loop runs n-increment times for each outer
loop
– Complexity O(n)
• Total complexity of outer + first inner loop O(nlog2n)
• The inner most loop also runs a few iterations
– Complexity > O(nlog2n)
• Based on experimental analysis, the complexity of shell sort
algorithm is ~O(n1.25)
 Bubble Sort
• Simplest sorting algorithm
• Worst performance of all sorting algorithms
• Typically has complexity O(n2)
• For a list of N elements, it requires N-1 passes
• In each pass, the ith element is compared to
(i+1)th element
• For ascending order, the smallest element is
bubbled from unsorted to sorted list
Bubble Sort Concept
 Bubble Sort Example

Sorting in Ascending Order using Bubble Sort

Pass # 23 78 45 8 56 32
1 23 45 8 56 32 78
2 23 8 45 32 56 78
3 8 23 32 45 56 78
4 8 23 32 45 56 78
5 8 23 32 45 56 78
Bubble Sort Algorithm
 Bubble Sort Efficiency
• Bubble sort requires nested loops

• Outer loop runs n times, inner loop runs (n+1)/2 times

on average
• Complexity O(n2)
• A simple improvement to naïve bubble sort
– include a flag which is raised if there is no exchange
– This signifies that the list is already sorted and hence the
loop can be prematurely terminated
 Variation of Bubble Sort
• Sometime naively called Exchange Sort
• A slight variation in the pseudo code as
follows
//N is the array/list size
loop i=0 to N-1
loop j=i+1 to N
if(array[i] > array[j])
swap(array[i], array[j]);
end if
end loop
end loop
 Quick Sort
• Bubble sort exchanges many elements in each pass
• Quick sort is efficient as it compares and exchanges
elements that are far apart
– The total exchanges are reduced to position an element to its
correct position
• Each iteration quick sort selects a pivot element which
splits the list into three parts
– Elements that are less than pivot
– The pivot element itself
– Elements that are greater then pivot
• The left and right partitions are then quick sorted
iteratively
• Pivot is usually the median value in the list
Quick Sort Concept
Quick Sort Algorithm
Quick Sort Algorithm
Quick Sort Median Algorithm
 Quick Sort Efficiency
• Quick sort algorithm has three iterative and
two recursive loops
 Quick Sort Efficiency
• Each pass in the quick sort divides the list into
three parts
• The first loop, in conjunction with the two nested
loops runs n times
– Complexity: O(n)
• The two recursive loops process one portion of
the array
– List is split into two lists in each loop
– Complexity: O(log n)
• Total Complexity: O(nlog2n)
 Merge Sort
• It is a divide and conquer algorithm that was
invented by John von Neumann in 1945
• Produces a stable sort (the input order of
equal elements is preserved)
• It is highly parallelizable
• The sort first divides the list into n sub-list
each with 1 element
• Then compares and repeatedly merges
adjacent elements to get the sorted list
Merge Sort Example
Merge Sort Example
Merge Sort Algorithm
Merge Sort Algorithm
 Merge Sort Efficiency
• The merge sort algorithm has two parts
– Splitting the list into 1 element lists
– Merging the sorted 1 element lists into a single sorted
list
• The first step uses a recursive algorithm which
split the list of n elements into two parts
– Complexity: O(log2n)
• The merge step takes at most n iterations to
merge all sub-lists
– Complexity: O(n)
• Combine complexity: O(nlog2n)
 Radix Sort
• Non-comparative integer sorting algorithm
• A stable sort which is usually used for sorting of strings, specially
formatted floating point numbers and integers
• Groups keys by the individual digits which share the same
significant position and value
• Two subclasses (based on direction of sorting)
– least significant digit (LSD) radix sorts
– most significant digit (MSD) radix sorts
• LSD Radix Sort
– Starts with short keys first followed by long keys
– For keys with same length, the lexicographic order is preserved
– Example: 6 7 23 8 1 will be sorted to 1 6 7 8 23
• MSD Radix Sort
– Uses lexicographic order, which is suitable for sorting strings, such as
words, or fixed-length integer
– Example: 6 7 23 8 1 will be sorted to 1 23 6 7 8
 LSD and MSD Radix Sort
• It is a fast stable sorting algorithm which sorts keys in integer
representation order
• Keys may be a string of characters, or numerical digits in a given
'radix‘
• Processing of keys begins at the least significant digit (i.e., the
rightmost digit), and proceeds to the most significant digit (i.e., the
leftmost digit)
• The sequence in which digits are processed by an LSD radix sort is
the opposite of the sequence in which digits are processed by a
most significant digit (MSD) radix sort.
• An LSD radix sort operates in O(nk) time, where n is the number of
keys/elements, and k is the key length
• Radix sorting algorithm was originally used to sort punched cards in
several passes
• In many large applications needing speed, the computer radix sort
is an improvement on (slower) comparison sorts
 Radix Sort Examples
Sorting in Ascending Order using LSD Radix Sort
Pass # 362 436 291 487 207 253 397
1 291 362 253 436 487 207 397
2 207 436 253 362 487 291 397
3 207 253 291 362 397 436 487

Sorting in Ascending Order using MSD Radix Sort

Pass # 237 318 216 462 211 268 460
1 237 216 211 268 318 462 460
2 216 211 237 268 318 462 460
3 211 216 237 268 318 460 462
Next Lecture
• Lecture 06-Recursion