Chapter 8

Co-sequential Processing and Sorting of Large Files

Co-sequential Processing
• Co-sequential operations involve the coordinated processing of
• two or more sequential lists to produce a single output list.
• This is useful for merging (or taking the union) of the items on the
two lists and for matching (or taking the intersection) of the two lists.
• These kinds of operations are extremely useful in file processing.
 Co-sequential Processing : Merge(Sort)
• List 1 List 2 Algorithm:
•0 1
Synchronization
•2 3  Let Item(1) be the current item from list 1 and
Item(2) be the current item from list 2.
•4 5  Rules:
 If Item(1) < Item(2), output Item(1) and get the next item from list 1.
• 15 6  If Item(1) > Item(2), Output Item(2) and get the next item from list 2.
 If Item(1) = Item(2), output the two items and get the next items from the two lists.
• 16 7
• 19 29
• What if the file can’t fit into memory, What will be the solution?
• Solution:
• (1) Break the file into several sorted subfiles (runs),
• Each Run should fit into memory
• Sort them using an internal sorting method ( On Memory)
• Write Them Back to Hard Disk
• (2) Merge the sorted runs using MergeSort on a new sorted file on Hrad-Disk
• If weIf we are merging 2 runs, we need to have two numbers only in memory like the example in
previous slide
• are merging 3 run, we need to have three numbers only in memory

Example : Sort the following data given that memory can’t hold except 4 numbers only:
• 0,19,6,4,2,16,8,3
0,19,6,4,2,16,8,3
• Run/file 1 : 0,9,6,4
• Run/file 2: 2, 16,8,3

Internal Sorting:

• Run 1 : 0,4,6,9
• Run 2: 2, 3,8,16
• Run 1: 0,4,6,9
• Run 2: 2, 3,8,16

Merging:
0,20
4,2  0,2
4,3 0,2,3
4,8  0,2,3,4
6,8 0,2,3,4,6
9,8 0,2,3,4,6,8
9,16 0,2,3,4,6,8,9
-, 16 0,2,3,4,6,8,9,16
-,-  0,2,3,4,6,8,9,16
 3,4,6,2,9,4,8,7,5,6,3,1,2 Input file

3,4 6,2 9,4 8,7 5,6 3,1 2 PASS 0

3,4 2,6 4,9 7,8 5,6 1,3 2 Internal Sort

2,3 4,7 1,3 PASS 1

4,6 8,9 5,6 2

2,3
4,4 1,2 PASS 2
6,7 3,5
8,9 6

1,2
PASS 3
2,3
3,4
4,5
6,6
7,8
9
Selecting The Minimum value From a set of
Run Values using Selection Tree
• Advantages:
• It can be applied to files of any size.

• Reading of the input during the run-creation step is sequential  Not much
seeking.
Heap Sort
Sort On Memory
Heap Sort
Heap Sort
• The Second stage occur virtually as well at the same time we write
sorted numbers to output file.

• So, All the sort process happens while we are reading and writing the
data to file which will reduce time needed to sort the data on file.
• Sorting time and reading/writing times has been merged into one since the
sorting and reading/writing will happen in parallel fashion rather than
sequential fashion
Heap Sort Example
How To Improve K-Way Merge Sort
• Increasing the amount of memory: helps make the buffers larger and
thus reduce the numbers of seeks.
• Increasing the Number of Dedicated Disk Drives: If we had one separate
read/write head for every run, then no time would be wasted seeking.
• Increasing the Number of I/O Channels: With a single I/O Channel, no
two transmission can occur at the same time. But if there is a separate
I/O Channel for each disk drive, then I/O can overlap completely.
But what if hardware based improvements are not possible Increase Run
Size By Replacement Selection
How To Improve K-Way Merge Sort
• Replacement Selection Procedure:
1. Read a collection of records and sort them using heapsort. The resulting heap is called
the primary heap.
2. Instead of writing the entire primary heap in sorted order, write only the record whose
key has the lowest value.
3. Bring in a new record and compare the values of its key with that of the key that has just
been output.
4. If the new key value is higher, insert the new record into its proper place in the primary
heap along with the other records that are being selected for output.
5. If the new record’s key value is lower, place the record in a secondary heap of records
with key values smaller than those already written.
6. Repeat Step 3 as long as there are records left in the primary heap and there are records
to be read. When the primary heap is empty, make the secondary heap into the primary
heap and repeat steps 2 and 3.
How To Improve
K-Way Merge Sort