Priority Queues (Heaps)

Chapter 6

Introduction
A queue is a FIFO data structure. Each item in the queue is treated equally. There may be times when we wish to prioritize the items in the queue. A priority queue is a data structure that orders its items by priority. Items can be enqueued, but will be dequeued according to priority (minimum value next).
2

Priority Queues
Consider jobs sent to a print queue. It is desirable to print small jobs first so that they dont wait a disproportionately long time for a large job to finish. Also consider processes waiting in a queue to run on the processor. It is desirable to run short processes first so that they arent kept waiting on longer processes to finish.
3

Model
A priority queue is a data structure that supports at least two operations:
insert to add items to the queue (enqueue) deleteMin to remove the minimum item in the queue (similar to dequeue).

Model

insert

deleteMin

Priority Queue

Implementations
Linked list: insert to front O(1), search for minimum O(N). Sorted linked list: insert in correct position is O(N), but finding minimum is O(1). Balanced Search Tree: insert and remove minimum in O(log N). Binary Heap: insert and remove minimum in O(log N) time.
6

Binary Heap
The structure of a heap is a binary tree that is completely filled except possibly the bottom level, which is filled left to right. (Complete Binary Tree). A complete binary tree of height h always has between 2h and 2h+1 1 nodes, which leads to performance of O(log N).

2h

2h+1 1

A
h=2

A C
h=2

22 = 4

22+1 1 = 7

A complete binary tree has between 2h and 2h+1 1 nodes.

8

Binary Tree Terminology

Full Binary Tree: every node is either a leaf or has two children. Complete Binary Tree: every node has two children except the last level which is filled from left to right . Perfect Binary Tree: has exactly 2h+1 1 nodes where h is the height. A A A

B
D E

C
D

B
E

C
F D

B
E

C
F
9

Full

Complete

Perfect

1 A

2
D 4

B E 5

C 3 F 6 G 7

Here the nodes are numbered top-down, left-to-right. Notice that any left childs number is 2x its parents number. Notice that any right childs number is 2x+1 its parents number. Any parent number is x/2 (integer division).

10

Heaps in Arrays
Using the numbering of the previous slide, it is easy to store a heap in an array. An advantage is that no links (pointers) are required between elements, and operations to traverse the tree are fast and easy.

11

Binary Heap
A binary heap can be stored in an array:
1

A B C D E F G
0 1 2 3 4 5 6 7 8 9 2 B

A
3 C F 6 G 7
12

Parent: i/2
Left child: 2i Right Child 2i+1

D 4

E 5

Heap-Order Property
To find the minimum quickly, it is helpful to have the smallest value at the root. If we consider that any subtree should also be a heap, then any node should be smaller than all of its descendants. This is called the heap-order property. This allows the findMin operation in O(1).
13

Insert
To insert, we have to do two things: Insert into the next available node (to preserve the structure) Re-arrange the heap to preserve the heaporder property.

14

Insert
First create a new node in the next available position in the heap. Then check the ordering, if it is okay, end. If not, swap it with its parent. Repeat swapping until it is in its proper position. This is called a percolate up.
15

Create new node for 14. 21

13 16

Move up by swapping with 31. 21

13 16

24
65 26

31
32

19

68
(1) 65

24
26 32 31

19

68
(2)

Move up, swap with 21.

24 65 26 21 32

13
16 19 31

(3)

Fill hole with 14. Done.

14

13
16 21 19

(4)

68 65

24 26

68

32 31

/** * Insert into the priority queue, maintaining heap order. * Duplicates are allowed. * @param x the item to insert. */ public void insert( AnyType x ) { if( currentSize == array.length - 1 ) enlargeArray( array.length * 2 + 1 ); // Percolate up int hole = ++currentSize; for(array[0]=x; x.compareTo( array[ hole / 2 ] ) < 0; hole /= 2 ) array[ hole ] = array[ hole / 2 ]; array[ hole ] = x; } Insert x in hole. Slide parent into hole. While hole not at root and x < parent of hole. Go up one level.
17

Put x in array[0] initially to serve as a stopping point.

deleteMin
To deleteMin, we have to do two things: Remove the root value (the minimum). Re-arrange the heap to preserve the heaporder property.

18

deleteMin
First delete the value at the root node. A delete means there is one less node in the tree and since it still must be a complete tree, the last node in the tree will be removed and its value put elsewhere. Next slide the smaller of the roots two children into the root. Repeat sliding the hole node down until the last node value can occupy this node while preserving the heap-order property. This is called a percolate down.
19

deleteMin of 13 14

13 16

Create hole at root, test 31. 14 16

19
65 26

21
32 31 14

19

68
(1) 65

19
26

21
32 31 14

19

68
(2)

Slide 14 up

(3)
16

Slide 19 up 19

(4)
16

19 65 26

21 32 31

19

68 65 26

21 32 31

19

68
20

Slide 26 up 19 26 65 21 32

14 16 19 31 68 (3)

Put 31 in hole 19 26 65 31 21 32

14 16 19 68 (4)

21

/** * Remove the smallest item from the priority queue. * @return the smallest item, or throw an UnderflowException if empty. */ public AnyType deleteMin( ) { if( isEmpty( ) ) throw new UnderflowException( ); AnyType minItem = findMin( ); array[ 1 ] = array[ currentSize-- ]; percolateDown( 1 ); return minItem; } Save minimum item

Move last element value to root and reduce heap size by 1. Percolate root down into the proper position in the heap.
22

Return minimum item

/** * Internal method to percolate down in the heap. * @param hole the index at which the percolate begins. */ private void percolateDown( int hole ) { int child; AnyType tmp = array[ hole ]; // save the value to percolate down for( ; hole * 2 <= currentSize; hole = child ) // go down level by level { child = hole * 2; // left child if( child != currentSize && array[ child + 1 ].compareTo( array[ child ] ) < 0 ) child++; // if right child exists and is less than left child, use right child if( array[ child ].compareTo( tmp ) < 0 ) // if tmp > child, slide child up array[ hole ] = array[ child ]; else break; // found proper position } array[ hole ] = tmp; // put the extra value into the empty node }
23

Other Heap Operations

Finding the maximum is not as easy. The maximum will reside in a leaf due to the heap property. But the leaves represent half of the nodes in the tree, so it is basically a linear scan. In a large heap this is not good performance. We could of course have a heap supporting maximum values if this is the desired operation instead of finding minimums. For other operations, we might require another data structure (such as a hash table) in addition to the 24 heap.

Other Heap Operations

Other heap operations might include these: decreaseKey requires percolate up increaseKey requires percolate down delete decreaseKey to minimum then deleteMin. buildHeap takes N items and builds a heap.
25

buildHeap
/** * Establish heap order property from an arbitrary * arrangement of items. Runs in linear time. */ private void buildHeap( ) { for( int i = currentSize / 2; i > 0; i-- ) percolateDown( i ); }
currentSize/2 is rightmost node at height=1, this is chosen because percolateDown will slide nodes down to the leaves.
26

buildHeap
buildHeap can be done in O(N) time. To see this, first note that a percolate down could occur for every parent node. Each percolate down could move that node to the bottom of the tree. This distance corresponds to the nodes height. So, we could compute the sum of the heights of all of the nodes and show that this is O(N).
27

Proof
A perfect binary tree has 2i nodes at height h-i: 20 nodes at h-0, 21 nodes at h-1, etc. The sum of the heights of these nodes is: S = 2i(h-i) = h + 2(h-1) + 4(h-2) + + 2h-2(2) + 2h-1(1)
i=0 h

We can now compute 2S by multiplying both sides by 2. 2S = 2h + 4(h-1) + 8(h-2) + + 2h-1(2) + 2h(1) Now if the two equations are subtracted, many terms cancel: 2S S = -h + 2 + 4 + 8 + + 2h-1 + 2h = (1 + 2 + 4 + 8 + + 2h-1 + 2h) + (-h) - 1 S = (2h+1 1) (h+1) Since a complete tree has between 2h and 2h+1 1 nodes, S is 28 O(N), therefore buildHeap is O(N).

Applications of Priority Queues

The Selection Problem: find the kth largest element. First we consider two algorithms without using heaps:
1. Sort list, return element at position k. 2. Sort first k elements. For each remaining element, insert it into proper position in list if larger than kth element.
29

Algorithm 1 Find 3rd largest element 4598 2822

5679
3231 9457 sort

3231
3457 4598

2822
3457 6371 8743

5679
6371 8743 9457
30

3rd largest

Algorithm 2 Find 3rd largest element 4598 3rd largest sort 1st 3 3231 4598 5679 4598 5679 9457
Insert 9457

5679
3231 9457

5679 6371 9457

Skip 2822 and 3457, insert 6371

6371 8743 9457

Insert 8743

2822
3457 6371 8743

31

Selection Problem
Now we consider two heap algorithms: First Algorithm: read elements into an array using buildHeap, then apply k deleteMin operations. (need to use a max-ordered heap for this). buildHeap is O(N), each deleteMin is O(log N). Result is O(N) + O(k log N) If k is O(N/log N), this will be O(N). If k is larger, this will be O(k log N).
32

Selection Problem
Second Algorithm: to find kth largest element, the first k elements are placed into a heap by calling buildHeap in O(k) time. Let Sk be the kth largest in the set. For each remaining element larger than Sk, remove Sk and insert the new element. When done, the k largest elements will be in the heap, with the kth largest at the root. buildHeap is O(k), each deleteMin and insert is O(log k). Result is O(k) + O( (N-k) log k) = O(N log k)
33

Event Simulation (Bank)

A computer simulation can simulate the passage of time by incrementing a counter at regular intervals and checking whether any event is due at this time. This is not very efficient since there may be long spans between events. A simulation can instead be conducted as a series of events. For example, a bank involves customers arriving and then later leaving the teller, each of which is an event. To create the simulation, we can place all events into a priority queue and then see which event is next.
34

Customer arrival times Name Arrival Duration Joe 3 7

Customer departure times Name Arrival Duration Departs Bob Joe Tim 6 3 9 2 7 3 8 10 12

Bob
Tim

6
9

2
3

Next Event Priority Queue 3: Joe arrives 6: Bob arrives 8: Bob departs 9: Tim arrives 10: Joe departs 12: Tim departs
35

d-Heaps
A d-Heap is a generalization of a binary heap, except a d-Heap has d children per parent. This results in a shallower tree which improves performance of inserts. However, the deleteMin operation may suffer because finding the minimum child of a node is now harder.
36

A 3-heap 1

2 4 11 9 7 10 13

3 15 6 8

5 17 12

37

Leftist Heaps
Leftist heaps are designed to support the efficient merging of two heaps. Leftist heaps use a null path length of a node, which is the length of the shortest path from the node to a node without two children. The leftist heap property is that for every node, the npl of its left child is at least as large as the npl of its right child.
38

Leftist Heaps
A node with zero or one child has npl=0. A null has npl=-1. The npl of any node is one more than the minimum npls of its children.

39

1 1 0 0 0 0 0 0 1*

1 0 1 0

Leftist heap

Not leftist heap

40

Merging
To merge two leftist heaps, recursively merge the heap with the larger root with the right subheap of the heap with the smaller root. If a merge results in a leftist violation, swap the two subheaps.

41

h1
3 10 21 23 14 26 17 8 18 33 12

h2
6 7 24 37 18

h1 has smaller root, so merge h2 with right subtree of h1 h1

8 12 17 18 26 33 24 37 18 7

h2
6

h2 has smaller root, so merge h1 with right subtree of h2

42

h1
8 17 26

h2
7 37 18

h2 has smaller root, so merge h1 with right subtree of h2 h1

8 17 26

h2
18

h1 has smaller root, so merge h2 with right subtree of h1

8 17 18

26
43

7 37 17 26 8 18

8 becomes right subtree of 7

7 8 17 18 37

26

subtree swap due to leftist violation

44

6 12 18 33 24 7

8
18

37

17 26

7 becomes right subtree of 6

3 10 6 21 23 14 12 7

18
33

24

8 18

37

17 26

6 becomes right subtree of 3

45

6 12 18 33 24 7 8 18 37 21

10 14

23

17 26

subtree swap due to leftist violation merge complete

46

Merging
A leftist heap of r nodes on the right path must have at least 2r-1 nodes (proof is in textbook). If N >= 2r-1, then N+1>=2r, and r <= log(N+1) Because a merge is proportional to the sum of the lengths of the right paths, a merge is O(log N). An insert can be viewed as merging a one-node tree into another tree, so an insert is O(log N). A deleteMin can be performed by deleting the root and merging the two subheaps, so deleteMin is also O(log N).
47

Skew Heaps
A skew heap performs merges in amortized O(log N) time. The recursive merge routine is similar to leftist heaps, but the left and right subheaps are always swapped.

48

h1
3 10 21 23 14 26 17 8 18 33 12

h2
6 7 24 37 18

h1 has smaller root, so merge h2 with right subtree of h1 h1

8
12 17 18 26 33 24 37 18

h2
6 7

h2 has smaller root, so merge h1 with right subtree of h2

49

h1
8 17 26 37

h2
7 18

h2 has smaller root, so merge h1 with right subtree of h2

h1
8 17

h2
18

26

h1 has smaller root, so merge h2 with right subtree of h1

50

8 18 26 17

merge and swap subtrees

7 8 18 26 17 37

merge and swap subtrees

51

6 7 8 18 26 17 37 18 33 12 24

merge and swap subtrees

3 10 12 37 17 26 18 33 24 21 23 14

6 7 8 18

merge and swap subtrees, done

52

Binomial Queues
Binomial queues give O(log N) time on insert, deleteMin, and merge, but give O(1) average time on an insert. Binomial queues are essentially a forest of heaps where each heap of height k has exactly 2k nodes, and there can be at most one heap of each height.
53

16 h1 18

12
24 21 65

13 h2

14 26 merge h1 and h2

23 24 51 65

13

23 24 51 65

12 24 21 65 26 18 14 16

54

Java Priority Queues

Java 1.5 introduced a PriorityQueue class. Methods:
add is like the insert operation remove is like the deleteMin operation element is like the findMin operation

55

End of Slides

56