10 Heaps

PRIORITY QUEUES - HEAPS
•A priority queue is a data structure that

allows at least the following
operations:
• insert
• delete the element with the minimum
value (deleteMin)
• we need to work with Comparable objects
• insert ≡ enqueue
• deleteMin ≡ dequeue
1
PRIORITY QUEUES
•Possible implementations:
• Linked Lists
• O(1) insertion time (insert at the front)
• O(N) deleteMin time
2
PRIORITY QUEUES
• Linked Lists
• Sorted Linked Lists
• O(N) insertion time (insert at the right place)
• O(1) deleteMin time (delete the first element)
3
PRIORITY QUEUES
• Linked Lists
• Sorted Linked Lists
• O(N) insertion time (insert at the right place)
• O(1) deleteMin time (delete the first element)
• Binary Search Trees/ AVL Tree
• O(log N) insertion time
• O(log N ) deleteMin time (asymmetry, tree
becomes right heavy!)
4
PRIORITY QUEUES
• Binary Search Trees look OK but are actually

an overkill:
• we do not need find or findMax operations
• we do not need the pointers (parents, children)
• A simplified binary tree called a HEAP is

sufficient for implementing priority queues.
5
BINARY HEAPS
•Binary heaps are binary trees with two
properties:
• Structure Property
• Heap-order Property
6
BINARY HEAPS
•Structure Property
• A heap is a binary tree that is completely
filled with the possible exception of the
bottom level, which is filled from left to
right.
7
BINARY HEAPS
• A heap is a binary tree that is completely filled with
the possible exception of the bottom level, which is
filled from left to right.
A
B C
A binary heap
D E F G
H I J
8
BINARY HEAPS
• Such a tree is known as a complete binary tree
B C
D E F G
H I J
9
BINARY HEAPS
• Other examples of complete binary trees
B C
D E F G
H I
10
BINARY HEAPS
• Other examples of complete binary trees
B C
D E F G
H I J K
11
BINARY HEAPS
• Examples of binary trees which are NOT COMPLETE
B C
D E F G
H I K
12
BINARY HEAPS
B C
D E G
H I
13
BINARY HEAPS
B C
D E F G
H J K
14
Perfect Binary Trees
• A perfect binary tree is a tree in which every node other
than the leaves has two children and all the leafs are
located at the bottom level.
A
A
B C
Perfect binary tree of height 0 Perfect binary tree of height 1
• Total number of nodes in a

A perfect binary tree of height h
is 2h+1 – 1
B C
• Number of nodes at a level of
depth d is 2d
D E F G
Perfect binary tree of height 2 • Prove these!

15
COMPLETE BINARY TREES
• A complete binary tree of height h has

between 2h and 2h+1-1 nodes.
• This implies that the height of a complete

binary tree of N nodes is ⎣log2 N⎦ which is
O(log N)
16
•Structure Property:
• Since the complete binary tree is very

regular, it can be represented with an
array and does NOT require any pointers.
• For an entry at array location i, (1 based)

• the children are at locations 2i and 2i+1
• the parent (if any) is at location ⎣ i/2 ⎦.
17
A
B C
D E F G
H I J
0 1 2 3 4 5 6 7 8 9 10 11
18
A
B C
D E F G
H I J
0 1 2 3 4 5 6 7 8 9 10 11
19
A
B C
D E F G
H I J
A B C
0 1 2 3 4 5 6 7 8 9 10 11
20
A
B C
D E F G
H I J
A B C D E
0 1 2 3 4 5 6 7 8 9 10 11
21
A
B C
D E F G
H I J
A B C D E F G
0 1 2 3 4 5 6 7 8 9 10 11
22
A
B C
D E F G
H I J
A B C D E F G H I
0 1 2 3 4 5 6 7 8 9 10 11
23
A
B C
D E F G
H I J
A B C D E F G H I J
0 1 2 3 4 5 6 7 8 9 10 11
24
A
B C
D E F G
H I J
A B C D E F G H I J
0 1 2 3 4 5 6 7 8 9 10 11
Note that we could also have used an array starting with 0

25
A
A possible problem is that the
B
maximum array size needs to be
C
known.
D E F G Typically this can be handled by

resizing
H I J
A B C D E F G H I J
0 1 2 3 4 5 6 7 8 9 10 11
26
HEAP CLASS
template <class Comparable>
class BinaryHeap
{
public:
BinaryHeap( int capacity = 100 );
bool isEmpty( ) const;

bool isFull( ) const;
const Comparable & findMin( ) const;
void insert( const Comparable & x );

void deleteMin( );
void deleteMin( Comparable & minItem );
void makeEmpty( );
private:
int currentSize; // Number of elements in heap
vector<Comparable> array; // The heap array
void buildHeap( );
void percolateDown( int hole );
}; 27
HEAP CLASS
class BinaryHeap
{
public:


void deleteMin( );
void makeEmpty( );
private:
void buildHeap( );
}; 28
HEAP CLASS
class BinaryHeap
{
public:


void deleteMin( );
void makeEmpty( );
private:
void buildHeap( );
}; 29
HEAP CLASS
class BinaryHeap
{
public:


void deleteMin( );
void makeEmpty( );
private:
void buildHeap( );
}; 30
HEAP ORDER PROPERTY
• In addition to the structure property, we also

need the heap-order property
• For every node X (except for the root), the key of
the parent of X is smaller or equal to the key of X.
• The minimum element is at the root of the heap.
31
HEAP ORDER PROPERTY
13
21 16
24 31 19 100
65 26 32
32
HEAP ORDER PROPERTY
13
21 16
24 31 19 100
65 26 32
33
HEAP ORDER PROPERTY
13
21 16
24 31 19 100
65 26 32
34
HEAP ORDER PROPERTY
13
21 16
24 31 19 100
65 26 32
35
HEAP ORDER PROPERTY
13
21 16
24 31 19 100
65 26 32
36
HEAP ORDER PROPERTY
13
21 16
24 31 19 100
65 26 32
37
HEAP ORDER PROPERTY
13
21 16
24 31 19 100
65 26 32
Note that siblings are not ordered in any
particular way!
38
HEAP ORDER PROPERTY
13
21 16
6 31 19 100
65 26 32
This complete tree does NOT have the
heap-order property
39
BASIC HEAP OPERATIONS
•INSERT X
• We create a hole in the next available
location (to maintain the structure
property)
• Place X in the hole
• If heap order property is violated keep on
interchanging X with its parent until
property is established
40
INSERTION
13
We want to insert 14
21 16
24 31 19 100
65 26 32
41
INSERTION
13
21 16
Create a hole and put 14 there
24 31 19 100
65 26 32 14
42
INSERTION
13
21 16
24 31 19 100 Heap property violated
65 26 32 14
43
INSERTION
13
21 16

Exchange
65 26 32 31
44
INSERTION
13
21 16

Exchange
32 31
65 26
Heap property violated
45
INSERTION
13
14 16

Exchange
65 26 32 31
Exchange
46
INSERTION
13
14 16

Exchange
65 26 32 31
Exchange
Heap Property is now

established
47
INSERTION
13
14 16
24 21 19 100
65 26 32 31
This strategy is called percolation or
bubble-up
48
insert
/**
* Insert item x into the priority queue, maintaining heap order.
* Duplicates are allowed.
* Throw Overflow if container is full.
*/
void BinaryHeap<Comparable>::insert( const Comparable & x )
{
if ( isFull( ) )
throw Overflow( );
// Percolate up
// Note that instead of swapping we move the hole up
int hole = ++currentSize;
for ( ; hole > 1 && x < array[ hole / 2 ]; hole /= 2 )
array[ hole ] = array[ hole / 2 ];
array[ hole ] = x;
} 49
INSERTION
13
21 16
Create a hole
24 31 19 100
65 26 32
14
0 1 2 3 4 5 6 7 8 9 10 11
13 21 16 24 31 19 100 65 26 32
Hole
50
INSERTION
13
21 16
24 19 100
65 26 32 31
14
0 1 2 3 4 5 6 7 8 9 10 11
13 21 16 24 19 100 65 26 32 31
Hole
51
INSERTION
13
16
24
2 19 100
1
65 26 32 31
14
0 1 2 3 4 5 6 7 8 9 10 11
13 16 24 21 19 100 65 26 32 31
Hole
52
INSERTION
13 We want to insert 14
14 16
24 21 19 100
65 26 32 31
14 Heap Property is now

established
0 1 2 3 4 5 6 7 8 9 10 11
13 14 16 24 21 19 100 65 26 32 31
Hole
53
insert
• If an element needs to be percolated up d
levels, the insert operation uses d+1
assignment operations
• A swap would have required 3 assignments per
swap, for a total 3d swaps for percolating d
levels.
• Thus insert takes O(log N) time at the

maximum.
• When the new element is also the new
minimum.
54
DELETING THE MINIMUM
• Finding the minimum is easy, the hard part

is removing it!
• Get the minimum value from the root
• Since the new heap has one less element put
the last element at the now vacant root.
• Percolate down until the heap property is
established
55
13 Remove 13 from the root
14 16
24 21 19 100
65 26 32 31
56
Remove 13 from the root
14 16
24 21 19 100
65 26 32 31
57
31
14 16
Put the last element to the root
and remove the last node
24 21 19 100
65 26 32
58
14
31 16
24 21 19 100
Percolate 31 one level down
65 26 32
59
14
21 16
24 31 19 100
65 26 32
60
14
21 16
24 31 19 100
65 26 32
Now heap-order property is

re-established
One can also think of this as the hole at root moving down!
61
deleteMin()
/**
* Remove the smallest item from the priority queue.
* Throw Underflow if empty.
*/
void BinaryHeap<Comparable>::deleteMin( )
{
if ( isEmpty( ) )
throw Underflow( );
// move the last element to the first and shrink the array
array[ 1 ] = array[ currentSize–– ];
percolateDown( 1 );
}
62
deleteMin(...)
/**
* Remove the smallest item from the priority queue
* and place it in minItem. Throw Underflow if empty.
*/
void BinaryHeap<Comparable>::deleteMin( Comparable & minItem )
{
if ( isEmpty( ) )
throw Underflow( );
minItem = array[ 1 ];
array[ 1 ] = array[ currentSize-- ];
percolateDown( 1 );
}
63
(private) percolateDown
/**
* Internal method to percolate down in the heap.
* hole is the index at which the percolate begins.
*/
void BinaryHeap<Comparable>::percolateDown( int hole )
{
int child;
Comparable tmp = array[ hole ]; // tmp is the item that will
// be percolated down
for( ; hole * 2 <= currentSize; hole = child )
{
child = hole * 2;
if ( child != currentSize && array[ child + 1 ] < array[ child ] )
child++;
if ( array[ child ] < tmp )
array[ hole ] = array[ child ];
else
break;
}
array[ hole ] = tmp; 64
/**
*/
{
int child;
{
child = hole * 2;
child++; // child is the minimum of the children
array[ hole ] = array[ child ];
else
break;
}
/**
*/
{
int child;
{
child = hole * 2;
array[ hole ] = array[ child ]; // swap hole and min child
else
break;
}
/**
*/
{
int child;
{
child = hole * 2;
array[ hole ] = array[ child ]; // swap hole and min child
else
break;
}
percolateDown
13 14 16 24 21 19 68 65 26 32 31
0 1 2 3 4 5 6 7 8 9 10 11
68
percolateDown
13 14 16 24 21 19 68 65 26 32 31
0 1 2 3 4 5 6 7 8 9 10 11
14 16 24 21 19 68 65 26 32 31
31 tmp = 31
0 1 2 3 4 5 6 7 8 9 10 11
69
percolateDown
13 14 16 24 21 19 68 65 26 32 31
0 1 2 3 4 5 6 7 8 9 10 11
14 16 24 21 19 68 65 26 32 31
31 tmp = 31
0 1 2 3 4 5 6 7 8 9 10 11
14 14
31 16 24 21 19 68 65 26 32 31 tmp = 31
child = 2
0 1 2 3 4 5 6 7 8 9 10 11
70
percolateDown
13 14 16 24 21 19 68 65 26 32 31
0 1 2 3 4 5 6 7 8 9 10 11
14 16 24 21 19 68 65 26 32 31
31 tmp = 31
0 1 2 3 4 5 6 7 8 9 10 11
14 14
31 16 24 21 19 68 65 26 32 31 tmp = 31
child = 2
0 1 2 3 4 5 6 7 8 9 10 11
14 21 16 24 31 19 68 65 26 32 31 tmp = 31
child = 5
0 1 2 3 4 5 6 7 8 9 10 11
71
percolateDown
14 16 24 19 68 65 26 32 31 tmp = 31
21 31
child = 10
0 1 2 3 4 5 6 7 8 9 10 11
Since the only child of node 5 is NOT less than tmp,

tmp is stored at 5
Running time for percolate down is O(log N)
72
PRIORITY QUEUES - HEAPS
• A priority queue is a data structure that

allows at least the following operations:
• insert
• delete the element with the minimum value
(deleteMin)
• Both operations can be implemented in

O(log N) time, by using HEAPS
73
OTHER HEAP OPERATIONS
• buildHeap
• take N items and place them into an empty
heap.
• obviously this can be done with N inserts
• O(1) average (experimental), O(log N) worst case
• Total O(N) average, O(N log N) worst case
• Can we guarantee a O(N) worst-case bound?
74
buildHeap
/**
* Establish heap order property from an arbitrary
* arrangement of items. Runs in linear time.
*/
void BinaryHeap<Comparable>::buildHeap(
vector<Comparable> Input)
{
array = Input; // copy input array to private array;
currentSize = Input.size();
for ( int i = currentSize / 2; i > 0; i–– )
percolateDown( i );
}
75
buildHeap
150 80 40 30 10 70 110 100 20 90 60 50 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
76
buildHeap
150 80 40 30 10 70 110 100 20 90 60 50 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
After percolateDown(7)
150 80 40 30 10 70 110 100 20 90 60 50 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
77
buildHeap
150 80 40 30 10 70 110 100 20 90 60 50 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
150 80 40 30 10 70 110 100 20 90 60 50 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
150 80 40 30 10 50 110 100 20 90 60 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
78
buildHeap
150 80 40 30 10 50 110 100 20 90 60 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
79
buildHeap
150 80 40 30 10 50 110 100 20 90 60 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
150 80 40 20 10 50 110 100 30 90 60 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
80
buildHeap
150 80 40 30 10 50 110 100 20 90 60 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
150 80 40 20 10 50 110 100 30 90 60 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
150 80 40 20 10 50 110 100 30 90 60 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
81
buildHeap
After percolateDown(2) 1st level
150 10 40 20 80 50 110 100 30 90 60 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
82
buildHeap
150 10 40 20 80 50 110 100 30 90 60 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
After percolateDown(2) 2nd level
150 10 40 20 60 50 110 100 30 90 80 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
83
buildHeap
10 150 40 20 60 50 110 100 30 90 80 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
84
buildHeap
10 150 40 20 60 50 110 100 30 90 80 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
10 20 40 150 60 50 110 100 30 90 80 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
85
buildHeap
10 150 40 20 60 50 110 100 30 90 80 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
10 20 40 150 60 50 110 100 30 90 80 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
After percolateDown(1) 3rd level
10 20 40 30 60 50 110 100 150 90 80 70 120 140 130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
We now have the heap-order property established.
86
COMPLEXITY OF buildHeap
0 ... N/8 ... N/4 ... .. N/2 .. .. .. .. .. .. .. N
87
0 ... N/8 ... N/4 ... .. N/2 .. .. .. .. .. .. .. N
For these we make 0 calls to percolateDown
88
0 ... N/8 ... N/4 ... .. N/2 .. .. .. .. .. .. .. N
For these we make AT MOST 1 call to percolateDown
89
0 ... N/8 ... N/4 ... .. N/2 .. .. .. .. .. .. .. N
90
0 ... N/8 ... N/4 ... .. N/2 .. .. .. .. .. .. .. N
For these we make AT MOST 3 calls to percolateDown
91
Theorem: The sum of the heights of the nodes
in a perfect binary tree of height h is
2h+1 – 1 – (h + 1)
Proof: Let S be the sum of the heights
92
Since a complete tree is not a perfect tree, S is an

upper bound on the sum of the heights of the nodes in
a complete tree of height h.
Since a complete tree of height h has between 2h and

2(h+1)-1 nodes, S is O(N).
Thus, buildHeap is O(N)
93
APPLICATIONS
• Heap Sort
• The Selection Problem

• Algorithm 1
• Algorithm 2
94
HEAP SORT
•Heap Sort
• Input an array of N items
• buildHeap (takes O(N) time)
• perform N deleteMin operations (takes
O(N log N) time
• Total Sort time = O(N) + O(N log N)
= O(N log N)
95
SELECTION
•The selection problem is the following

• Given a list of N elements, and an integer k,
• find the kth smallest element.
96
SELECTION
•Algorithm 1
• Read the N elements into an array
• buildHeap (takes O(N) time)
• perform k deleteMins (takes O(k log N)
time)
• the last element obtained is our result
97
SELECTION
• Algorithm 1
• If k is small (k = O(N / log N)), the running time is
dominated by the buildHeap operation ⇒
O(N)
• For larger values of k, selection takes O(k log N)
time.
• k = ⎡N/2⎤ ⇒ Selection takes Θ(N log N) time
• Note that heap sort is a special case of this
algorithm.
• By redefining the heap-order property we can
also find the kth largest element.
98
SELECTION
•Algorithm 2 (but let us find the kth
largest element)
• We maintain a set of S of the k largest
elements.
• Let Sk be the smallest element in S.
• Look at a new element.
• If the new element is larger than Sk, then it
replaces it in S.
• S then will have a new smallest element.
• After looking at all the other elements Sk is the
answer
99
SELECTION
•Algorithm 2 (but let us find the kth
largest element)
• Take the first k elements and buildHeap
(takes O(k) time)
• For the remaining N-k elements
• O(1) to see if it has to replace Sk
• O(log k) to delete Sk and find the new minimum
if necessary.
• Total time is O(k + (N-k) log k) = O(N log k)
100

10 Heaps

Uploaded by

Copyright:

Available Formats

You might also like

10 Heaps

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

10 Heaps

Uploaded by

Copyright:

Available Formats

PRIORITY QUEUES - HEAPS

•A priority queue is a data structure that

• Binary Search Trees look OK but are actually

• A simplified binary tree called a HEAP is

• Other examples of complete binary trees

Perfect binary tree of height 0 Perfect binary tree of height 1

• Total number of nodes in a

Perfect binary tree of height 2 • Prove these!

• A complete binary tree of height h has

• This implies that the height of a complete

• Since the complete binary tree is very

• For an entry at array location i, (1 based)

Note that we could also have used an array starting with 0

D E F G Typically this can be handled by

bool isEmpty( ) const;

void insert( const Comparable & x );

bool isEmpty( ) const;

void insert( const Comparable & x );

bool isEmpty( ) const;

void insert( const Comparable & x );

bool isEmpty( ) const;

void insert( const Comparable & x );

• In addition to the structure property, we also

24 31 19 100 Heap property violated

24 14 19 100 Heap property violated

24 14 19 100 Heap property violated

24 21 19 100 Heap property violated

24 21 19 100 Heap property violated

Heap Property is now

14 Heap Property is now

• Thus insert takes O(log N) time at the

• Finding the minimum is easy, the hard part

Now heap-order property is

Since the only child of node 5 is NOT less than tmp,

Running time for percolate down is O(log N)

• A priority queue is a data structure that

• Both operations can be implemented in

150 80 40 30 10 70 110 100 20 90 60 50 120 140 130

150 80 40 30 10 70 110 100 20 90 60 50 120 140 130

150 80 40 30 10 50 110 100 20 90 60 70 120 140 130

150 80 40 30 10 50 110 100 20 90 60 70 120 140 130

150 80 40 30 10 50 110 100 20 90 60 70 120 140 130

150 80 40 20 10 50 110 100 30 90 60 70 120 140 130

150 80 40 30 10 50 110 100 20 90 60 70 120 140 130

150 80 40 20 10 50 110 100 30 90 60 70 120 140 130

150 80 40 20 10 50 110 100 30 90 60 70 120 140 130

150 10 40 20 80 50 110 100 30 90 60 70 120 140 130

150 10 40 20 80 50 110 100 30 90 60 70 120 140 130

After percolateDown(2) 2nd level

150 10 40 20 60 50 110 100 30 90 80 70 120 140 130

10 150 40 20 60 50 110 100 30 90 80 70 120 140 130

10 150 40 20 60 50 110 100 30 90 80 70 120 140 130

After percolateDown(1) 2nd level

10 20 40 150 60 50 110 100 30 90 80 70 120 140 130

10 150 40 20 60 50 110 100 30 90 80 70 120 140 130

After percolateDown(1) 2nd level

10 20 40 150 60 50 110 100 30 90 80 70 120 140 130

10 20 40 30 60 50 110 100 150 90 80 70 120 140 130