Tree ADTs

CS 1037a Topic 15

Overview
General trees and terminology Binary trees and their traversals Binary search trees (BST)
a more efficient alternative to OrderedList

15-2

Related materials
from Main and Savitch Data Structures & other objects using C++
Chapter 10: Trees
- Sec. 10.1: Introduction to Trees - Sec. 10.2: Tree representations - Sec. 10.3 Binary tree nodes - Sec. 10.4 Tree traversals - Sec. 10.5 Binary Search Trees

Tree Data Structures

Trees are non-linear data structures that are often used to represent entities in a hierarchical relationship. Examples:
Class hierarchies in C++ programs Computer file system (folders, subfolders) Decision-making process (decision trees) Family tree Table of contents in a book
15-4

Example: Computer File System

Root directory of C drive

Documents and Settings

Program Files

My Music

Desktop

Favorites

Start Menu

Adobe

Microsoft Office

15-5

Example: Spanning Tree

(from Assignment 4)
9 8 9 5 2

15
pixels
(nodes)
7

9
6

3
1

2
2

3
1

8
2

3
1

2
2

0
9

2
7

6
direction toParent
1

5
4

3
7

2
8

3
9

seed
(root)

11

10

A tree spanning image pixels

(describing live-wire paths to the seed)
15-6

Tree Data Structures

A tree is a collection of items stored in nodes
Relations between nodes are called edges Root: the origin of the tree;
- a tree has a single root

15-7

General Trees
Simplest tree is the empty tree: no nodes, no edges General tree is the empty tree, or a set of one or more nodes partitioned into disjoint subsets:
A single root node The subtrees of the root, also general trees, each connected to the root by a distinct edge
15-8

Root node

General Trees

Interior nodes Leaf nodes

15-9

General Trees

Subtrees of the root node

15-10

General Trees
E

Subtrees of the node labeled E

15-11

General Trees
If nodes X and Y are connected by an edge and X is closer to the root than Y
X is the parent (or predecessor) of Y Y is a child (or successor) of X The ancestors of Y are X, the parent of X, the parent of the parent of X, , and the root Y, the children of Y, the children of the children of Y, etc, are descendants of X
15-12

ancestors of Y

General Trees
parent of Y

parent

edges

child of X

child of X

child

child

child

descendants of X 15-13

Example: Spanning Tree

(from Assignment 4)
9 8 9 5 2

15
7

9
6

3
1

2
2

3
1

8
child 2

3
1 parent

2
2

0
9

2
7

6 8

5 6

3
7

2
8

3
9

root
(seed)

edges

child

11

10

Tree spanning image pixels

(describing live-wire paths to the seed)
15-14

General Trees
The root is the only node in a tree that has no parent Every other node has exactly one parent Leaves are nodes with no children An interior node is a node that is not a leaf Two nodes are siblings if they have the same parent
15-15

General Trees
A path is a sequence of edges leading from one node to another
- examples of paths from a node to the root
9

15 8

9 3

3 2

2 0

3 2

6
8

5
6

3
5

Part of the assignment was to compute a tree with optimal paths to the seed in case when a cost of a path from any given node includes weights of nodes on the path, e.g.

11 10

6=1+2+1+2

Example 1

Example 2: live-wire (Assignment 4)

- tracing a path along given spanning tree

General Trees
A path is a sequence of edges leading from one node to another The length of a path is the number of edges on the path The depth of a node N is the length of the path from the root to N The height of a tree rooted at node N is the length of the longest path from N to a leaf
15-17

General Trees
The height of a (non-empty) tree is the height of its root node In other words, the height of a tree is the length of the longest path from the root to a leaf node Height of a tree with only one node is 0 Height of the empty tree is often considered to be -1
15-18

General Trees
All nodes with depth N are said to be at level N of the tree The degree (or arity) of a node is the number of children it has The degree (or arity) of a tree is the maximum of the degrees in the trees nodes
15-19

General Trees
Level 0 Root node has degree 4; leaf nodes have degree 0

E
Level 1

Level 2 The tree has height 3

Level 3

Node E has depth 1; the subtree rooted at E has height 2 Node E has degree 3
15-20

Binary Trees
Binary tree: a tree with degree (arity) 2 Each node in a binary tree has a data portion, plus references to its left and right successors Recursive nature of binary tree structure is very important when we design algorithms to operate on the tree
15-21

Binary Tree Nodes

We will use binary tree nodes that contain
data Item two pointers to its left and right children
comment: pointer to a parent node can also be used in trees (as in Assignment 4)

data item

In our drawings we use simplified diagrams

Node* left

Node* right
15-22

Binary Tree Traversals

Traversal of a data structure requires that each item in the data structure be visited once What we do when we visit an item depends on the application (modify the value of the item, print out the value, etc) There is no forced order for a traversal
15-23

Binary Tree Traversals

However, with linear data structures, there is usually a natural order wed choose for a traversal With trees, there are several standard orders for traversal; which one(s) we use depends upon our needs

15-24

Preorder Traversal
Start at the root, and use recursion:
if the tree is not empty {

Visit the item at the root node r

/* 1 */
/* 2 */

Perform a preorder traversal of the left subtree of r

Perform a preorder traversal of the right subtree of r /* 3 */ }

15-25

Preorder Traversal
A B C

Well trace the different traversals using this tree; recursive calls, returns, and visits will be numbered in the order they occur
15-26

Preorder Traversal
A 2 1: visit A 24 23 45 B 3: visit B C 25: visit C 4 16 26 38 15 22 37 44 39: D 5: visit D E 17: visit E F 27: visit F G visit G 18 8 28 21 40 19 20 34 35 36 41 42 43 7 14 H 9: visit H I 29: visit I 30 33 10 13 1112 31 32

. .

. .
15-27

. .

. .

Nodes are visited in the order ABDHECFIG

Inorder Traversal
Only difference from preorder traversal is that the data object is visited after the recursive call for the left subtree:
if the tree is not empty { Perform an inorder traversal of the left subtree of r. Visit the item at the root node r. /* 2 */ /* 1 */

Perform an inorder traversal of the right subtree of r. /* 3 */

}
15-28

Inorder Traversal
A 1 23: visit A 24 22 45 B 14: visit B C 37: visit C 2 15 25 38 13 21 36 44 41: D 5: visit D E 18: visit E F 33: visit F G visit G 16 6 26 20 39 17 19 32 34 35 40 42 43 4 12 H 9: visit H I 29: visit I 27 31 7 11 8 10 28 30

. .

. .
15-29

. .

. .

Nodes are visited in the order DHBEAIFCG

Postorder Traversal
In this case, the data object in a node is not visited until after the contents of both its subtrees have been visited:
if the tree is not empty { Perform a postorder traversal of the left subtree of r. /* 1 */

Perform a postorder traversal of the right subtree of r. /* 2 */

Visit the item at the root node r.

}

/* 3 */

15-30

Postorder Traversal
A 1 45: visit A 23 22 44 B 21: visit B C 43: visit C 2 14 24 36 13 20 35 42 41: D 12: visit D E 19: visit E F 34: visit F G visit G 15 5 25 18 37 16 17 31 32 33 38 39 40 4 11 H 10: visit H I 30: visit I 26 29 6 9 7 8 27 28

. .

. .
15-31

. .

Nodes are visited in the order HDEBIFGCA

Recursive Binary Tree Traversals

Note that the relative order of the recursive calls in preorder, inorder and postorder traversals is the same The only differences stem from where the visiting actually takes place

15-32

Iterative Traversals of Tree Nodes

Recursive tree traversals use the call stack to keep track of where were working in the tree In iterative traversals, the programmer keeps track Iterative traversal uses a container to store (addresses of) nodes not yet visited Order of visiting will depend on the type of container being used (stack, queue, bag, etc)
15-33

Iterative Traversals of Tree Nodes

// PRECONDITION: the tree is not empty Create an empty container to hold (pointers to) nodes to be visited Put the root node address into the container While the container is not empty {

Remove a pointer T from the container

Visit the node T points to Put pointers to all non-empty successors of T in the container

}
NOTE: image pixels in labs 9-10 and assignment 4 form a grid (graph), not a tree. Yet, iterative traversals of tree nodes are graph nodes (e.g. image pixels) are similar. Compare this code with floodFill_xxx() based on different active front containers.

Iterative Traversals of Tree Nodes

If the container is a bag, order of visits will be unpredictable (exercise: implement FloodFill_Bag() in lab 9-10)
If the container is a stack, and we push the right child of a node before the left child, we get preorder traversal (analogous to Depth-First-Search in labs 9-10)

If the container is a queue, and we enqueue the left successor before the right, we get a level order traversal (analogous to Breadth-First-Search in labs 9-10)
15-35

Level Order Traversal

A B C

Nodes will be visited in the order ABCDEFGHI

15-36

Tree Traversal Exercises

Trace the iterative tree traversal algorithm using different kinds of containers (bag, stack, queue) Try to figure out iterative traversal algorithms that will yield inorder and postorder traversals

15-37

Algorithms Based on Traversals

Example 1: The shape of a binary tree can be described by a string of zeroes and ones; each node is described by two digits:
00 a leaf: no successors 01 has a right successor only 10 has a left successor only 11 has both left and right successors
15-38

Algorithms Based on Traversals

11
(1)

11
(2)

01
(6)

String representing a tree can be built using a preorder traversal of the tree: visiting a node corresponds to appending two digits to the string.

00
(3)

10
(4)

00
(7)

00
(5)

This tree is described by string

11110010000100
root left and right sub-trees

Algorithms Based on Traversals

11 11 00 00
Tree can be re-build from string Conversely, a tree can be built from the string description in preorder fashion: Save first two digits from the string. Build the root node.

01 10 00

If first digit is 1, use remainder of string to recursively build left sub-tree of root If second digit is 1, use remainder of string to recursively build right sub-tree of root

11110010000100

Algorithms Based on Traversals

Example 2: The destructor for a binary tree can be written using a postorder traversal:
Visiting a node is node deletion Both subtrees of a node N are deleted before N is deleted We do this in the binary search tree code, shown later
15-41

Algorithms Based on Traversals

(7)

a tree can be deleted in post-order fashion (using pseudo-code): deleteSubtree ( root ) {

(4)

(6)

deleteSubtree( left child ); deleteSubtree( right child );

(1) (3) (5)

delete root; }

// visiting a node

(2)

Operations on Binary Trees

Searching a general binary tree to locate an object requires a traversal of the tree; we may have to visit all nodes Removal involves searching for a node with a specific value and deleting the node. Often, this requires reconfiguring the tree so that its still a binary tree; algorithm may vary depending on the application
15-43

Operations on Binary Trees

Inserting involves searching for an appropriate location and then adding a node in such a way that the resulting structure is still a binary tree; again, the algorithm may depend on the application

15-44

Binary Search Trees (BST)

15-45

Binary Search Trees

A binary search tree (BST) is a special case of binary tree (its nodes should satisfy BST property, see next slide) BST is a very efficient data structures for storing and maintaining an ordered set of data
a better alternative to our OrderedList basic operations (e.g. insert and remove) for BST have better complexity than similar operations for OrderedList

15-46

Binary Search Trees

Definition: BST is a binary tree where each node N satisfies the following BST property:
all nodes L in the left sub-tree of N have data items such that (Item at L) <= (Item at N) all nodes R in the right sub-tree of N have data items such that (Item at R) >= (Item at N)

Items in BST should have at least one overloaded comparison operator <, >, <=, or >=
15-47

Binary Search Trees

A binary tree is a binary search tree if all nodes in the tree have the BST property
5 3 7 7 2 5 8 5 5 8
15-48

2 3

Traversal of Binary Search Tree

An inorder traversal visits the nodes of the binary search tree in sorted order
5
(4)
if the tree is not empty { Perform an inorder traversal of the left subtree of r. print out item value at the root node r.

3
(2)

7
(5)
}

Perform an inorder traversal of the right subtree of r*

2
(1)

5
(3)

8
(6)

Print out:

235578

15-49

Searching Binary Search Trees

14 To find item k starting at tree root: If the tree is empty, return false. 8 26 If k < item at root Return the result from recursively

searching the left sub-tree

3 12 19 Else if item at root < k Return the result from recursively 9 23

searching the right sub-tree

Else root item is equivalent to k, return true
15-50

Searching Binary Search Trees

14 14

26

26

12

19

12

19

23

23

Search for 13: visited nodes are coloured red; return false when node containing 12 has no right successor

Search for 22: return false when node containing 23 has no left successor 15-51

Binary Search Tree Insertion

When we insert an item, we need to add the new node in such a way that the resulting tree is still a BST New node will be added as a leaf Algorithm is very similar to the search algorithm

15-52

Binary Search Tree Insertion

To insert a value k into a tree with a given "root"
If the tree is empty (no root node) Build a new root node containing k Else if k < (item at root) Insert value k into the left sub-tree of the root Else Insert value k into the right sub-tree or the root

15-53

Binary Search Tree Insertion

14 To insert 13: 8 26 Same nodes are visited as when searching for 13. Instead of returning false when the node containing 12 has no right successor, build the new node, attach it as the right successor of the node containing 12, and return true.

12

19

23

13
15-54

Removing Min/Max from Binary Search Tree

5 4 6
To remove min: - go to the left from the root until no left child, then the node with min value is found and its value (1) can be returned

5 7 4

8 11

3 2

15-55

Removing Min/Max from Binary Search Tree

5 4 6
To remove min: - go to the left from the root until no left child, then the node with min value is found and its value (1) can be returned - the node should be deleted, but what to do about its possible right sub-tree?

5 7 4

8 11

3 2

15-56

Removing Min/Max from Binary Search Tree

5 4 6
To remove min: - go to the left from the root until no left child, then the node with min value is found and its value (1) can be returned - the node should be deleted, but what to do about its possible right sub-tree?

5 7 4

8 11

- since the node has one child only, its parent can "adopt it.

3 2

BST property is preserved, why?

15-57

Removing Min/Max from Binary Search Tree

To remove minimum Item from a BST tree
PRECONDITION: the tree root can not be empty

If root has a left child - return min Item removed from the left sub-tree (recursively)

Else, the root has no left child, thus its value is the minimum in the tree - save current root value as SAVED - the right child (if any) becomes a new root, the old root is deleted - return SAVED value

Maximum item can be removed and returned similarly.

15-58

Removing Arbitrary Item from Binary Search Tree

if Item is a leaf node (no children)
delete the node, the tree needs no further arrangement as it still satisfies the BST property

if Item is an interior node with one child

Re-link the node's parent with its only child and then delete the node

Why BST property is preserved?

15-59

Removing Arbitrary Item from Binary Search Tree

if Item is an interior node with both children
keep the node, but replace its value with the minimum Item removed from the right sub-tree alternatively, replace node's value by the maximum Item removed from the left sub-tree

Why BST property is preserved?

15-60

Time Complexities for Tree Operations

Binary Tree:
All traversals are O(n): each node is visited exactly once

Binary Search Tree:

find, insert, and remove operations are O(tree height)

- in the worst case, this is O(n); - in best and average cases, its O(log2(n))
15-61

BST.h
#include Queue.h #include "StandardFunctors.h" template <class Item, class Order=IsLess> class BST { public:
class Node { public: Node( Item val, Node* ln = NULL, Node* rn=NULL ) : value(val), left(ln), right(rn) { } Item value; Node * left, *right; };

BST( ); ~BST( );

// constructor for empty binary search tree // BST destructor // contd..

15-62

BST.h
bool isEmpty( ) const; int size( ) const; // true if BST is empty, false otherwise // number of nodes in the BST

bool find( Item key ) const;

// true if an Item equivalent to key // is found, and false otherwise

void insert( Item item ); // insert item in a new leaf node // at an appropriate place in the tree

Item removeMin( ); // removes the minimum Item from BST // Precondition: BST should not be empty
Item removeMax( ); // removes the minimum Item from BST // Precondition: BST should not be empty // contd..

15-63

BST.h
Queue<Item> * preorder( ) const; // Return a queue that contains, in preorder traversal order, // copies of items stored inside BST
Queue<Item> * inorder( ) const; // Return a queue that contains, in inorder traversal order, // copies of items stored inside BST Queue<Item> * postorder( ) const; // Return a queue that contains, in postorder traversal order, // copies of items stored inside BST contd..

15-64

BST.h
private: Node * m_root; int m_size; void deleteSubtree( Node * &here ); // call by reference // delete the subtree rooted at here bool find( Item key, Node * here ) const; // return true if item identical to key is found in the sub-tree // rooted at here; return false otherwise void insert( Item item, Node* &here ); // call by reference // insert item in the sub-tree rooted at node here

contd..
15-65

BST.h
Item removeMax( Node* &here ); // call by reference // removes the Maximum item from the sub-tree rooted at // node here and returns its value. // PRECONDITION: sub-tree can not be empty

Item removeMin( Node* &here ); // call by reference // removes the Minimum item from the sub-tree rooted at // node here and returns its value. // PRECONDITION: sub-tree can not be empty contd..

15-66

BST.h
void preorder( Queue<Item> *q, Node * here ) const; // Traverse the nodes in the sub-tree rooted at here in preorder // fashion, and enqueue copies of Items into q in that order void inorder( Queue<Item> *q, Node * here ) const; // Traverse the nodes in the sub-tree rooted at here in inorder // fashion, and enqueue copies of Items into q in that order

void postorder( Queue<Item> *q, Node * here ) const; // Traverse the nodes in the sub-tree rooted at here in postorder // fashion, and enqueue copies of Items into q in that order
}

#include BST.template
// end of BST.h
15-67

BST.template
template <class Item, class Order> BST<Item,Order> :: BST( ) : m_root (NULL), m_size (0) { } template <class Item , class Order> BST<Item,Order> :: ~BST( ) { deleteSubtree( m_root ); // call by reference } template <class Item , class Order> bool BST<Item,Order> :: isEmpty( ) const {return (m_size == 0); }

template <class Item , class Order>

int BST<Item,Order> :: size( ) const {return m_size; } // contd..
15-68

BST.template
template <class Item , class Order> bool BST<Item,Order> :: find( Item key ) const { return find( key, m_root ); } template <class Item , class Order> void BST<Item,Order> :: insert( Item item ) { insert( item, m_root ); // call by reference } // contd..

15-69

BST.template
template <class Item , class Order> Item BST<Item,Order> :: removeMax( ) { // PRECONDITION: tree can not be empty return removeMax( m_root ); // call by reference } template <class Item , class Order> Item BST<Item,Order> :: removeMin( ) { // PRECONDITION: tree can not be empty return removeMin( m_root ); // call by reference } // contd..

15-70

BST. template
template <class Item , class Order > Queue<Item> * BST<Item,Order> :: preorder( ) const { Queue<Item> * q = new Queue<Item>( ); preorder( q, m_root ); return q; } template <class Item , class Order> Queue<Item> * BST<Item,Order> :: inorder( ) const { Queue<Item> * q = new Queue<Item>( ); inorder( q, m_root ); return q; } template <class Item , class Order> Queue<Item> * BST<Item,Order> :: postorder( ) const { Queue<Item> * q = new Queue<Item>( ); postorder( q, m_root ); return q; } // contd..

15-71

BST. template
template <class Item , class Order> void BST<Item,Order> :: deleteSubtree( Node * &here ) { if (here == NULL) return; // base case deleteSubtree( here->left ); // call by reference

deleteSubtree( here->right ); // call by reference

delete here; m_size--; here = NULL; // this is where call by reference helps!!! } // contd..
15-72

BST. template
template <class Item , class Order> bool BST<Item,Order> :: find( Item key, Node * here ) const { if (here == NULL) return false; // base case Item current = here->value; if (Order::compare(key , current)) return find(key,here->left);

else if (Order::compare(current , key)) return find(key,here->right); else } /* "current == key" */ return true; // contd..

15-73

BST. template
template <class Item, class Order> void BST<Item,Order> :: insert( Item item, Node* &here ) { if ( here == NULL ) { // base case here = new Node(item); // this is where call by reference helps!!! m_size++;

}
else if (Order::compare(item , here->value)) insert(item,here->left); else insert(item,here->right);

// contd..

15-74

BST. template
template <class Item, class Order> Item BST<Item,Order> :: removeMax( Node* &here ) {
// PRECONDITION: sub-tree can not be empty (i.e. here!=NULL) if (here->right != NULL) return removeMax(here->right);

else { // no right child => here is a node with MAX value

Item saved = here-> value; Node * tmp = here; here = here->left; // this is where call by reference helps!!!

delete tmp; // deleting a node

m_size--; return saved; } } // 15-75 contd..

BST. template
template <class Item, class Order> Item BST<Item,Order> :: removeMin( Node* &here ) {
// PRECONDITION: sub-tree can not be empty (i.e. here!=NULL) if (here->left != NULL) return removeMin(here->left);

else { // no left child => here is a node with MIN value

Item saved = here-> value; Node * tmp = here; here = here->right; // this is where call by reference helps!!!

delete tmp; // deleting a node

m_size--; return saved; } } // 15-76 contd..

BST. template
template <class Item, class Order>

void BST<Item,Order>:: preorder( Queue<Item> *q, Node * here ) const {

if ( here == NULL ) return; // base case q->enqueue( here->value ); preorder( q, here->left ); preorder( q, here->right ); } // contd..

15-77

BST. template
template <class Item, class Order>

void BST<Item,Order> :: inorder( Queue<Item> *q, Node * here ) const {

if ( here == NULL ) return; // base case inorder( q, here->left ); q->enqueue( here->value ); inorder( q, here->right ); } // contd..

15-78

BST. template
template <class Item, class Order>

void BST<Item,Order> :: postorder( Queue<Item> *q, Node * here ) const {

if ( here == NULL ) return; // base case postorder( q, here->left ); postorder( q, here->right ); q->enqueue( here->value ); } // end of BST.template

15-79

Additional BST Functions

Exercise: implement two extra member functions
public: bool remove ( Item key ); // if item identical to key is in the BST, // remove it and return true, otherwise // do not change tree and return false; private: bool remove ( Item key, Node * & here ); // if item identical to key is in the sub-tree rooted at node here, // remove it and return true, o.w. do nothing and return false;

HINT: see code for function find() and slides 53-54

15-80

BST vs Ordered List

Binary Search Tree is a very efficient data structure for maintaining ordered data sets with average case complexity O(log2(n)) for insert and remove operations
- additional tricks maybe necessary to keep the tree balanced (height
log2(n))

In contrast, OrderedList operations insert and remove are O(n)

- applies to array-based and LinkedList-based OrderedLists (as discussed earlier)

15-81

BST as a Priority Queue

priority queue differs from standard queue as follows: instead of FIFO order, the items are dequeued according to their priorities, which can be defined by overloaded comparison operators < or >. For example, dequeue for priority queue can return an item with the largest (or the smallest) value in the queue.
average case complexity

Can use BST to implement a priority queue. For example, use

for enqueue use insert for dequeue use removeMax

- O(log2(n))

- O(log2(n))

Question: what would be complexity of enqueue and dequeue if priority queue used OrderedList container? 15-82

Priority Queue
In fact, priority queue can be implemented even more efficiently with guaranteed worst case complexity O(log2(n)) for enqueue and dequeue.
based on binary trees (heaps) different from BST - optional text-book reading: Sec. 11.1 Heaps

15-83

That is it for CS1037!

(see summary on the next slide)

Summary of CS1037
dynamic and static memory allocation, pointers C++, classes, templates, object-oriented programming basic data structures: (array-based lists, linked-lists, stacks, queues, trees) basic algorithms (e.g. sorting) and analysis of their complexity

This is only where it starts... if you take Computer Science courses on algorithms, data structures, optimization, ...