CS251: Algorithms

Data Structures Review

Linked Lists
• A linked list consists of a finite sequence of elements or
nodes that contain information plus (except possibly the
last one) a link to another node.
• There is a link to the first element called the head of the
list.
• If there is a link from the last element to the first, the list is
called circular.
• If in a linked list each node (except possibly the first one)
points also to its predecessor, then the list is called a doubly
linked list.
• If the first and the last nodes are also connected by a pair
of links, then we have a circular doubly linked list.
 Linked Lists

a) Single linked list

b) Circular singly
linked list
c) doubly linked list
d) Doubly circular
linked list

The two primary operations on linked lists are insertion and deletion.
Stacks

• A stack is a linked list in which insertions and

deletions are permitted only at one end, called the
top of the stack.
• stack supports two basic operations: (If S is a stack)
• pop(S) returns the top of the stack and removes it
permanently.
• push(S, x) adds x to S and updates the top of the
stack so that it points to x.
Queues
• A queue is a list in which insertions are permitted
only at one end of the list called its rear, and all
deletions are constrained to the other end called the
front of the queue.
• The operations supported by queues are
• ADD(Q,x) adds x at the rear of the queue named Q.
• Delete(Q,x) delete x from the front of the queue
named Q.
Binary Trees

 A binary tree is a tree in which nodes can have a

maximum of 2 children.

 A binary tree differentiates between the children by

calling them the left child and right child and every
node with one child designates it either as a left
child or right child.
 A node in a binary tree is full if it has degree=2.
 Tree vocabulary
 Root: A (only node with parent==null)
 Children(B) = E, F
 Siblings(X) = {Nodes with the same
 parent as X, excluding X}
 Siblings(B) = {C, D}, Siblings(A) = {}
 Descendants(X) = {Nodes below X}
 Descendants(A) = {B,C,D,E,F,G}
 Ancestors(X) = {Nodes between X and
 the root}
 Ancestors(E) = {B, A}
 Nodes with no children are called
 leaves, or external or terminal nodes: {C, E, F, G}
 Internal nodes: Nodes with children are called / all
vertices other than leaves : {A, B, D}
 The subtree rooted at X is the tree of
 all descendents of X, including X.
Tree Measures
• Depth (level) of a node :The length of the unique
path from the root to that node
– Depth(x) = number of ancestors of x.
– Notice: Depth(x) = 1 + Depth(x.parent)
– Root depth = 0

• The depth of a tree: is equal to the depth of the

deepest leaf

• Height of a node :length of the longest path from that

node to a leaf
– all leaves are at height 0
• The height of a tree : is equal to the height of
the root :
– The maximal level of any node in the tree (equal to height of
its root)
Measuring Trees - Example

• The path from node 88 to node 43 is highlighted in red.

• The length of the path from node 88 to node 43 is 3.
• The height of node 43 is 3.
• The height of the tree is the height of node 10 which is 4.

The depth (level) of node 67 is the length of the 

path from node 10 to node 67 which is 2 10
The degree of node 43 is 3 and the degree of 
55
.node 25 is 2 and the degree of node 11 is 0 43 25
.The degree of the tree is 3 
21 17 67 13 90

15 99

11 88
 Special Types of Trees
• Full - Every node has exactly 4

two children in all levels, except

1 3
the last level. Nodes in last
2 16 9 10
level have 0 children
Full binary tree

• Complete - Full up to second

last level and last level is filled
4
from left to right
1 3
2 16 9 10
14 8 7
12
Complete binary tree
Binary Tree Properties
 There are at most 2i nodes at level i of a binary tree.
 A binary tree with depth d has at most 2d+1 -1 nodes.
k

 2 i
 2 k 1
 1
 Let n be the number of vertices in
i1
a binary tree T of
height k. Then:

 The equality holds if T ree is full.

 If Binary tree has height k, then O(log n) < k < n-1

Samples of Binary Trees

A A Complete Binary Tree

1 A
B B
Skewed Binary Tree
2
C B C

D 3 D E F G

E 5
4 H I
Binary Tree representation

 Sequential representation
If a complete binary tree with n nodes (depth =log n )
is represented sequentially, then for any node with
index i, 1<=i<=n, we have:
 parent(i) is at i/2 If i=1, i is at the root and has no parent.
 leftChild(i) is at 2i. If 2i >n, then i has no left child.
 rightChild(i) is at 2i+1 . If 2i +1 >n, then i has no right child.

 Linked Representation
data

left right
Traversing a Binary Tree
• Inorder :
– Root is printed between the values of its left and
right subtrees: left, root, right
• Preorder :
– root printed first: root, left, right
• Postorder :
– root printed last: left, right, root

5 Inorder: 2 3 5 5 7 9
Preorder: 5 3 2 5 7 9
3 7
2 5 9 Postorder: 2 5 3 9 7 5
Traversing a Binary Tree
INORDER (x)
1. if x  NULL
2. then INORDER ( x  left )
3. print ( x  key)
4. INORDER (( x  right )

• Running time:
– (n), where n is the size of the tree rooted at x
 Binary Search Trees
• Tree representation:
– A linked data structure in which each
node is an object
• the binary-search-tree property. parent
L R
– If y is in left subtree of x, key data
then key [y] < key [x]

– If y is in right subtree of x, Left child Right child

then key [y] ≥ key [x]

3 7
2 5 9
Binary Search Trees
• Support many dynamic set operations
– SEARCH, MINIMUM, MAXIMUM, INSERT, DELETE…

• Running time of basic operations on binary search

trees is bounded by the height of the tree.
– On average: (logn) The expected height of the tree is logn
– In the worst case: (n) The tree is a linear chain of n nodes
• The operations are fast if the height of the tree is small –
otherwise their performance is similar to that of a linked list
Searching for a Key
• Given a pointer to the root of a tree and a key k:5
– Return a pointer to a node with key k
3 7
if one exists
2 4 9
– Otherwise return NIL
• Idea
– Starting at the root: trace down a path by comparing k
with the key of the current node:
• If the keys are equal: we have found the key
• If k < key[x] search in the left subtree of x
• If k > key[x] search in the right subtree of x
TREE-SEARCH

TREE-SEARCH(x, k)
1. if x = NULL or k = x key
2. then return x
3. if k < x key
4. then return TREE-SEARCH(x  left ,k )
5. else return TREE-SEARCH(x  right , k )

15

6 18
3 7 17 20
2 4 13 • Search for key 13:
9 – 15  6  7  13
Finding the Minimum/Maximum in a BST

Minimum Following left child pointers from the root,

until a NIL is encountered
MINIMUM(x)
1. while x  left  NULL
2. do x ← x  left
3. return x
Maximum: Following right child pointers from the root,
until a NIL is encountered 15
MAXIMUM(x)
4. while x  right  NULL 6 20

5. do x ← x  right 3 7 17 22
2 4 13
6. return x
9
 Insertion
• Goal: Insert value v into a binary search tree
• Idea:
– Beginning at the root, go down the tree:
– If key [x] < v move to the right child of x,
else move to the left child of x
– When x is NULL, we found the correct position

12
Insert value 13
5 18
2 9 15 19
1 3 13 17
Deletion
• Goal:
– Delete a given node z from a binary search tree
• Idea:
– Case 1: z has no children
• Delete z by making the parent of z point to NIL

15 15

5 16 5 16

3 12 20 3 12 20
z
10 13 18 23 10 18 23

6 delete 6

7 7
Deletion
• Case 2: z has one child
– Delete z by making the parent of z point to z’s child,
instead of to z

15 delete 15
z
5 16 5 20

3 12 20 3 12 18 23

10 13 18 23 10

6 6

7 7
 Deletion
• Case 3: z has two children
– z’s successor (y) is the minimum node in z’s right subtree
– Replace z’s key and with y’s.
– Delete y from the tree (via Case 1 or 2)

6 15 15
delete z
5 16 6 16

3 12 20 3 12 20

10 13 18 23 10 13 18 23

y 6 7

7
 The Heap Data Structure
• A heap is a complete binary tree with the following two
properties:
– Structural property: complete. all levels are full, except
possibly the last one, which is filled from left to right
– Order (max-heap) property: for any node x, Parent(x) ≥ x

7 4
5 2
From the heap property, it follows that:
Heap The root is the maximum element of the heap!”
 Array Representation of Heaps
• A heap can be stored as an array A.
– Root of tree is A[1]
– Left child of A[i] = A[2i]
– Right child of A[i] = A[2i + 1]
– Parent of A[i] = A[ i/2 ]
– Heapsize[A] ≤ length[A]
• The elements in the subarray
A[(n/2+1) .. n] are leaves

26
Heap Operations
• Maintain/Restore the max-heap property
– Shift up / shift down
• Insert / delete
• Create a max-heap from an unordered array
• Sort an array in place

27
Shift-up
 Suppose the key stored in the 10th position of the heap shown
in Fig. 1 is changed from 5 to 25.

move H[i] up along the unique path from H[i] to the root until its proper
location along this path is found.

At each step along the path, the key of H[i] is compared with the key of its
parent H[[ i/2 ].
 Procedure SHIFT-UP
Input: An array H[1..n] and an index i between 1 and n.
Output: H[i] is moved up, if necessary, so that it is not larger than its
parent.
1. done = false
2. if i = 1 then exit {node i is the root}
3. repeat
4. if key(H[i]) > key(H[ [i/2]]) then interchange H[i] and H[ [i/2]]
5. else done = true
6. i = [i/2]
7. until i = 1 or done
 Shift-down
 Suppose we change the key 17 stored in the second position of the heap
shown in Fig. 1 to 3.

 At each step along the path, its key is compared with the maximum of
the two keys stored in its children nodes (if any).
Procedure SHIFT-DOWN
Input: An array H[1..n] and an index i between 1 and n.
Output: H[i] is percolated down, if necessary, so that it is
not smaller than its children.
1. done = false
2. if 2i > n then exit {node i is a leaf}
3. repeat
4. i = 2i
5. if i + 1 ≤ n and key(H[i + 1]) > key(H[i]) then i =i + 1
if key(H[ [i/2]]) < key(H[i]) then
interchange H[i] and H[ [i/2] ]
7. else done = true
8. end if
9. until 2i > n or done
Insertion into a Heap
• Increase the size of H by 1.
• append x to the end of H
• shift x up, if necessary (to satisfy heap property).

Algorithm INSERT
Input: A heap H[1..n] and a heap element x.
Output: A new heap H[1..n + 1] with x being one of its elements.
1. n =n + 1 {increase the size of H}
2. H[n] = x
3. SHIFT-UP(H , n)

Time Complexity is O(logn)

Deleting an element from a heap
• To delete an element H[i] from a heap H of size n,
– replace H[i] by H[n],
– decrease the heap size by 1,
– shift H[i] up or down, if necessary.

Algorithm DELETE
Input: A nonempty heap H[1..n] and an index i between 1 and n.
Output: A new heap H[1..n - 1] after H[i] is removed.
1. x  H[i]; y  H[n]
2. n  n - 1 {decrease the size of H}
3. if i = n + 1 then exit {done}
4. H[i]  y
5. if key(y) ≥ key(x) then SHIFT-UP(H , i)
6. else SHIFT-DOWN(H , i)
7. end if
Time Complexity is O(logn)
Building a Heap
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are leaves
• Apply shift-down on elements between 1 and n/2

4
BUILD-MAX-HEAP(A) 2 3

1 3
1. n = length[A] 4 5 6 7

2 16 9 10
2. for i ← n/2 downto 1
8 9 10

14 8 7
3. do shift-down(A, i, n)
:A 4 1 3 2 16 9 10 14 8 7

34
 Example: A 4 1 3 2 16 9 10 14 8 7

i=5 i=4 i=3

1 1 1

4 4 4
2 3 2 3 2 3

1 3 1 3 1 3
4 5 6 7 4 5 6 7 4 5 6 7

8
2 9 10
16 9 10 8 2 9 10
16 9 10 8 14 9 10
16 9 10
14 8 7 14 8 7 2 8 7

i=2 i=1
1 1 1

4 4 16
2 3 2 3 2 3

1 10 16 10 14 10
4 5 6 7 4 5 6 7 4 5 6 7

8
14 9 10
16 9 3 8
14 9 10
7 9 3 8
8 9 10
7 9 3
2 8 7 2 8 1 2 4 1
35
 Running Time of BUILD MAX HEAP

BUILD-MAX-HEAP(A)
1. n = length[A]
2. for i ← n/2 downto 1
O(n)
3. do shift-down(A, i, n) O(lgn)

 Running time: O(nlogn)

• This is not an asymptotically tight upper bound.
Running time: O(n) ?????
• Build heap is a linear time algorithm???.
36
 Running Time of BUILD MAX HEAP
Height Level No. of nodes
h0 = 3 (lgn) i=0 20

h1 = 2 i=1 21

h2 = 1 i=2 22

h3 = 0 i = 3 (lgn) 23

• hi = h – i height of the heap rooted at level i

• ni = 2 i number of nodes at level i
• Shift-down takes O(h)  the cost of shift-down on a node i is
proportional to the height of the node i in the tree
h h
 T (n)   ni hi   2i h  i   O(n)
i 0 i 0
 Heapsort
• Goal:Sort an array in place using heap representations
• Idea:
– Build a max-heap from the array
• Maximum element is at A[1]
– Discard by swapping with element at A[n]
• Decrement heap_size[A]
• A[n] now contains correct value
– Restore heap property at A[1] by calling shiftdown()
– Repeat, always swapping A[1] for A[heap_size(A)]

38
Example: A=[7, 4, 3, 1, 2]

Shift-down(A, 1, 4) Shift-down(A, 1, 3) Shift-down(A, 1, 2)

Shift-down(A, 1, 1)

39
Algorithm HEAPSORT

1. BUILD-MAX-HEAP(A) ……….……. O(n)

2. for j  n downto 2
3. interchange A[1] and A[j] ….. O(1)
4. SHIFT-DOWN(A[1..j - 1], 1) ….. O(log n) n-1 times
5. end for

 The call to BuildHeap() takes O(n) time

 Each of the n - 1 calls to shift-down) takes O(lg n) time
 Thus the total time taken by HeapSort()
= O(n) + (n - 1) O(lg n)
= O(n) + O(n lg n)
= O(n lg n)