Slides

DESIGN & ANALYSIS OF
ALGORITHMS
Lecture # 1 & 2
INTRODUCTION
Objectives
■ Design and analysis of modern algorithms
■ Different variants
■ Accuracy
■ Efficiency
■ Comparing efficiencies
■ Motivation thinking new algorithms
■ Advanced designing techniques
■ Real world problems will be taken as examples
■ To create feelings about usefulness of this course
2
Expected Results
■ On successful completion, students will be able to
– Argue and prove correctness of algorithms
– Derive and solve mathematical models of problems
– Reasoning when an algorithm calls certain approach
– Analyze average and worst-case running times
– Integrating approaches in dynamic and greedy algos.
– Use of graph theory in problems solving
– Advanced topics such as
– Computational geometry, number theory etc.
– Several other algorithms such as
– String matching, NP completeness, approximate algorithms etc.
Recommended Material
■ Lecture Slides
■ Introduction to The Design & Analysis of

Algorithms by Anany Levitin
■ Computers and Intractability, Guide to the

Theory of NP-Completeness by Garey and
Johnson
4
Tentative Grading Scheme
■ 10 Class Tasks (10%)
■ 5 Assignments (10%)
■ 3-5 Quizzes (10%)
■ 1 Mid Term (25%)
■ 1 Presentation (5%)
■ 1 Final Term (40%)
5
Course Access
■ LMS
■ YouTube Channel
■ WhatsApp Group
■ Email
sulaman87@gmail.com
Course Contents
What is Algorithm?
■ A computer algorithm is a detailed step-by-step method for solving

a problem by using a computer.
■ An algorithm is a sequence of unambiguous instructions for
solving a problem in a finite amount of time.
■ An Algorithm is well defined computational procedure that takes
some value, or set of values, as input and produces some value,
or set of values as output.
Algorithm
■ An algorithm is a sequence of instructions that one must perform
in order to solve a well-formulated problem.
■ A well-formulated problem is unambiguous and precise, leaving

no room for misinterpretation. The number of steps are always
finite.
■ In other words, algorithm is the method of translating the inputs

into the outputs.
6 Characteristics of an Algorithm
Finiteness
■ Algorithms are finite in two aspects:
– Length of Algorithm
– Number of Computational Steps
■ If algorithm is not of finite length, then we do not have infinite

space/memory to save and implement it.
■ If the number of computational steps are not finite, then it means
the problem is not solved at all, as infinity is undefined.
Popular Algorithms
■ Most basic and popular algorithms are
– Sorting algorithms
– Searching algorithms
■ Which algorithm is best?
– Mainly, it depends upon various factors, for example in case
of sorting:
■ The number of items to be sorted.
■ The extent to which the items are already sorted.
■ Possible restrictions on the item values.
■ The kind of storage device to be used etc.
Problems vs Algorithms vs Programs
■ For each problem or class of problems, there may be many
different algorithms.
■ For each algorithm, there may be many different implementations
(programs).
Testing Correctness
■ How do we know whether an algorithm is
actually correct?
■ First, the logical analysis of the problem we
performed in order to design the algorithm
should give us confidence that we have
identified a valid procedure for finding a
solution.
■ Second, we can test the algorithm by choosing
different sets of input values, carrying out the
algorithm, and checking to see if the resulting
solution does, in fact, work.
Testing Correctness
■ BUT… no matter how much testing

we do, unless there are only a finite
number of possible input values for
the algorithm to consider, testing can
never prove that the algorithm
produces correct results in all cases.
Testing Correctness
■ We can attempt to construct a formal,
mathematical proof that, if the algorithm is
given valid input values then the results
obtained from the algorithm must be a solution
to the problem.
■ We should expect that such a proof be
provided for every algorithm.
■ In the absence of such a proof, we should view
the purported algorithm as nothing more than
a heuristic procedure, which may or may not
yield the expected results.
Algorithm Design Techniques
■ There are many types of algorithm designing techniques, but the most
commonly studied are:
1. Recursive Algorithms 7. Dynamic Programming
2. Iterative Algorithms 8. Greedy Algorithms
3. Divide & Conquer Algorithms 9. Brute Force Algorithms

4. Decrease & Conquer 10. Backtracking Algorithms
5. Transform & Conquer 11. Randomized Algorithms
6. Iterative Improvement
12. Time & Space Trade-off
Problem Solving Phases
■ Analysis
– How does system work?
– Breaking a system down to known components
– How components (processes) relate to each other
– Breaking a process down to known functions
■ Synthesis
– Building tools
– Building functions with supporting tools
– Composing functions to form a process
– How components should be put together?
– Final solution
Problem Solving Process
Verification
Implementation
Analysis
•Correctness
Algorithm •Time & Space
•Input •Optimality
Strategy •Output
•Steps
•Design
Problem
Time Complexity of Algorithms
■ When we talk about time complexity of algorithms, what do we
actually mean?
■ It is not the actual running time. It is not the standard.

– It varies from machine to machine & compiler to compiler.
– It even varies on one machine due to availability of resources.
Model of Computation (Assumptions)
■ Design assumption
– Level of abstraction which meets our requirements
– Neither more nor less e.g. [0, 1] infinite continuous interval
■ Analysis independent of the variations in
– Machine
– Operating system
– Programming languages
– Compiler etc.
■ Low-level details will not be considered

■ Our model will be an abstraction of a standard generic
single-processor machine, called a random access
machine or RAM.
■ A RAM is assumed to be an idealized machine
– Infinitely large random-access memory
– Instructions execute sequentially
■ Every instruction is in fact a basic operation on two values in the
machines memory which takes unit time.
■ These might be characters or integers.
■ Example of basic operations include
– Assigning a value to a variable
– Arithmetic operation (+, - , × , /) on integers
– Performing any comparison e.g. a < b
– Boolean operations
– Accessing an element of an array.
■ In theoretical analysis, computational complexity
– Estimated in asymptotic sense, i.e.
– Estimating for large inputs
■ Big O, Big Omega, Theta etc. notations are used to compute the
complexity
■ Asymptotic notations are used because different implementations
of algorithm may differ in efficiency
■ Efficiencies of two given algorithm are related
– By a constant multiplicative factor
– Called hidden constant.
Drawbacks in Model of Computation
■ First poor assumption
– We assumed that each basic operation takes constant time, i.e. model allows
■ Adding
■ Multiplying
■ Comparing etc.
two numbers of any length in constant time
■ Addition of two numbers takes a unit time!
– Not good because numbers may be arbitrarily
■ Addition and multiplication both take unit time!
– Again very bad assumption
Justification for the Model
■ But with all these weaknesses, our model is not so bad

because we have to give the
– Comparison not the absolute analysis of any algorithm.

– We have to deal with large inputs not with the small size
– Model seems to work well describing computational power of
modern nonparallel machines
Can we do Exact Measure of Efficiency?
■ Exact, not asymptotic, measure of efficiency can be sometimes

computed but it usually requires certain assumptions concerning
implementation.
Summary : Computational Model
■ Analysis will be performed with respect to this computational
model for comparison of algorithms
■ We will give asymptotic analysis not detailed comparison i.e. for
large inputs
■ We will use generic uniprocessor random-access machine (RAM)
in analysis
– All memory equally expensive to access
– No concurrent operations
– All reasonable instructions take unit time, except, of course, function
calls
Conclusion
■ What, Why and Where Algorithms?
■ Designing Techniques
■ Problem solving Phases and Procedure
■ Model of computations
– Major assumptions at design and analysis level
– Merits and demerits, justification of assumptions taken
■ We proved that algorithm is a technology
■ Compared algorithmic technology with others
■ Discussed importance of algorithms
– In almost all areas of computer science and engineering
– Algorithms make difference in users and developers
ALGORITHMS
Lecture # 3
MatheMatical algorithMic
toolbox
Review of Lecture No 1 & 2
■ Introduction to Algorithms
■ Designing Techniques
■ Algorithm is a Technology
■ Model of Computation
■ Even we have supercomputers, it requires algorithm
■ Algorithms makes difference in users and modeler
■ Some of the applications areas of algorithms
 Management and manipulation of data
 Electronic commerce
 Manufacturing and other commercial settings
 Shortest paths etc.
Topics Covered Today
■ Sets
■ Sequences
■ Order pairs
■ Cross Product
■ Relation
■ Functions
■ Operators over above structures
■ Conclusion
Set
■ A set is well defined collection of objects, which are
– unordered
– distinct
– same type
– with common properties
■ Notation:
– Elements of set are listed between a pair of curly braces
S1 = {R, R, R, B, G} = {R, B, G} = {B, G, R}
■ Empty Set
– S3 = { } = , has not elements, called empty set
Representation of Sets
■ Three ways to represent a set
• Descriptive Form
• Tabular form
• Set Builder Form (Set Comprehension)
Example
■ Descriptive Form
S = set of all prime numbers
■ Tabular form
S = {2, 3, 5, . . .}
■ Set Builder Form
S = {x : xN  i{2, 3, . . ., x-1}  i (x mod i ≠ 0)}
Representation of Sets
Some More Examples
1. {x : x  Z  x2 = x} = {0, 1}
2. {x : x  N  x mod 2 = 0} = {0, 2, 4, . . . }
3. {x : x  N  x mod 2 ≠ 0} = {1, 3, 5, . . . }
4. {x : x  Z  x ≥ 0  x ≤ 6} = {0, 1, 2, 3, 4, 5, 6}
5. {x2 : x  Z  x ≥ 0  x ≤ 6} = {0, 1, 4, . . ., 25, 36}
6. {x3 : x  N  x mod 2 ≠ 0} = {1, 27, 125, . . . }
Assignment #1
 Note that people disagree on the exact definitions of whole numbers and
natural numbers.
 Research Assignment#1
 0 is included in Whole numbers or Natural numbers?
 What is the definition of Natural numbers and Whole numbers?
 What is your opinion about zero w.r.t the definitions of Natural numbers and
Whole numbers?
 Validate your opinion.
7
Assignment #1
 Your article should contain sections like:
 Abstract
 Background
 Status of zero
 Definitions of Natural & Whole Numbers
 My opinion
 Proof
 References
 Each section should contain a proper heading
 Each reference should be properly cited
 Extra marks for the best group
8
Assignment #1
 Format
 Font Size: 12
 Font Name: Times New Roman
 Heading Size: 16
 Heading Font: Arial Black
 Page Size: A4
 Margins: Normal
 Group of a maximum of 4 and minimum of 3 students can submit one article.
 Only softcopy *.docx file is acceptable.
 Deadline: by 15 November, 2020
 Groups with same or similar article will get zero marks.
9
All collections are not sets
■ The prime numbers
Primes = {2, 3, 5, 7, . . . }
■ The four oceans of the world
Oceans = {Atlantic, Arctic, Indian, Pacific}
■ The passwords that may be generated using eight lower-case letters,
when repetition is allowed
■ Hard working students in BS-CS class session 2020-21 at DSU
■ Intelligent students in this class
■ Kind teachers at DSU
Operators Over Sets
Membership Operator
■ If an element e is a member of set S then it is denoted as
e  S and is read as “e is in S”.
■ Let S is sub-collection of X (any Universal set)
X = a set
SX
Now
 : X × PX Bool
 (x, S) = 1, if x is in S
= 0, if x is not in S
Subset
If each element of A is also in B, then A is said to be a subset of B,
A  B and B is superset of A, B  A.
Let X = a universal set, A  X, B  X

Now,
 : X x X  Bool
 (A, B) = 1, if x : X, x  A  x  B
= 0, if x : X, x  A  x  B
More Operators Over Sets
Intersection
 : X x X  X
 (A, B) = {x  X | x  A and x  B}
Union
 : X x X  X
 (A, B) = {x  X | x  A or x  B}
Set Difference
\ : X x X  X
\ (A, B) = {x  X | x  A but x  B}
Cardinality and Power Set of a given Set
Cardinality of a Set
■ A set, S, is finite if there is an integer n such that the elements of S can
be placed in a one-to-one correspondence with {1, 2, 3, …, n}, and we
say that cardinality is n. We write as: |S| = n
Power Set
■ How many distinct subsets does a finite set on n elements have? There
are 2n subsets.
■ How many distinct subsets of cardinality k does a finite set of n
elements have?
There are n n!
C (n, k ) Ck    
n
 k  (n  k )!k!
Partitioning of a set
A partition of a set A is a set of non-empty subsets of A such that
every element x in A exactly belong to one of these subsets.
Equivalently, set P of subsets of A, is a partition of A
1. If no element of P is empty
2. The union of the elements of P is equal to A. We say the elements
of P cover A.
3. The intersection of any two elements of P is empty. That means
the elements of P are pair wise disjoints.
Mathematical Way of Defining Partitioning
A partition P of a set A is a collection {A1, A2, . . ., An} such that
following are satisfied
1.  Ai  P, Ai  ,
2. Ai  Aj = ,  i, j  {1, 2, . . ., n} and i  j
3. A = A1  A2  . . . An
Example: Partitioning of Z Modulo 4
{x  Z | x mod 4 = 0 } = {. . .-8, -4, 0, 4, 8, . . . }
{x  Z | x mod 4 = 1 } = {. . .-7, -3, 1, 5, 9, . . . }
{x  Z | x mod 4 = 2 } = {. . .-6, -2, 2, 6, 10, . . . }
{x  Z | x mod 4 = 3 } = {. . .-5, -1, 3, 7, 11, . . . }
Sequence
■ It is sometimes necessary to record the order in which objects are
arranged, e.g.,
– Data may be indexed by an ordered collection of keys
– Messages may be stored in order of arrival
– Tasks may be performed in order of importance.
– Names can be sorted in order of alphabets etc.
■ Definition
– A group of elements in a specified order is called a sequence.
– A sequence can have repeated elements.
Sequence
■ Notation: Sequence is defined by listing elements in order, enclosed in
parentheses. e.g.
S = (a, b, c), T = (b, c, a), U = (a, a, b, c)
■ Sequence in a set
S = {(1, a), (2, b), (3, c)} = {(3, c), (2, b), (1, a)}
■ Permutation: If all elements of a finite sequence are distinct, that

sequence is said to be a permutation of the finite set consisting of the
same elements.
■ No. of Permutations: If a set has n elements, then there are n! distinct
permutations over it.
Operators over Sequences
■ Operators are for manipulations
■ Concatenation
– (a, b, c ) + ( d, a ) = ( a, b, c, d, a )
■ Other operators
– Extraction of information: ( a, b, c, d, a )(2) = b
– Filter Operator: {a, d} ( a, b, c, d, a ) = (a, d, a)
■ Note:
– We can think how resulting theory of sequences falls within our existing
theory of sets
– And how operators in set theory can be used in case of sequences
Tuples and Cross Product
■ A tuple is a finite sequence.
• Ordered pair (x, y), triple (x, y, z), quintuple
• A k-tuple is a tuple of k elements.
Construction to ordered pairs
■ The cross product of two sets, say A and B, is
A  B = {(x, y) | x  A, y  B}
| A  B | = |A| |B|
■ Some times, A and B are of same set, e.g.,

Z  Z, where Z denotes set of Integers
Binary Relations
Definition: If X and Y are two non-empty sets, then
X x Y = {(x, y) | x X and y  Y}
Example: If X = {a, b}, Y = {0,1} Then
X x Y = {(a, 0), (a, 1), (b, 0), (b, 1)}
Definition: A subset of X x Y is a relation over X x Y

Example: Compute all relations over X x Y, where
X = {a, b}, Y = {0,1}
R1 = , R2 = {(a, 0)}, R3 = {(a, 1)}
R4 = {(b, 0)}, R5 = {(b, 1)}, R6 = {(a, 0), (b, 0)}, . . .
There will be 24 = 16 number of relations
Binary Relations
Definition: XY = P(X x Y)
Example: If X = {a, b}, Y = {0,1} Then
X x Y = {(a, 0), (a, 1), (b, 0), (b, 1)} and
P(X x Y) = {R1, R2, R3, R4, R5, R6, R7, R8, R9, R10,R11, R12, R13, R14, R15, R16}
Lemma 1: if |X| = m, |Y| = n then

|P(X x Y)| = 2m + n
Algorithm: All binary relations, cardinality of it

Equivalence Relation
A relation R  X  X, is
■ Reflexive:
(x, x)  R,  x  X
■ Symmetric:
(x, y)  R  (y, x)  R,  x, y  X
■ Transitive:
(x, y)  R  (y, z)  R  (x, z)  R
■ Equivalence:
If reflexive, symmetric and transitive
Applications : Order Pairs in Terms of String
Example 1
If  = {0, 1}, then construct set of all strings of length 2 and 3
■ Set of length 2 =  x  = {0,1} x {0,1} = {(0,0), (0,1), (1,0), (1,1)}
= {00, 01, 10, 11}
■ Set of length 3 =  x  x  = {0, 1} x {0, 1} x {0,1}
= {(0,0,0), (0,0,1), (0,1,0), (0,1,1), (1,0,0), (1,0,1), (1,1,0), (1,1,1)}
= {000, 010, 100, 110, 001, 011, 101, 111}
Example 2
If  = {0, 1}, then construct set of all strings of length ≤ 3
■ Construction = {} U  U  x  U  x  x 
Similarly we can construct collection of all sets of length n.
Partial Function
A partial function is a relation that maps each element of X to at
most one element of Y.
pf: XY denotes the set of all partial functions.
Function
If each element of X is related to unique element of Y then partial
function is a total function denoted by f: X  Y.
Is Algorithm a Function?
Not good algorithms

Conclusion
■ We started with sets and built other structures e.g.
– Sequences
– Relations
– Functions, etc.
■ We discussed operators over above structures
■ All these tools are fundaments of mathematics
■ Sets play key role in algorithms design and analysis
■ Finally proving correctness of algorithms, we required logic.
■ Our next lecture will be on proving techniques
Class Task 1 (Graded)
■ Give mathematic definitions (set-builder mathematical
pseudocode) for:
– Partial Function
– Total Function
Note: Submit your answers on LMS by 12 Nov, 2020.

Late submissions will not be entertained.
ALGORITHMS
Lecture # 4, 5 & 6
ProPositional &
Predicate loGic
Logic
■ Logic is the study of the principles and methods that distinguish
between a valid and an invalid argument.
2
Statement / Proposition
■ A statement is a declarative sentence that is either true or false
but not both.
■ A statement is also referred to as a proposition.
■ Proposition: Simplest statements also called atomic formula.
■ EXAMPLES:
a. 2+2 = 4,
b. It is Sunday today.
3
Truth Values
■ If a proposition is true, we say that it has a truth value of "true”.
If a proposition is false, its truth value is "false".
The truth values “true” and “false” are, respectively, denoted by the
letters T and F.
■ EXAMPLES:
Propositions
1) Grass is green.
2) 4 + 2 = 6
3) 4 + 2 = 7
4) There are four fingers in a hand.
4
Invalid Propositions
■ Not Propositions
1) Close the door.
2) x is greater than 2.
3) He is very rich.
■ Either they don’t have any truth value or they are not declarative.
5
Remarks
■ If the sentence is preceded by other sentences that make the pronoun
or variable reference clear, then the sentence is a statement.
■ Example:
x=1
x>2
“x > 2” is a statement with truth-value FALSE.
■
Example
Bill Gates is an American.
He is very rich.
“He is very rich.” is a statement with truth-value TRUE.
6
Examples
■ x + 2 is positive. Not a statement
■ May I come in? Not a statement
■ Logic is interesting. A statement
■ It is hot today. A statement
■ -1 > 0 A statement
■ x + y = 12 Not a statement
7
Compound Statements
■ Simple statements could be used to build a compound statement.
■ LOGICAL CONNECTIVES
EXAMPLES:
1. “3 + 2 = 5” and “Lahore is a city in Pakistan.”
2. “The grass is green” or “ It is hot today.”
3. “Discrete Mathematics is not difficult to me.”
■ Here, AND, OR, NOT are called LOGICAL CONNECTIVES.

8
Notation of Statement
■ Statements are symbolically represented by letters such as p, q,
r,...
■ EXAMPLES:
p: “Islamabad is the capital of Pakistan”
q: “17 is divisible by 3”
9
Common Connectives
Note: Some books used “” symbol for negation.

10
Common Connectives
■ EXAMPLES
– p : “Islamabad is the capital of Pakistan”
– q : “17 is divisible by 3”
– p q : “Islamabad is the capital of Pakistan and 17 is divisible by 3”
– p q : “Islamabad is the capital of Pakistan or 17 is divisible by 3”
– ~p : “It is not the case that Islamabad is the capital of Pakistan”

or simply “Islamabad is not the capital of Pakistan”
11
TRANSLATION
TRANSLATING FROM ENGLISH TO SYMBOLS
Let p = “It is hot”, and q = “ It is sunny”
12
TRANSLATION
13
TRANSLATION
14
Truth Table
■ A truth table specifies the truth value of a compound proposition
for all possible truth values of its constituent propositions.
15
Negation
■ NEGATION (~):
If p is a statement variable, then negation of p, “not p”, is denoted
as “~p”
■ It has opposite truth value from p i.e., if p is true, then ~ p is false;

if p is false, then ~ p is true.
16
Truth Table for Negation
17
Conjunction
■ CONJUNCTION ( ):
If p and q are statements, then the conjunction of p and q is “p
and q”, denoted as
“p q”.
■ Remarks
– p q is true only when both p and q are true.
– If either p or q is false, or both are false, then p q is false.
18
Truth Table for Conjunction
19
Disjunction
■ DISJUNCTION ( ) or INCLUSIVE OR
If p & q are statements, then the disjunction of p and q is “p or q”,
denoted as
“p q”.
■ Remarks:
– p q is true when at least one of p or q is true.
– p q is false only when both p and q are false.
20
Truth Table for Disjunction
21
Truth Table for Disjunction
■ Note it that in the table F is only in that row where both p and q
have F and all other values are T.
■ Thus for finding out the truth values for the disjunction of two
statements we will only first search out where the both
statements are false and write down the F in the corresponding
row in the column of p q and in all other rows we will write T in
the column of p q.
22
Remark
■ Note that for Conjunction of two statements we find the T in both
the statements, But in disjunction we find F in both the
statements.
■ In other words, we will fill T in the first row of conjunction and F in

the last row of disjunction.
23
Example
■ ~p^(qv~r)
As there are three variables so 23=8 rows.
p q r
T T T
T T F
T F T
T F F
F T T
F T F
F F T
F F F
24
Example
■ ~p^(qv~r)
p q r ~p
T T T F
T T F F
T F T F
T F F F
F T T T
F T F T
F F T T
F F F T
25
Example
■ ~p^(qv~r)
p q r ~p ~r
T T T F F
T T F F T
T F T F F
T F F F T
F T T T F
F T F T T
F F T T F
F F F T T
26
Example
■ ~p^(qv~r)
p q r ~p ~r q v ~r
T T T F F T
T T F F T T
T F T F F F
T F F F T T
F T T T F T
F T F T T T
F F T T F F
F F F T T T
27
Example
■ ~p^(qv~r)
p q r ~p ~r q v ~r ~p ^ (q v ~r)
T T T F F T F
T T F F T T F
T F T F F F F
T F F F T T F
F T T T F T T
F T F T T T T
F F T T F F F
F F F T T T T
28
Exclusive OR
■ When OR is used in its exclusive sense, The statement “p or q”
means “p or q but not both” or “p or q and not p and q” which
translates into symbols as (p q) ~ (p q).
■ It is denoted as p q or p XOR q
■ TASK: Prove That

p q ≡ (p q) ~ (p q) ≡ (p q) ( p ~ q)
by using Truth Tables.
NOTE: “≡” symbol is used for equivalence.
29
Truth Table of XOR
30
LOGICAL EQUIVALENCE
■ If two logical expressions have the same logical values in the truth
table, then we say that the two logical expressions are logically or
scemantically equivalent. In the following example, ~ (~ p ) is
logically equivalent p. So it is written as ~(~p) ≡ p
31
QUESTION
■ Are
~(p q) and ~p ~q
logically equivalent ?
32
De-Morgan’s Laws
33
TAUTOLOGY
■ A tautology is a statement form that is always true regardless of the truth
values of the statement variables. A tautology is represented by the symbol
“t”.
EXAMPLE: The statement form p ~ p is tautology.
p ~p≡t
34
CONTRADICTION
■ A contradiction is a statement form that is always false regardless of the truth values of
the statement variables. A contradiction is represented by the symbol “c”.
EXAMPLE:
The statement form p ~ p is a contradiction.
■ Since in the last column in the truth table we have F in all the entries, so it is a
contradiction i.e. p ~ p ≡c
35
■ Give one statement that is a Tautology and one that is a
Contradiction with three variables x,y and z?
■ Also write the description of variables in simple English.
Note:Submit your answers on LMS by 16 Nov, 2020.

36
Laws of Logic
■ 1) Commutative Laws
p q≡q p
p q≡q p
■ 2) Associative Laws
(p q) r≡p (q r)
(p q) r≡p (q r)
■ 3) Distributive Laws
p (q r)≡(p q) (p r)
p (q r)≡(p q) (p r)
■ 4) Identity Laws
p t≡p
p c≡p
37
Laws of Logic
■ 5) Negation Laws
p p≡t
p p≡c
■ 6) Double Negation Law

( p) ≡ p
■ 7) Idempotent (something that remains same) Laws

p p≡p
p p≡p
■ 8) De-Morgan’s Laws
~ ( p q ) ≡ ~p q
~ ( p q ) ≡ ~p q
38
Laws of Logic
■ 9) Universal Bound Laws
p t≡t
p c≡c
■ 10) Absorption Laws

p (p q)≡p
p (p q)≡p
■ 11) Negation of t and c

~t≡c
~c≡t
39
Conditional Statement
■ If p and q are statement variables, the conditional of q by p is “If p then
q”
or “p implies q” and is denoted p → q.
■ p → q is false when p is true and q is false; otherwise it is true.
■ The arrow "→ " is the conditional operator.
■ In p → q, the statement p is called the hypothesis (or antecedent) and

q is called the conclusion (or consequent).
40
Truth Table
41
BICONDITIONAL STATEMENTS
■ If p and q are statement variables, the biconditional of p and q is
“p if and only if q”.
■ It is denoted p↔q. “if and only if” is abbreviated as iff.
■ The double headed arrow " ↔" is the biconditional operator.
42
Truth Table
■ p ↔ q is true only when p and q both are true or
both are false.
■ p ↔ q is false when either p or q is false.
■ Just the negation of XOR.
43
Translation Summary
of p → q
44
Translation of p → q
p: You will attend the classes.
q: You will be allowed to participate in the exams.
■ If p, then q.
■ If you will attend the classes, then you will be allowed

to participate in the exams.
45
■ If p, q.
■ If you will attend the classes, you will be allowed to

participate in the exams.
46
■ q, if p.
■ You will be allowed to participate in the exams if you

will attend the classes.
47
■ q when p.
■ You will be allowed to participate in the exams when

you will attend the classes.
48
■ q whenever p.
■ You will be allowed to participate in the exams

whenever you will attend the classes.
49
■ q follows from p.
■ You will be allowed to participate in the exams follows from

your attendance in the classes.
50
■ p implies q.
■ You will attend the classes implies that you will be

allowed to participate in the exams.
51
■ p is sufficient for q.
■ You will attend the classes is sufficient to allow you to

52
■ A sufficient condition for q is p.
■ A sufficient condition to allow you to participate in the

exams is that you will attend the classes.
53
■ q is necessary for p.
■ To allow you to participate in the exams is necessary if

you will attend the classes.
54
■ A necessary condition for p is q.
■ A necessary condition to attend the classes is to allow

you to participate in the exams.
55
■ p only if q.
■ You will attend the classes only if you will be allowed to

56
Remarks
■ In mathematical reasoning, we consider conditional statements of
a more general sort than we use in English.
■ Propositional language is an artificial language; we only parallel

English usage to make it easy to use and remember.
■ The if-then construction used in many programming languages is

different from that used in logic.
57
BICONDITIONAL STATEMENTS
■ If p and q are statement variables, the biconditional of p and q is
“p if and only if q”.
■ It is denoted p↔q. “if and only if” is abbreviated as iff.
■ The double headed arrow " ↔" is the biconditional operator.
58
Truth Table
■ p ↔ q is true only when p and q both are true or
both are false.
■ p ↔ q is false when either p or q is false.
■ Just the negation of XOR.
■ Translation:
–
–
–
59
BICONDITIONAL STATEMENT
■ It is the conjunction of
p → q and q → p
i.e. (p → q) ^ (q → p)
p q p→q q→p p→q ^ (q→p)
T T T T T
T F F T F
F T T F F
F F T T T
60
Precedence of Logical Operators
Precedence Operator
1 ~
2 ^
3 v
4 →
5 ↔
61
Laws of Logic
■ 1) Commutative Laws
p q≡q p
p q≡q p
■ 2) Associative Laws
(p q) r≡p (q r)
(p q) r≡p (q r)
■ 3) Distributive Laws
p (q r)≡(p q) (p r)
p (q r)≡(p q) (p r)
■ 4) Identity Laws
p t≡p
p c≡p
62
Laws of Logic
■ 5) Negation Laws
p p≡t
p p≡c
■ 6) Double Negation Law

( p) ≡ p
■ 7) Idempotent (something that remains same) Laws

p p≡p
p p≡p
■ 8) De-Morgan’s Laws
~ ( p q ) ≡ ~p q
~ ( p q ) ≡ ~p q
63
Laws of Logic
■ 9) Universal Bound Laws
p t≡t
p c≡c
■ 10) Absorption Laws

p (p q)≡p
p (p q)≡p
■ 11) Negation of t and c

~t≡c
~c≡t
64
Applications
■ Using Laws of Logic, simplify the following compound statement.
p v [~(~p ^ q)]
■ Solution:
p v [~(~p ^ q)]
≡ p v [~(~p) v (~q)] De-Morgan’s Law
≡ p v [p v ~q] Double Negation Law
≡ [p v p] v ~q Associative Law
≡ p v ~q Idempotent Law
65
Applications
■ Using Laws of Logic, verify the logical equivalence
~ (~ p q) (p q) ≡ p
■ Solution:
L.H.S
= ~(~p q) (p q)
≡ (~(~p) ~q) (p q) De-Morgan’s Law

≡ (p ~q) (p q) Double Negative Law
≡p (~q q) Distributive Law
≡p c Negation Law
≡p Identity Law
=R.H.S 66
Applications
■ Rephrase the condition more simply.
“You will get an A if you are hardworking and the sun shines, or you are hardworking and it
rains.”
■ Solution Steps:
1. Identify the Simple Propositions
2. Translate the Condition in symbols
3. Simplification by apply the Laws of Logic
4. Re-Translation
67
Applications
■ Identifying the Simple Propositions.
“You will get an A if you are hardworking and the sun shines, or you are hardworking and it rains.”
■ Translation
 a if h and s or h and r
 a if h ^ s v h ^ r
 a if (h ^ s) v (h ^ r)
 [(h ^ s) v (h ^ r)] → a
68
Applications
■ Simplification
[(h ^ s) v (h ^ r)] → a
≡ [h ^ (s v r)] → a Distributive Law
■ Re-Translation
If you are hardworking and either the sun shines or it rains, you will get an A.
OR
You will get an A if you are hardworking and either the sun shines or it rains.
69
Practice
■ Rewrite each of the following sentences more simply:
1. It is not true that I am tired and you are smart.

 {I am not tired or you are not smart.}
2. It is not true that I am tired or you are smart.

 {I am not tired and you are not smart.}
70
Practice
■ Rewrite each of the following sentences more simply:
1. I forgot my pen and my bag or I forgot my pen and my glasses.

■ {I forgot my pen and EITHER I forgot my bag or my glasses.
2. It is raining and I have forgotten my umbrella, or it is raining and

I have forgotten my hat.
■ {It is raining and I have forgotten EITHER my umbrella or my hat.}
71
More Laws of Logic
1. Commutative Law: p ↔ q ≡ q ↔ p
2. Implication Law: p → q ≡ ~ p q≡~(p ~ q)
3. Exportation Law: (p q)→r ≡ p →(q →r)
4. Equivalence: p ↔ q ≡ (p →q) (q →p)
5. Reductio ad absurdum: p →q ≡ (p ~q) →c

(Reduction to Contradiction)
72
Application
■ Rewrite the statement p ~ q→ r without using the symbol →.
■ Solution:
p ~q → r
≡ (p ~q)→ r
≡ ~ (p ~ q) r Implication Law
73
Application
■ Rewrite the statement ( p→ r ) ↔ ( q → r ) without using the
symbol → or ↔.
■ Solution:
(p→r)↔(q →r)
≡ (~p r) ↔ (~q r)
Implication Law
≡ [(~p r) →(~q r)] [(~q r) →(~p r)]

Equivalence Law
≡ [~(~p r) (~q r)] [~(~q r) (~p r)]

Implication Law
74
Simplifying It Further
≡ [~(~p r) (~q r)] [~(~q r) (~p r)]
≡ [~(~p) ~r (~q r)] [~(~q) ~r (~p r)]

De-Morgan’s Law
≡ [p ~r ~q r] [q ~r ~p r]
Double Negation Law
75
■ Rewrite the statement form ~p q → r ~q to a logically
equivalent form that uses only ~ and

76
■ Show that ~(p→q) → p is a tautology without using truth tables.

77
Inverse, Converse & Contrapositive
■ Consider a conditional statement:
p→q
■ Its INVERSE will be:

~p → ~q
■ CONVERSE of the original statement will be:

q→p
■ While its CONTRAPOSITIVE can be written as:

~q → ~p
78
Remarks - INVERSE
79
Remarks - CONVERSE
80
Task - CONTRAPOSITIVE
■ Prove that conditional statement and its contra-
positive are equivalent.
[Hint: You can use TRUTH TABLES]
81
Example
■ Conditional statement: p→q
If I am elected, I will reduce the taxes.
■ INVERSE: ~p → ~q
If I am NOT elected, I will NOT reduce the taxes.
■ CONVERSE: q→p
If I reduce the taxes, I will be elected.
■ CONTRAPOSITIVE: ~q → ~p
If I DO NOT reduce the taxes, I will NOT be elected.
82
Practice Question
■ Which of the following statement are propositions?
1. Today is Monday.
2. It is raining.
3. x is greater than 2.
4. x is less than y.
■ 1 and 2 are propositions while 3 and 4 are not.

■ Limitation of Propositional Logic
83
Predicate Logic
■ A more powerful type of Logic.
 The statement “x is greater than 2” has two parts.
– Subject: Variable, x
– Predicate: The property that the subject of the

statement can have.
■ Here, “x” is the variable; “is greater than 2” is the predicate.

84
Notation
■ We denote propositions and predicates in a different way.
■ For example:
1. m: Today is Monday.
2. r: It is raining.
3. G(x): x is greater than 2.
4. L(x,y): x is less than y.
85
Propositional Function
■ When variable is assigned a value, it becomes a proposition. Hence,
G(x) and L(x,y) are called the PROPOSITIONAL FUNCTION.
■ Let P (x) : x > 3. What are the truth values of P (4) and P (2)?
■ Solution:
We obtain the statement P (x) by substituting the value of x.
P (4), which is the statement “4 > 3,” is TRUE.

P (2), which is the statement “2 > 3,” is FALSE.
86
Example
■ Let A(x) denote the statement “Computer x is under attack by an
intruder.”
Suppose that of the computers on campus, only CS2 and MATH1

are currently under attack by intruders.
What are truth values of A(CS1), A(CS2), and A(MATH1)?
87
Predicate with 2 Variables
■ Let Q(x, y) denote the statement “x = y + 3”
What are the truth values of the propositions

Q(1, 2) and Q(3, 0)?
88
Task
■ Let A(c, n) denote the statement “Computer c is connected to
network n,” where c is a variable representing a computer and n
is a variable representing a network.
Suppose that the computer MATH1 is connected to network

CAMPUS2, but not to network CAMPUS1.
What are the values of A(MATH1, CAMPUS1) and A(MATH1,

CAMPUS2)?
89
Predicate wit 3 Variables
■ Let R(x, y, z) denote the statement
‘x + y = z”
What are the truth values of the propositions R(1, 2, 3) and R(0,
0, 1)
90
Remark
■ Consider two examples:
1. P(x): x<x+1; x N
2. Q(x): x<3; x N
■ The first Predicate is TRUE for all values of x; while the second
Predicate is TRUE only if x=1 or x=2.
91
Quantifiers
■ Quantification expresses the extent to which a predicate is TRUE
over a range of elements.
1. P(x): x<x+1; x N
2. Q(x): x<3; x N
■ The first Predicate is TRUE for all values of x; while the second
Predicate is true only if x=1 or x=2 i.e. for some values of x.
92
Notation
1. P(x): x<x+1; x N
2. Q(x): x<3; x N
■ We can write xP(x).

Here x means for all values of x and is called the UNIVERSAL
QUANTIFIER.
■ The second one can be expressed as xQ(x).

Here x means for some values of x and is called the EXISTENTIAL
QUANTIFIER.
93
Example
■ Let P (x) be the statement “x + 1 > x”. What is the truth value of the
quantification xP (x),
where the domain consists of all real numbers?
■ Solution: Because P (x) is true for all real numbers x, the quantification
xP(x) is TRUE.
94
COUNTEREXAMPLE
■ Let Q(x) be the statement “x < 2”. What is the truth value of the
quantification xQ(x), where the domain consists of all real numbers?
■ Solution: Q(x) is not true for every real number x, because, for instance, Q(3)
is FALSE. That is, x = 3 is a COUNTEREXAMPLE for the statement xQ(x).
Thus, xQ(x) is FALSE.
95
Example
■ Let P (x) denote the statement “x > 3”. What is the truth value of the
quantification xP (x),
where the domain consists of all real numbers?
■ Solution: Because “x > 3” is sometimes true—for instance, when x = 4—the

existential quantification of P (x), which is xP (x), is TRUE.
96
Remark
■ If x1, x2, x3, …, xn is the domain for P(x), then
xP(x) means P(x1) ^ P(x2) ^ P(x3) ^ … ^ P(xn)
And
xP(x) means P(x1) v P(x2) v P(x3) v … v P(xn)
97
Task
■ Let Q(x) denote the statement “x = x + 1.”
What is the truth value of the quantification

xQ(x)
Where the domain consists of all real numbers?
98
Task
■ What is the truth value of xP(x), where P(x) is the statement “x2 >
10” and the universe of discourse consists of the positive integers
not exceeding 4?
Note: Universe of discourse is also known as the domain.
99
Precedence of Quantifiers
■ The quantifiers and have higher precedence than all logical
operators from propositional calculus.
■ For example, x P(x) Q(x) is the disjunction of xP (x) and Q(x).
■ In other words, it means ( xP (x)) Q(x) rather than x(P (x)

Q(x)).
100
Binding & Free Variables
■ Consider x(x + y = 1).
Here, the variable x is bound by the existential quantification x,
but the variable y is free because it is not bound by a quantifier.
■ This illustrates that in the statement

x(x + y = 1), x is bound, but y is free.
101
Remarks
■ We can distribute a universal quantifier over a conjunction.
x(P (x) Q(x)) ≡ xP (x) xQ(x)
■ But, we cannot distribute a universal quantifier over a disjunction.
x(P (x) v Q(x)) ≡/ xP (x) v xQ(x)
102
Remarks
■ We cannot distribute an existential quantifier over a conjunction.
x(P (x) Q(x)) ≡/ xP (x) xQ(x)
■ But, we can distribute an existential quantifier over a disjunction.
x(P (x) v Q(x)) ≡ xP (x) v xQ(x)
103
Negating Quantified Expressions
■ Consider “Every Student has taken a course in Calculus”.
xP(x)
■ NEGATION
“Every Student has NOT taken a course in Calculus”.
OR
■ “There is at least one Student who have NOT taken a course in
Calculus.
x~P(x)
104
Remarks
■ So, ¬ xP(x) ≡ x ¬P(x)
■ Similarly, ¬ xP(x) ≡ x¬P(x)
■ These are referred to as:
De Morgan’s Laws for Quantifiers
105
Task
■ What are the negations of the statements?
1. “There is an honest politician.”
2. “All Pakistani’s eat junk food”?
106
Practice Question
■ Use predicates and quantifiers to express the system
specification:
“Every mail message larger than one megabyte will be
compressed”.
107
Practice Question
■ Use predicates and quantifiers to express the system
specification:
“If a user is active, at least one network link will
be available.”
108
Nested Quantifiers
■ Assume that the domain for the variables x and y consists of all
real numbers. The statement
x y(x + y = y + x)
says that
x+y=y+x
for all real numbers x and y.
109
Example
■ Similarly,
x y(x + y = 0)
says that
For every real number x there is a real number y;

such that x + y = 0.
110
Example
■ Similarly, the statement
x y z(x + (y + z) = (x + y) + z)
Says that
The associative law for addition is applicable on all real numbers.
111
■ Translate the following compound statement into Simple English:
x y((x > 0) (y < 0) → (xy < 0))
■ where the domain for both variables consists of all real numbers.

112
ALGORITHMS
Lecture # 7, 8 & 9
ARGUMENTS & PRoof
THEoRY
Argument
■ It is a sequence of statements that end with a conclusion.
■ All statements except the conclusion are called premises.
2
Format of Argument
Premise 1
Premise 2
.
. If any
.
Premise n
--------------------------------------
Conclusion
3
Rules of Inference
4
Modus Ponens
■ It is Friday.
■ If it is Friday THEN Jumma Prayer will be offered.
■ -------------------------------------
■ Jumma Prayer will be offered.
5
Modus Tollens
Suppose it is SATURDAY
■ Jumma Prayer will NOT be offered.
■ If it is Friday THEN Jumma Prayer will be offered.
■ -------------------------------------
■ It is NOT Friday.
6
Hypothetical Syllogism
■ If you lie, you commit a sin.
■ If you commit a sin, you will be oppressed.
■ --------------------------------------
■ If you lie, you will be oppressed.
7
Disjunctive Syllogism / Tollendo Syllogism
■ Students come to the class on time OR The teacher comes to the

class on time.
■ Students DO NOT come to the class on time.
■ --------------------------------------
■ The teacher comes to the class on time.
8
Addition
■ “Multan Sultans” has a match today.
■ --------------------------------------
■ “Multan Sultans” or “Lahore Qalandars” has a match today.
9
Simplification
■ “Multan Sultans” and “Karachi Kings” have a match today.
■ --------------------------------------
10
Conjunction
■ “Karachi Kings” has a match today.
■ --------------------------------------
■ “Multan Sultans” and “Karachi Kings” have a match today.
11
Resolution
■ Today is the first match of Lahore Qalandars OR Today is the first
match of Karachi Kings.
■ Lahore Qalandars WILL NOT play their first match today OR Today
is the first match of Multan Sultans.
■ --------------------------------------
■ Today is the first match of Multan Sultans OR Today is the first
match of Karachi Kings.
12
Valid / Invalid Arguments
■ An argument is valid if the conclusion is true when all the
premises are true.
■ Alternatively, an argument is valid if conjunction of its premises

imply conclusion.
■ That is (P1 P2 P3 ... Pn) → Con is a tautology.
13
Valid / Invalid Arguments
■ An argument is invalid if the conclusion is false when all the
premises are true.
■ Alternatively, an argument is invalid if conjunction of its premises

does not imply
conclusion.
■ That is (P1 P2 P3 ... Pn) → Con is not a tautology.
14
Valid Arguments
15
Invalid Arguments
16
Invalid Arguments
17
Fallacies
■ These resemble the rules of inference, but are based on
contingencies.
‫مغالطہ‬
18
Fallacy of Affirming the Conclusion
■ If you do every problem in this book, then you will learn discrete
mathematics.
■ You have learned discrete mathematics.
■ --------------------------------------
■ You did every problem in this book.
p→q
q .
--------
p .
19
Fallacy of Denying the Hypothesis
■ If you do every problem in this book, then you will learn discrete
mathematics.
■ You DIDN’T do every problem in this book.
■ --------------------------------------
■ You have NOT learned discrete mathematics.
p→q
~p .
--------
~q .
20
21
22
Building Arguments
■ Show that the premises
– “It is not sunny this afternoon and it is colder than yesterday,”
– “We will go swimming only if it is sunny,”
– “If we do not go swimming, then we will take a canoe trip,” and
– “If we take a canoe trip, then we will be home by sunset”
■ lead to the conclusion
– “We will be home by sunset”
23
Step-1: Identify Statements
■ Premises
– “It is not sunny this afternoon and it is colder than yesterday,”
– “We will go swimming only if it is sunny,”
– “If we do not go swimming, then we will take a canoe trip,” and
– “If we take a canoe trip, then we will be home by sunset”
■ Conclusion
– “We will be home by sunset”
24
Step 2: Symbolization
■ Premises
– ~s^c ~s^c
– w→s w→s
~w→c
– ~w→c
c→h
– c→h
--------------
■ Conclusion h
– h
25
Step 3: Applying the Rules
~s^c
--------------
~s Simplification Rule
~s
w→s
-------------- ~s^c
w→s
~w Modus Tollens
~w→c
c→h
--------------
h
26
~w ~s
~w→c ~w
-------------- c
c Modus Ponens
c
c→h
~s^c
-------------- w→s
h Modus Ponens ~w→c
c→h
--------------
h
27
Example
■ Show that the premises
– “If you send me an e-mail message, then I will finish writing the
program,”
– “If you do not send me an e-mail message, then I will go to sleep early,”
and
– “If I go to sleep early, then I will wake up feeling refreshed”
■ lead to the conclusion
– “If I do not finish writing the program, then I will wake up feeling
refreshed”
28
■ Premises
– “If you send me an e-mail message, then I will finish writing the
program,”
– “If you do not send me an e-mail message, then I will go to sleep early,”
and
– “If I go to sleep early, then I will wake up feeling refreshed”
■ Conclusion
– “If I do not finish writing the program, then I will wake up feeling
refreshed”
29
■ Premises
– e→f e→f
– ~e→s ~e→s
s→r
– s→r
--------------
■ Conclusion ~f→r
– ~f→r
30
e→f
--------------
≡ ~ e v f Implication Law
~e→s
-------------- e→f
≡ ~(~e) v s Implication Law ~e→s
≡evs Double Negation Law s→r
------------
~f→r
31
~evf ~evf
evs evs
--------------
fvs Resolution
s→r
e→f
--------------
~e→s
≡ ~s v r Implication Law s→r
------------
~f→r
32
fvs ~evf
~s v r evs
fvs
--------------
~s v r
fvr Resolution
fvr e→f
-------------- ~e→s
s→r
≡ ~ f → r Implication Law
------------
(Reversed)
~f→r
33
ALTERNATE SOLUTION
■ We can also solve a problem in a number of ways.
■ Here, we present an alternate way to solve the same problem.
34
~e→s
s→r
--------------
~ e → r Hypothetical Syllogism
~e→r
--------------
e→f
≡ ~(~e) v r Implication Law ~e→s
≡evr Double Negation Law s→r
------------
~f→r
35
e→f ~e→r
-------------- evr
≡ ~ e v f Implication Law
~evf
evr e→f
-------------- ~e→s
fvr Resolution s→r
------------
~f→r
36
fvr ~evf
-------------- evs
~evf
≡ ~ f → r Implication Law
fvr
(Reversed)
e→f
~e→s
s→r
------------
~f→r
37
Rules for Quantified Statements
38
Universal Instantiation
■ If P(x) is true for all x and c is a particular member of the domain,
then P(c) is also true.
All women are wise.

-------------------
Lisa is wise.
(Because she belongs to the domain of all women)
Here the member can be either arbitrary or specific
39
Existential Instantiation
■ If P(x) is true for some x then there is at least one
member of the domain c for which P(c) is also true.
Some students have not attended all the classes.

-------------------
Mr. ABC has not attended all the classes.
(Because Mr. ABC belongs to the domain of students of the class
and he did not attend all the classes)
Here the member is specific not arbitrary
40
Universal Generalization
■ Reverse of Universal Instantiation.
■ If P(c) is true for all elements c in the domain then xP(x) is also true.
(Lisa) has two eyes.

.
.
.
-------------------
Every human has two eyes.
(Because this is true for all elements of the domain)
Here the member is arbitrary.
41
Existential Generalization
■ Reverse of Existential Instantiation.

■ When a particular element c with P(c) is known, then xP(x) is
also TRUE.
Ali has passed the Discrete Mathematics.

-------------------
At least one student has passed Discrete Mathematics.
Here the member is specific.
42
Composite Rules - 1
■ Universal Instantiation & Modus Ponens are often used together.
We combine them as UNIVERSAL MODUS PONENS
43
Composite Rules - 2
■ Similarly, Universal Instantiation & Modus Tollens are also
combined as UNIVERSAL MODUS TOLLENS
44
Example: UNIVERSAL MODUS PONENS
■ Assume that For all positive integers n,

– “If n is greater than 4, then n2 is less than 2n”
Use UNIVERSAL MODUS PONENS to show that
– 1002 < 2100
45
■ Premises
– “For all positive integers n, if n is greater than 4, then n2 is less than 2n”
■ Conclusion
– 1002 < 2100
■ We know that
– 100 is greater than 4
46
■ Premises
– n (G(n)→L(n)) n (G(n)→L(n))
– G(100) G(100)
--------------
L(100)
■ Conclusion
– L(100)
47
n (G(n)→L(n))
G(100)
--------------
L(100) Or 1002 is less than 2100
By UNIVERSAL MODUS PONENS

n (G(n)→L(n))
G(100)
------------
L(100)
48
Proof
■ A PROOF is a valid argument that establishes the truth of a
mathematical statement.
■ A PROOF can use:

– The hypothesis of a theorem
– Axioms
– Previously proved theorems
■ Final Step:
– The truth of statement is proved
49
Basic Types of Proofs
■ Formal Proofs
– All steps are provided
– One rule used at a time
– Rules for each step are stated
■ Informal Proofs
– Some steps may be skipped
– More than one rule can be used at a step
– Some axioms are assumed
– Rules not explicitly stated
50
Basic Terms
■ THEOREM ‫قضيہ کليہ‬
– It is a statement that can be shown to be true.
– A general proposition not self-evident but proved by a chain of

reasoning.
– A truth established by means of accepted truths.
– Also called FACTS or RESULTS after being proved.
51
Basic Terms
■ LEMMA ‫قضيہ‬
– A less important theorem that is helpful in the proof of other results.

– An intermediate theorem in an argument or proof.
■ COROLLARY ‫ منطقی نتيجہ‬٬ ‫ضمنی نتيجہ‬
– A proposition that follows from (and is often appended to) one already
proved.
– A direct or natural consequence or result of a theorem.
52
Basic Terms
■ CONJECTURE ‫ اندازه‬،‫تخمينہ‬
– An opinion or conclusion formed on the basis of
incomplete information.
– Form an opinion or supposition about (something) on

the basis of incomplete information.
– If it is proved, it becomes theorem; otherwise not.
– On the basis of:

■ Partial evidence ‫استقراء ناقص‬
■ Heuristic argument ‫اﻻشباه و النظائر‬
■ Intuition ‫ ذوق‬،‫ وجدان‬of an expert
53
Basic Terms
■ AXIOM / POSTULATE ‫مسلمہ – اصول – بدیهی‬
– A thing suggested or assumed to be true.
– It is used as the basis for reasoning, discussion, or belief.
– These are considered to be self-evident.
– These are assumed to be true without any proof or demonstration.
54
Methods
55
Basic Concepts
■ An integer n can be even if and only if 2 is one of its factors.
– i.e. n=2 * k
■ An integer n can be odd if and only if

n=2 * k + 1 OR n=2*k–1
Here, k represents some integer.
56
Basic Concepts
■ An integer n is prime if and only if it has only two
factors; one of them is 1.
– i.e. if n=r.s; then r = 1 or s = 1
■ An integer n is composite if and only if it has at

least a pair of factors such that none of them is 1.
– i.e. if n=r.s; then r ≠ 1 and s ≠ 1
Let r and s represents some positive integer.
57
Basic Concepts
■ An integer n is divisible by d if and only if n=d.k for some integer k
and d ≠ 0.
■ An integer n is ‫ا‬perfect square if and only if n=k2 for some integer

k.
■ A real number r is rational only if r= and b≠0.

58
Direct Proof
■ In order to directly prove a conditional statement of the form "If p,
then q", it is sufficient to consider the situations in which the
statement p is true.
59
Direct Proof
■ The implication p →q can be proved by showing that if p is true,
then q must also be true.
■ This shows that the combination p true and q false never occurs.
No need to consider where

p=FALSE
60
Practice Questions
■ Prove that the sum of two odd integers is even.
61
Practice Questions
■ Prove that if n is any even integer, then(-1)n=1.
62
Practice Questions
■ Prove that the product of an even integer and an odd integer is
even.
63
Practice Questions
■ Prove that the square of an even integer is even.
64
Practice Questions
■ Prove that if n is an odd integer, then n3 + n is even.
65
Practice Questions
■ Prove that, if the sum of any two integers is even, then so is their
difference.
66
Practice Questions
■ Prove that the sum of any two rational numbers is
rational.
67
Practice Questions
■ Given any two distinct rational numbers r and s with r < s. Prove
that there is a rational number x such that r < x < s.
68
69
Practice Questions
■ Prove that the sum of any three consecutive integers is divisible
by 3.
70
Practice Questions
■ Prove the statement: There are real numbers a and b such that
71
72
Proof by Contraposition
■ We know that a conditional statement and its contrapositive are
logically equivalent.
■ So, if we show that the contrapositive is true, then the original

statement will also be true.
■ Here, we directly prove the contrapositive of a statement. Thus, it

is an indirect proof in which the negation of consequence is
proved directly.
73
Proof by Contraposition
■ The method of proof by contrapositive may be summarized as:
– Express the statement in the form if p then q.

p→ q
– Rewrite this statement in the contrapositive form if not q then not p. So it

becomes indirect.
~q→~p
– Prove the contrapositive by a direct proof.
74
Example
75
Example
76
Example
Proof:
Contrapositive: if n is odd, then n3+5 is even.
As n is odd, then n2 is also odd because product of two odd integers is also odd.
Similarly, n3 will also be odd being the product of two odd numbers.
As, sum of two odd numbers is always even, so n3+5 will be an even number.
77
Example
78
Example
79
Proof by Contradiction
Reductio ad absurdum: p →q ≡ (p ~q) →c
(Law of Reduction to Contradiction)
■ The negation of the implication p→q is

~(p→q) ≡ ~(~p q)
≡ ~~p ~q De-Morgan’s Law
≡ p ~q
■ If the implication is true, then its negation must be false, i.e., leads to a
contradiction.
■ Hence p→q ≡ (p ~q) → c, where c is a contradiction.

80
Proof by Contradiction
■ Thus, to prove an implication p → q by contradiction
method, we suppose that the condition p and the
negation of the conclusion q, i.e., (p ~q) is true and
ultimately arrive at a contradiction.
■ The method of proof by contradiction, may be

summarized as follows:
– Suppose the statement to be proved is false.
– Show that this supposition is also false and hence
leads logically to a contradiction.
– Conclude that the statement to be proved was true.
81
Example
82
Example
83
Example
84
Example
85
ALGORITHMS
Lecture # 10, 11 & 12

MatheMatical induction
Strong induction
IN S T R U C T O R : S u la m a n A h m a d N a z
Mathematical Induction
■ Mathematical induction is a mathematical proof technique.
■ It requires two cases to be proved:

1. The first case, called the BASIS STEP, proves that the property holds for
the number 0.
2. The second case, called the INDUCTION STEP, proves that, if the property
holds for one natural number n, then it holds for the next natural
number n + 1.
2
■ These two steps establish the property P(n) for
every natural number n = 0, 1, 2, 3, ...
■ The BASIS STEP need not begin with zero.
■ Often it begins with the number one, and it can

begin with any natural number, establishing the
truth of the property for all natural numbers
greater than or equal to the starting number.
3
■ Logically, it can be expressed as:
Basis Step
↓
↑
Induction Step
4
Example: Summations
5
Example: Summations
6
Task 1
7
Task 1
8
Task 2
9
Task 2
10
Example: Divisibility
11
Example: Divisibility
12
Task 3
13
Task 3
14
Example: Inequality
15
Example: Inequality
16
Task 4
17
Task 5
18
Task 5
19
Example: Property of Sequence
5ak-1
20
Example: Property of Sequence
5ak-1
21
Task 6
22
Task 6
23
Task 6
So
or As k>1, so it is obvious that the equality is true.
24
The Well Ordering Principle
■ It is one of those facts in mathematics that are very obvious.
■ Statement:
Any nonempty subset of natural numbers has a least/minimum
element.
■ This is only true for the natural numbers.

■ It is not true for the Integers, the Rational Numbers, the Real
numbers and the Complex numbers.
The Well Ordering Principle
■ Let A be a set such that:
– A  N.
– A ≠ .
So, not only the set has to be a subset of natural numbers, it has to be nonempty.
■ Then there is some natural number m such that:
– m  A.
– m  a, for all a  A.
This m is the minimum of (less than or equal to) all the elements in A and, most
important, m is in A.
Example 1
■ If A={2,5,7,10,22}, then it is clear that
– A  N.
– A ≠ .
■ The well ordering principle says that A has a minimum.

– In this case, m=2 is the minimum of A.
Example 2
■ If A={nN | n2+3n-200>0}, then it is clear that
– AN by the definition of A
– A≠ needs to be checked
The well ordering principle does not tell us

what “m” is exactly; it just tells us that “m” exists

Example 3
■ If A={nN | n.sin(2n)>8}, then it is clear that
– AN by the definition of A
– A≠ needs to be checked
The well ordering principle does not tell us

what “m” is exactly; it just tells us that “m” exists

Mathematical Induction is a Valid Proving Technique
Proof by Contradiction:
Suppose, n P(n) is false.
P(1) It means there are some +ive integers for which P(n) is
k (P(k)P(k+1)) false. Let it be set S with least k (by using well ordering
principle), so P(k) is also false.
------------------------------- We know that, 1S. Hence, 1<k
 n P(n) In other words k-1 is also positive.
As k-1<k, so k-1S and P(k-1) must be true.
From second premise, using modus ponens, P(k) is
also true which leads to contradiction.
Hence, n P(n) must be true.
Types of Induction
■ Weak Induction
– Mathematical Induction
– Principle of mathematical Induction
– Incomplete Induction
■ Strong Induction
– Second Principle of Mathematical Induction
– Complete Induction
■ Surprisingly, both are just equivalent.

Weak Induction vs. Strong Induction
P(1) P(1)
k (P(k)P(k+1)) k (P(1)P(2)  …  P(k)P(k+1))
------------------------------- -----------------------------------------------------------
 n P(n)  n P(n)
Example
■ Prove that for all n>1, n is divisible by a prime number.
■ We will use strong mathematical induction to prove it.

■ P(n) : n is divisible by a prime number, for all n>1
P(2)
k (P(2)P(3)  …  P(k)P(k+1))
-----------------------------------------------------------
 n P(n)
Example
■ BASE CASE:
P(2) : 2 is divisible by a prime number.
It is TRUE as every number is divisible by itself.
■ INDUCTIVE CASE:
We assume that P(2), P(3), …, P(k) are all true,
Now, if P(2), P(3), …, P(k) are all true,
then it must imply that P(k+1) is also true.
If it happens, then we can conclude that n P(n) is also true
for all n>1.
Example
So, we assume that P(2), P(3), …, P(k) are all true,
Now k+1 can be either a prime or a composite number.
If it is prime, then nothing to prove more as every number is divisible by
itself.
If it is a composite number, then let it be a product of two numbers. Now,
k+1 = a.b | 1<a,b<k+1
As “a” and “b” are integers between 2 and k, it is obvious that that both of
these are divisible by a prime number.
In other words, P(k+1) is TRUE or k+1 is also divisible by a prime number.
Hence, we can conclude that for all n>1, n is divisible by a prime number.
Practice Questions
1. If n  Z, n >1, then n can be written as product of primes.
2. Show that any amount in cents ≥ 8 cents can be obtained

using 3 cents and 5 cents coins only.
n
when n2, i.e., A1  A2  ...  An  A1  A2  ...  An 

n
3. Prove  Aj   Aj
j 1
j 1
4. Prove that postage ticket of amount  12 cents can be formed

using only 4 cent and 5 cent stamps.
ALGORITHMS
Lecture # 13, 14 & 15

INTRODUCTION TO TIme COmplexITy,
TIme COmplexITy Of lOOpS
■ When we talk about time complexity of algorithms, what do we
actually mean?
■ It is not the actual running time. It is not the standard.

– It varies from machine to machine & compiler to compiler.
– It even varies on one machine due to availability of resources.
■ We actually calculate the estimation, not actual running time.
■ Complexity can be viewed as the maximum number of primitive
operations that a program may execute.
■ Regular operations are:
– Addition
– Multiplication
– Assignment
– Accessing array element, etc.
■ We may leave some operations uncounted and concentrate on
those that are performed the largest number of times.
■ We define complexity as a numerical function T(n) - time versus
the input size n.
■ We want to define time taken by an algorithm without depending

on the implementation details.
■ Talgorithm#1(n) = n(n-1) = n2 - n //quadratic time
■ Talgorithm#2(n) = 2(n-1) = 2n – 2 //linear time

Asymptotic Complexity
■ The resulting function gives only an
approximate measure of efficiency of the
original function.
■ However, this approximation is

sufficiently close to the original.
■ This measure of efficiency is called

asymptotic complexity.
Elimination of Lower Order Terms
■ Any terms that do not substantially change the function’s
magnitude should be eliminated from the function.
■ Consider the following quadratic function:
■ For small values of n, the last term, 1,000, is the largest. But
we calculate time complexity to see the behavior on large input
sets.
Elimination of Lower Order Terms
≈n2
Asymptotic Notations
■ Asymptotic notations are mathematical tools to represent the
complexity of algorithms for asymptotic analysis.
1. Big-Oh (O) Notation

2. Big-Omega (Ω) Notation
3. (Big) Theta () Notation
4. Small-Oh (o) Notation
5. Small-Omega (ω) Notation
Big-Oh (O) Notation
■ The Big O notation defines an upper bound of an algorithm.
■ It bounds a function only from above.
■ It is the lowest upper bound.
■ It specifically describes the worst case scenario.

f(n)c.g(n)
Big-Oh (O) Notation for all nN0
& c>0, n1
Let f(n) is the function of
growth of time for some input
n.
Then there is a function g(n)
such that c.g(n) is the lowest
upper bound of f(n).
In other words, after some
value N0, the value of c.g(n)
will always be greater than or
equal to g(n)
f(n)c.g(n)
Example for all nN0
& c>0, n1
■ Let f(n)=3n+2 and g(n)=n. Can we say that f(n)=O(g(n))?
■ Solution:
f(n)  c.g(n)
 3n+2  c.n
Now, c can be any number greater
Than or equal to 4.
 3n+2  4.n
and N0=2. Hence f(n)=O(g(n))=O(n)
Big-Omega (Ω) Notation
■ The Big Ω notation defines a lower bound of an algorithm.
■ It bounds a function only from below.
■ It is the greatest lower bound.
■ It specifically describes the best case scenario.

f(n)c.g(n)
Big-Omega (Ω) Notation for all nN0
& c>0, n1
Let f(n) is the function of
growth of time for some input
n.
such that c.g(n) is the greatest
lower bound of f(n).
In other words, after some
value N0, the value of c.g(n)
will always be less than or
equal to g(n)
f(n)c.g(n)
& c>0, n1
■ Let f(n)=3n+2 and g(n)=n. Can we say that f(n)=Ω(g(n))?
■ Solution:
f(n)  c.g(n)
 3n+ 2  c.n
Now, c can be any number smaller
than or equal to 3.
 3n+2  3.n
and N0=1. Hence f(n)= Ω(g(n))= Ω(n)
(Big)-Theta () Notation
■ The Theta  notation defines a both upper and lower bounds of
an algorithm.
■ It bounds a function both from above and below.
■ It is the also called as tight bound.
■ It is used when we have the same function as lower as well as

upper bound.
c1.g(n)f(n)c2.g(n)
(Big)-Theta () Notation for all nN0
& c1,c2>0, n1
Let f(n) is the function of growth C2 g(n)
of time for some input n.
such that c1.g(n) is the greatest
lower bound and c2.g(n) is the
lowest upper bound of f(n). C1 g(n)
In other words, after some value
N0, the value of c1.g(n) will
always be less than or equal to
f(n) and the value of c2.g(n) will
always be greater than or equal
to f(n).
c1.g(n)f(n)c2.g(n)
& c1,c2>0, n1
■ Let f(n)=3n+2 and g(n)=n. Can we say that f(n)=(g(n))?
■ Solution:
c1.g(n)  f(n)  c2.g(n)
From previous two examples, it is
obvious that:
3n  3n+2  4n
where N0=2.
Hence f(n)= (g(n))= (n)
Small-Oh (o) Notation
■ The small O notation also defines an upper bound of an
algorithm.
■ It bounds a function only from above.
■ It is the just the upper bound.
■ Big-Oh is the least upper bound while it is just any upper bound.
Small-Omega (ω) Notation
■ The small ω notation also defines a lower bound of an algorithm.
■ It bounds a function only from below.
■ It is the just the lower bound.
■ Big-Omega is the greatest lower bound while it is just any lower

bound.
time
Ω(n)
ω(n)
ω(n)
Growth of functions
Usually growth of functions
can be categorized into the
following:
– Constant time
– Logarithmic time
– Linear time
– N-logarithmic time
– Polynomial time
– Exponential time
– Factorial time
Remarks
■ Most of the time, we are concerned with the worst
case complexity of algorithms.
■ It means we want to rate the algorithm in the

appropriate category, such that on any input, what will
be the maximum time required to transform input to
output.
■ So, most of the time, we will use the Big-Oh notation.

Input Cases & Analysis Cases
■ There are three types of input cases.
– Best Case
– Worst Case
– Arbitrary/Random Case
■ There are three types of analysis cases.

– Best Case Analysis
– Worst Case Analysis
– Average Case Analysis
Example
■ Lets take the example of Linear Sequential Search.
■ It scans the array elements one by one and searches for the
required value.
■ If there are n elements in the array, and key is the value to be
searched, then its algorithm may be roughly refined as:
– found=false
– For i = 1 to n
– If Array[i]=key
– found=true & stop;
– End if
– End for
Example
num_1

num_2
– Best Case key=num_1
– Worst Case key=num_n or key not found
– Arbitrary/Random Case key=any number num_3
num_4
– Best Case Analysis 1 =Ω(1)
…
– Worst Case Analysis n =O(n)
– Average Case Analysis (n+1)/2 =O(n)
num_n
■ We have already talked about how to get the time complexity function
of any algorithm.
■ We consider the basic operations (computational steps) only:

– Arithmetic operations
– Logical operations
– Relational operations
– Accessing the element of an array
– Assignment operation
■ We do not consider all the operations. We consider only the most

dominant ones as we need the approximation.
Time Complexity of Iterations
■ What if a set of instructions is repeated many times in an
algorithm?
■ Usually a set of code is repeated using either of the two

techniques:
– Loops
■ For Loop
■ While Loop
– Recursions
Iterative vs. Recursive Time
A() A(n)
{ {
For j = 1 to n If (-----)
Print “Hello”
A(n/2)
}
}
Time Complexity (Iterations)
For i=1 to n T(n)=n
<computational instruction>
=O(n)
-----------------------------------------------------------
For i=1 to n
For j=1 to n
T(n)=n.n=n2
<computational instruction> =O(n )

2
i 1 2 3 4 5 6 7 … n
J n n n n n n n … n
For i=1 to n T(n)=n.n.n=n3
For j=1 to n
For k=1 to n
=O(n )
3
----------------------------------------------------------------
For i=1 to n
For j=1 to n T(n)=n.n.n.n=n 4
For k=1 to n
For l=1 to n =O(n )
4
i=1 T(n)=k
s=1
While(s<=n)
i=i+1
s=s+i
S 1 3 6 10 15 21 28 … n
i 1 2 3 4 5 6 7 … k
Things to remember
Sum of 1st m natural numbers=m(m+1)/2

i=1 T(n)=k =O( )
s=1
While(s<=n)
i=i+1
s=s+i
<computational instruction> n=k(k+1)/2
S 1 3 6 10 15 21 28 … n 2n=k +k
2
i 1 2 3 4 5 6 7 … k
k≈
For i=1; i2<=n;i++

T(n)=O( ) i <=n
2
i<=
T(n)=100+200+…+n.100
For i=1 to n =100(1+2+…+n)
For j=1 to i =100.n(n+1)/2
For k=1 to 100 =O(n )
2
i 1 2 3 4 … n
j 1 times 2 times 3 times 4 times … n times
k 1x100 2x100 3x100 4x100 … nx100
times times times times times
Things to remember
Sum of squares of 1st m natural numbers=m(m+1)(2m+1)/6

T(n)=1(n/2)+4(n/2)+9(n/2)+…+n2(n/2)
=(n/2)(1+4+9+…+n2)
For i=1 to n
For j=1 to i2
=(n/2)(12+22+32+…n2)
For k=1 to n/2 =(n/2).n(n+1)(2n+1)/6 = O(n 4)
I 1 2 3 4 … n
J 1 times 4 times 9 times 16 times … n2 times
k n/2x1 n/2x4 n/2x9 n/2x16 … n/2xn2
times times times times times
i 1 2 4 8 … n
For i=1 to n & i=i*2

---------------------------
For i=1 to n & i=i*3
---------------------------
For i=1 to n & i=i*m
i 1=20 2=21 4=22 8=23 … n=2k
For i=1 to n & i=i*2 T(n)=k+1

<computational instruction> =log2n+1
--------------------------- n=2k
=O(log2n)
For i=1 to n & i=i*3 log2n=log22k
<computational instruction> log2n=k.log22
--------------------------- log2n=k
For i=1 to n & i=i*m Or
<computational instruction> k=log2n
i 1=20 2=21 4=22 8=23 … n=2k
For i=1 to n & i=i*2 T(n)=k+1

<computational instruction> =log2n+1
--------------------------- n=2k
=O(log2n)
For i=1 to n & i=i*3 log2n=log22k
<computational instruction> log2n=k.log22
--------------------------- T(n)=O(log3n) log2n=k
For i=1 to n & i=i*m Or
<computational instruction> k=log2n
T(n)=O(logmn)
For i=n/2 to n T(n)=(n/2)(n/2)log2n
For j=1 to n/2 =O(n2.log2n)
For k=1 to n & k=k*2
-----------------------------------------------------------
For i=n/2 to n T(n)=(n/2).log2n.log2n
For j=1 to n & j=2*j =O(n.(log2n)2)
For k=1 to n & k=k*2
While(n>1)
T(n)=O(log2n)
n=n/2
i = 1 to n i = n to 1
←same→
i=i*2 i=i/2
𝒏 𝟐 𝟒
where = 0.5772156649…

T(n)=n+n/2+n/3+…+n/n
For i=1 to n
For j=1 to n & j=j+i =n(1+1/2+1/3+…+1/n)
<computational instruction> =n(Hn)
≈n.ln n
=O(n. ln n)
i 1 2 3 … n
j n times n/2 n/3 … n/n
times times times
ALGORITHMS
Lecture # 16, 17 & 18

RecuRsive AlgoRithms,
RecuRsive functions
Time Complexity of Iterations
■ What if a set of instructions is repeated many times in an
algorithm?
■ Usually a set of code is repeated using either of the two

techniques:
– Loops
■ For Loop
■ While Loop
– Recursions
Iterative vs. Recursive Time
A() A(n)
{ {
For j = 1 to n If (-----)
Print “Hello”
A(n/2)
}
}
Recursions
■ A function calls itself repeatedly
■ QUESTION: For how long a function will call itself?
■ ANSWER: Slight modification in parameters
■ PURPOSE: To reach the ANCHOR condition
■ ANCHOR CONDITION: The point at which the recursive calls

will come to an end
4
Example
Algo(n)
{
//some lines of code
If n=1
return 1
else
return Algo(n-1)
}
-------------------------------------------------------------------------
T(n)=c+T(n-1) for n>1; otherwise T(n) =c for n=1
-------------------------------------------------------------------------
Here, “c” is the time complexity of “//some lines of code” and the comparison
And, T(n-1) is the time complexity of the next recursive call
5
Example T(n) =c+c+…+c
Algo(n) =n.c
Algo(n-1) =O(n)
Algo(n-2)
c c
c
c Algo(1)
6
Methods of Solution
7
Back Substitution Method
■BASIC IDEA:
■ Substitute the value of next iteration into
current iteration and so on.
■ Do this until the last recursive call.
■ Last recursive call is controlled by the
Anchor condition.
8
■ Let’s consider an example:
T(n) =T(n-1)+c, for n>1
T(1) =c  This is the Anchor Condition
-------------------------------------------------------------------------
T(n) =T(n-1)+c …eq(i)
T(n-1) =T(n-1-1)+c
=T(n-2)+c …eq(ii)
T(n-2) =T(n-2-1)+c
=T(n-3)+c …eq(iii)
9
Putting eq(ii) in eq(i): T(n) =T(n-1)+c …eq(i)
T(n-1) =T(n-2)+c …eq(ii)
T(n) =T(n-1)+c T(n-2) =T(n-3)+c …eq(iii)
=T(n-2)+c+c
=T(n-2)+2c …eq(iv)
Putting eq(iii) in eq(iv):
T(n) = T(n-3)+c+2c
=T(n-3)+3c
Suppose we reach the anchor condition after k calls:
T(n) =T(n-k)+kc ..eq(v)
10
As we know that T(1) will be the last recursive call,
so for what value of k will T(n-k) become T(1).
n-k=1  k=n-1
So eq(v) will become:
T(n) =T(n-k)+kc
=T(1)+(n-1)c
= c+nc-c
= O(n)
11
■ Let’s consider another example:
T(n) =T(n/2)+n, for n>1
T(1) =n  This is the Anchor Condition
-------------------------------------------------------------------------
T(n) =T(n/2)+n …eq(i)
T(n/2) =T(n/2/2)+n/2
=T(n/22)+n/2 …eq(ii)
T(n/22) =T(n/22/2)+n/22
= T(n/23)+n/22 …eq(iii)
12
Putting eq(ii) in eq(i): T(n) =T(n/2)+n …eq(i)
T(n/2) =T(n/22)+n/2 …eq(ii)
T(n) =T(n/2)+n T(n/22) = T(n/23)+n/22 …eq(iii)
=T(n/22)+n/2+n
=T(n/22)+n/2+n …eq(iv)
T(n) =T(n/23)+n/22+n/2+n
=T(n/23)+n/22+n/2+n
T(n) =T(n/2k)+n/2k-1+…+n/22+n/2+n ..eq(v)
13
As we know that T(1) will be the last recursive call,
so for what value of 2k will T(n/2k) become T(1).
n/2k=1  n=2k
T(n) = T(n/2k)+n/2k-1+…+n/22+n/2+n
= T(1)+ n/2k-1+…+n/22+n/2+n
= n + n/2k-1+…+n/22+n/2+n
= n+n(1/2k-1+…+1/22+1/2+1)
 n+n(1+1) = n+2n = 3n = O(n)
14
Explanation
*Pictures taken from Wikipedia

Explanation
Recursion Tree Method
■BASIC IDEA:
■ Draw branches of tree for each recursive
call.
■ Do this until the last recursive call.
■ Last recursive call is controlled by the
Anchor condition.
17
■ Let’s consider an example:
T(n) = 2T(n/2)+c, for n>1
T(1) = c  This is the Anchor Condition
-------------------------------------------------------------------------
Note that here, at each step, 2 branches will be drawn; decreasing
the input size by a factor of 2 at each step.
And a total of c work will be done at each splitting.
18
T(n) c
T(n/2) T(n/2) 2c
T(n/22) T(n/22) T(n/22) T(n/22) 4c
T(n/23) T(n/23) 8c
T(n/2k) T(n/2k)
=T(1) =T(1)
19
T(n) 2 0c
T(n/2) T(n/2) 21c
T(n/22) T(n/22) T(n/22) T(n/22) 2 2c
T(n/23) T(n/23) 2 3c
T(n/2k) T(n/2k) 2kc

=T(1) =T(1)
20
As we know that T(1) will be the last recursive call, so for
what value of 2k will T(n/2k) become T(1).
n/2k=1  n=2k
T(n) = 20c + 21c + 22c + … + 2kc
= c(20 + 21 + 22 + … + 2k)
= c(20 + 21 + 22 + … + n)
= c(total number of nodes in the tree)
We know that, in a complete binary tree, if there are n
leaf nodes, then the tree has a total of 2n-1 nodes.
T(n) = c(2n-1) = O(n)
■ Let’s consider another example:
T(n) = T(n-1)+n, for n>1
T(1) = n  This is the Anchor Condition
-------------------------------------------------------------------------
Note that here, at each step, only 1 branches will be drawn;
decreasing the input size by 1 at each step.
And a total of n work will be done at branching.
22
T(n) n
T(n-1) n-1
T(n) = n+(n-1)+(n-3)+…+2+n
T(n-2) n-2
= n+(n-1)+(n-3)+…+2+1-1+n
= n(n+1)/2 -1 +n
T(n-3) n-3
= O(n2)
T(n-k) n
=T(1) 23
Master Theorem
■ The master theorem provides a solution in asymptotic
terms (using Big  or Big O notations) for recurrence
relations of types that occur in the analysis of many
divide and conquer algorithms.
■ The function must be in some specific pattern.
■ Not all the problems can be solved by Master Theorem.
■ It has various versions; we will use an easy one.
24
Master Theorem
■ Function Pattern:
– T(n)=aT(n/b)+ (nc.logdn)
– Where a,b,c,d are constants

– Constraints:
■ a>=1
■ b>1
■ c>=0
■ d is any real number
25
Master Theorem
■ Now, we have three possibilities:
1. If a>bc, then T(n)= (nlogba)
2. If a=bc, then:
– If d>-1, then T(n)= (nlogbalogd+1n)
– If d=-1, then T(n)= (nlogbalog2n)
– If d<-1, then T(n)= (nlogba)
3. If a<bc, then:
– If d>=0, then T(n) = (nclogdn)
– If d<0, then T(n) = O(nc) 26
Master Theorem
Lets consider an example:

T(n)=3T(n/2)+n2
Here, a=3, b=2, c=2, and d=0
Now, bc=22=4
Here a<bc, so third possibility is our selection.
As d=0, so solution will be:
T(n) = (n2.log0n) = (n2)
27
Master Theorem
■ Here are some practice questions.
■ Do solve them by yourselves and consult in case of
any problem.
1. T(n)=4T(n/2)+n2
2. T(n)=T(n/2)+n2
3. T(n)=2nT(n/2)+nn
4. T(n)=16T(n/4)+n
5. T(n)=2T(n/2)+n.log n
6. T(n)=2T(n/2)+n/log n
7. T(n)=2T(n/4)+n0.51
28
Announcement
■ Online Quiz #1
■ Topic:
– Time Complexity of Algorithms

– See Lectures 13 to 18
■ Date:
– Date: Thursday, 17 December, 2020
ALGORITHMS
Lecture # 19, 20 & 21

IntroductIon to
Space complexIty
Space Complexity
■ Until now, we mainly focused on the Time Complexity
of algorithms.
■ Now, we will see:

– What is Space Complexity
– How to calculate it?
– What to consider and what to not?
2
Space Complexity
■ Given an algorithm, how much space (memory) is required to
finish the job.
■ Job: Transform Input to Output
■ We define Space Complexity in terms of the input size, n.
■ By space, we mean the extra memory being used. Original input,

n, is not included.
3
Order of Growth
Just like Time Complexity,
the order of growth of
Space Complexity can also
be categorized as:
– Constant space
– Logarithmic space
– Linear space
– N-logarithmic space
– Polynomial space
– Exponential space
– Factorial space
4
space
Ω(n)
ω(n)
ω(n)
5
Remarks
■ Most of the time, we are concerned with the worst
case complexity of algorithms.
■ It means we want to rate the algorithm in the

appropriate category, such that on any input, what will
be the maximum space required to transform input to
output.
■ So, most of the time, we will use the Big-Oh notation.

6
Input Cases & Analysis Cases
– Best Case
– Worst Case
– Arbitrary/Random Case

– Best Case Analysis
– Worst Case Analysis
– Average Case Analysis
7
Example 1
1 2 3 4 5 … n
Algo( Array[n] )
{
For i = 1 to n
S(n) =/ n + 1
Array[i] = i;
}
S(n) = 1 = O(1) = O(c)
8
Example 2
1 2 3 4 5 … n
Algo( Array[n] )
{
For i = 1 to n
S(n) =/ n + 2
Array[i] = j;
}
S(n) = 2 = O(2) = O(1) = O(c)
9
Array
Example 3 1 2 3 4 5 … n
B
Algo( Array[n] )
1 2 3 4 5 … n
{
New B[n];
S(n) = n + 1
For i = 1 to n
B[i]=Array[i];
} S(n) = O(n)
10
Array
Example 4 1 2 3 4 5 … n
B
Algo( Array[n] )
1 2 3 4 5 … n
{
2
New B[n,n];
For i = 1 to n 3
For j = 1 to n
B[i,j]=Array[i] * j;
n
}
S(n) = n2 + 2
S(n) = O(n2)
11
Space Complexity of Recursive Algorithms
■ Before starting to analyze the space
complexity of recursive algorithms, we
need to know how recursions actually
work.
■ There is also a concept of the CALL

STACK or RECURSION STACK in this
respect.
12
STACK
■ Suppose you want to put these balls
on this tower.
13
STACK
■ First, we put a red ball.
14
STACK
■ Now, we put a green ball.
15
STACK
■ Next, we put a black ball on top of it.
16
STACK
■ At the end, we put the blue ball on
top of it.
17
STACK
■ Now, please answer the following questions.
– Can we get the black ball before getting the

blue ball?
– Can we insert a new ball between green and
red ball?
■ The answer to both of these questions is:

NO
18
STACK
■ It is because, there is only one way to
remove or insert a ball.
– We can insert any ball only at the top of other
balls (if any)
– We can remove only the topmost ball at a time.
– We have no direct access to any other balls,
except the one at the top.
■ Such a list is called a “Stack”.
19
STACK
■ In a stack, we have two operations:
– Push(X)
– Pop()
■ Push(X) means to insert “X” at the top of the

stack.
■ Pop() means to remove the topmost element
of the stack.
20
STACK
■ In a stack, we have two operations:
– Push(X)
– Pop()
■ Push(X) means to insert “X” at the top of the

stack.
■ Pop() means to remove the topmost element
of the stack.
■ If there is nothing in the stack, it is empty.
21
STACK
■ Similarly, initially stack is empty.
22
STACK
■ Similarly, initially stack is empty.
Last
■ After performing the following five
operations, the stack looks like this. Fourth
– Push(“First”)
– Push(“Second”) Third
– Push(“Third”)
Second
– Push(“Fourth”)
– Push(“Last”) First
23
STACK
■ Now, we will perform the following operation.
Last
– Pop()
Fourth
Third
Second
First
24
STACK
Last
■ Now, we will perform the following operation.
– Pop()
Fourth
■ You will see that the “Last” element has
been popped out first. Third
Second
■ In other words, we can say that the behavior
of Stack is LIFO (Last in First out). First
■ In other words, FILO (First in Last out).

25
STACK
The stack is a simple data structure. We may have been using a
stack in our lives without even realizing it!
26
The Call Stack
■ Your computer uses a stack internally called the call stack. Let’s
see it in action. Here’s a simple function:
■ This function greets you and then calls two other functions. Here
are those two functions:
27
The Call Stack
■ Let’s walk through what happens when you call a function.

■ Suppose you call greet(“maggie”).
■ First, your computer allocates a box of memory for that function call.
28
The Call Stack
■ Let’s walk through what happens when you call a function.

■ Suppose you call greet(“maggie”).
■ First, your computer allocates a box of memory for that function call.
■ Now let’s use the memory.
■ The variable name is set to “maggie”.
■ That needs to be saved in memory.
29
The Call Stack
■ Every time you make a function call, your

computer saves the values for all the variables
for that call in memory like this.
■ Next, you print hello, maggie!
■ Then you call greet2(“maggie”).
■ Again, your computer allocates a box of memory
for this function call.
30
The Call Stack
■ Your computer is using a stack for these

boxes.
■ The second box is added on top of the first
one.
■ You print how are you, maggie?
■ Then you return from the function call.
■ When this happens, the box on top of the
stack gets popped off.
31
The Call Stack
■ Your computer is using a stack for these

boxes.
■ The second box is added on top of the first
one.
■ You print how are you, maggie?
■ Then you return from the function call.
■ When this happens, the box on top of the
stack gets popped off.
32
The Call Stack
■ Now the topmost box on the stack is for the greet

function, which means you returned back to the
greet function.
■ When you called the greet2 function, the greet
function was partially completed.
■ This is the big idea behind this section: when you call
a function from another function, the calling function is
paused in a partially completed state.
■ All the values of the variables for that function are
still stored in memory.
33
The Call Stack
■ Now that you’re done with the greet2 function,

you’re back to the greet function, and you pick
up where you left off.
■ First you print getting ready to say bye…. You
call the bye function.
■ A box for that function is added to the top of
the stack.
34
The Call Stack
■ Then you print ok bye! and return from the

function call.
■ Bye function is popped out.
■ And now you’re back to the greet function.
35
The Call Stack
■ In greet function, there’s nothing else to be

done now.
■ So, we will pop it too.
36
The Call Stack
■ The stack is now empty.
■ This stack, used to save the variables for

multiple functions, is called the call stack.
37
The Call Stack with Recursion
■ Recursive functions use the call stack too!
■ Let’s look at this in action with the factorial function.
■ factorial(5) is written as 5!, and it’s defined like this:

5 * 4 * 3 * 2 * 1.
■ Similarly, factorial(3) is:
3 * 2 * 1.
38
■ Here’s a recursive function to calculate the factorial of a number:
■ Now let’s call fact(3) and step through this call line by line and
see how the stack changes.
39
40
41
42
43
A(3)
Example 5
A( n ) A(2) //do something
{
If n >= 1 A(1) //do something
A(n-1);
//do something
} A(0) //do something
A(0) 1 call
A(1) 2 calls
A(2) 3 calls
A(3) 4 calls
A(n) n+1 calls

44
A(3)
Example 5
{
A(n-1);
//do something
A(0) 1 call
A(1) 2 calls
A(2) 3 calls
A(3) 4 calls
A(n) n+1 calls 45

A(3)
Example 5
{
A(n-1);
//do something
A(0) 1 call
A(1) 2 calls
A(2) 3 calls
A(3) 4 calls
A(3)
A(n) n+1 calls 46

A(3)
Example 5
{
A(n-1);
//do something
A(0) 1 call
A(1) 2 calls
A(2)
A(2) 3 calls
A(3) 4 calls
A(3)
A(n) n+1 calls 47

A(3)
Example 5
{
A(n-1);
//do something
A(1)
A(0) 1 call
A(1) 2 calls
A(2)
A(2) 3 calls
A(3) 4 calls
A(3)
A(n) n+1 calls 48

A(3)
Example 5
{
A(n-1);
//do something A(0)

A(1)
A(0) 1 call
A(1) 2 calls
A(2)
A(2) 3 calls
A(3) 4 calls
A(3)
A(n) n+1 calls 49

A(3)
Example 5
{
A(n-1);
//do something
A(1)
A(0) 1 call
A(1) 2 calls
A(2)
A(2) 3 calls
A(3) 4 calls
A(3)
A(n) n+1 calls 50

A(3)
Example 5
{
A(n-1);
//do something
A(0) 1 call
A(1) 2 calls
A(2)
A(2) 3 calls
A(3) 4 calls
A(3)
A(n) n+1 calls 51

A(3)
Example 5
{
A(n-1);
//do something
A(0) 1 call
A(1) 2 calls
A(2) 3 calls
A(3) 4 calls
A(3)
A(n) n+1 calls 52

A(3)
Example 5
{
A(n-1);
//do something
A(0) 1 call
A(1) 2 calls
A(2) 3 calls
A(3) 4 calls
A(n) n+1 calls 53

A(3)
Example 5
{
A(n-1);
//do something
A(0) 1 call
You can see that for an A(1) 2 calls
Input of size n, we need A(2) 3 calls

A(3) 4 calls
a stack of height n+1.
A(n) n+1 calls 54
A(3)
Example 5
{
A(n-1);
//do something
A(0) 1 call
If each block of stack A(1) 2 calls
consists of k memory A(2) 3 calls

A(3) 4 calls
locations, then total
space required is k(n+1) A(n) n+1 calls 55
A(3)
Example 5
{
A(n-1);
//do something
A(0) 1 call
A(1) 2 calls
A(2) 3 calls
S(n) = k(n+1) = O(n) A(3) 4 calls
A(n) n+1 calls 56

A(3)
Example 6
A(2) P(3) A(2)
A( n )
{
A(1) P(2) A(1) A(1) P(2) A(1)
If n >= 1
A(n-1);
print n A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(n-1);
A(0) 1 call
} A(1) 3 calls
A(2) 7 calls
A(3) 15 calls
A(n) m calls
57
A(3)
Example 6
A(2) P(3) A(2)
A( n )
{
A(1) P(2) A(1) A(1) P(2) A(1)
If n >= 1
A(n-1);
A(n-1);
A(0) 1 = 20+1 ― 1 call
} A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
S(n) = k(2n+1-1) = O(2n)
A(n) m = 2n+1 ― 1 calls

58
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(n) m = 2n+1 ― 1 calls

59
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
60
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
61
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
62
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
63
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
64
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
65
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
66
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
67
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
68
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
69
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
70
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
71
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
72
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
73
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
74
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
75
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
76
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
77
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12131
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
78
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12131
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
79
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12131
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
80
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121312
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
81
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121312
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
82
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121312
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
83
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
84
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
85
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
86
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
87
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
88
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
89
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
You can see that for an A(1) 3 = 21+1 ― 1 calls
Input of size n, we need A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(n) m = 2n+1 ― 1 calls
90
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
If each block of stack A(1) 3 = 21+1 ― 1 calls
consists of k memory A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
space required is k(n+1) A(n) m = 2n+1 ― 1 calls
91
A(3) A( n )
Example 6 {
If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
S(n) = k(n+1) = O(n) A(3) 15 = 23+1 ― 1 calls
A(n) m = 2n+1 ― 1 calls

92
A( n )
{
Analyzing Time Complexity If n >= 1
A(n-1);
print n
T(n) = 2T(n-1) + c; n>=1
A(n-1);
T(0) = c  This is the Anchor Condition }
-------------------------------------------------------------------------
T(n) =2T(n-1)+c …eq(i)
T(n-1) =2T(n-1-1)+c
=2T(n-2)+c …eq(ii)
T(n-2) =2T(n-2-1)+c
=2T(n-3)+c …eq(iii)
93
A( n )
{
Putting eq(ii) in eq(i): T(n) =2T(n-1)+c …eq(i) A(n-1);
T(n-1) =2T(n-2)+c …eq(ii) print n
T(n) =2T(n-1)+c T(n-2) =2T(n-3)+c …eq(iii)
A(n-1);
=2[2T(n-2)+c] +c }
=22T(n-2)+2c+c …eq(iv)
T(n) =22[2T(n-3)+c]+2c+c
=23T(n-3)+22c+2c+c
T(n) =2kT(n-k)+2k-1c+…+22c+2c+c ..eq(v)
94
A( n )
{
As we know that T(0) will be the last recursive call, so A(n-1);
for what value of k will T(n-k) become T(0). print n
A(n-1);
n-k=0  n=k
}
T(n) = 2kT(n-k)+2k-1c+…+22c+2c+c
= 2nT(0)+2n-1c+…+22c+2c+c
= 2nc+2n-1c+…+22c+2c+c
= c(2n+2n-1+…+22+2+1)
= c(2n+1 ― 1)
= O(2n) 95
A( n )
{
Our Approach If n >= 1
A(n-1);
S(n) = O(n) A(3)
print n
A(n-1);
T(n) = O(2n)
}
A(2) P(3) A(2)
A(1) P(2) A(1) A(1) P(2) A(1)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
96
A( n )
{
New Approach If n >= 1
A(n-1);
print n
A(3) A(n-1);
}
A(2)
A(1) 0 1 2 3
Ø Ø Ø Ø
A(0)
97
A( n )
{
A(n-1);
print n
A(3) A(n-1);
}
A(2)
A(1) 0 1 2 3
<blank> Ø Ø Ø
A(0) P(1) R(0)
98
A( n )
{
A(n-1);
print n
A(3) A(n-1);
}
A(2)
A(1) P(2) R(1) 0 1 2 3
<blank> 1 Ø Ø
A(0) P(1) R(0)
99
A( n )
{
A(n-1);
print n
A(3) A(n-1);
}
A(2) P(3) R(2)
A(1) P(2) R(1) 0 1 2 3
<blank> 1 121 Ø
A(0) P(1) R(0)
100
A( n )
{
A(n-1);
S(n) = O(n) + 2(n+1) print n
= O(n) A(3) A(n-1);
T(n) = (n+1).c }
= O(n) A(2) P(3) R(2)
A(1) P(2) R(1) 0 1 2 3
<blank> 1 121 1213121
A(0) P(1) R(0)
101
Dynamic Programming
■ Dynamic programming is essentially recursion without
repetition.
■ Developing a dynamic programming algorithm generally
involves two separate steps:
– Formulate problem recursively. Write down a formula for the
whole problem as a simple combination of answers to smaller
sub-problems.
– Build solution to recurrence from bottom up. Write an
algorithm that starts with base cases and works its way up to
the final solution.
Dynamic Programming
■ Decompose into sub-problems like divide and conquer.
■ Sub-problems are dependant.
■ Record results of smaller sub-problems. Dynamic
programming algorithms need to store the results of
intermediate sub-problems.
■ Re-use it for further occurrence.
■ Mostly reduces complexity exponential to polynomial.
In our case, it reduced complexity from exponential time to linear time.
Assignment #2
■ Find the Time Complexity of two algorithms given
in next slides.
■ Both the algorithms solve the same problem.

Remember that sorting takes (n.lo g n) time.
■ Also compare the time complexity of both the

algorithms.
■ Date:
– Date: Sunday, 31 December, 2020 before 23:55 HRS
Groups up to 4 students are allowed, but not mandatory. 104
Assignment #2
Suppose you want to buy a car. You w ant the pick the fastest car. B ut
fast cars are expensive; you want the cheapest. You cannot decide
which is m ore im portant: speed or price. D efinitely you do not want a
car if there is another that is both faster and cheaper. W e say that
the fast, cheap car dom inates the slow , expensive car relative to
your selection criteria. So, given a collection of cars, we want to list
those cars that are not dom inated by any other.
H ere is how w e m ight m odel this as a form al problem .
■ Let a point p in 2-dim ensional space be given by its integer
coordinates, p = (p.x, p.y).
■ A point p is said to be dom inated by point q if p.x ≤ q.x and p.y ≤
q.y.
■ Given a set of n points, P = {p 1, p 2 , . . . , p n } in 2-space a point is
said to be m axim al if it is not dom inated by any other point in P.
105
Assignment #2
The car selection problem can be m odelled this w ay: For each car
we associate (x, y) pair w here x is the speed of the car and y is the
negation of the price. H igh y value m eans a cheap car and low y
m eans expensive car. Think of y as the m oney left in your pocket
after you have paid for the car. M axim al points correspond to the
fastest and cheapest cars.
The 2-dim ensional M axim a is thus defined as

■ Given a set of points P = {p 1, p 2 , . . . , p n } in 2-space, output the set
of m axim al points of P , i.e., those points p i such that p i is not
dom inated by any other point of P.
106
Assignment #2
107
Assignment #2
■ Algo#1
108
Assignment #2
■ Algo#2
109
110
ALGORITHMS
Lecture # 22, 23 & 24

DESIGN & ANALYSIS OF BASIC
SEARCHING ALGORITHMS
Searching
■ Searching is the process of determining
whether or not a given value exists in a data
structure or a storage media.
■ We will discuss two searching methods on 1D

(Linear) data structures:
– Linear search
– Binary search.
2
Linear Search
■ It is also called as Serial or Sequential Search.
■ The linear (or sequential) search algorithm on
an array is:
– Sequentially scan the array, comparing each array
item with the searched value.
– If a match is found; return the index of the
matched element; otherwise return NULL.
■ Note: linear search can be applied to both
sorted and unsorted arrays.
3
Linear Search
■ Step through array of records, one at a time.
■ Look for record with matching key.
■ Search stops when

– record with matching key is found
– or when search has examined all records without
success.
4
Linear Search
Linear Search ( Array A, KEY)
Step 1: Set i to L
Step 2: While i <= U do
Step 3: if A[i] = KEY then go to step 7
Step 4: Set i to i + 1
Step 5: End While
Step 6: Print “Element not found” & Exit
Step 7: Print “Element x Found at index i”
Step 8: Exit
5
Linear Search
procedure linear_search (Array, KEY)
For each item in the Array

If item == KEY
return the item's location
End If
End For
End procedure
6
Linear Search
Algorithm LinearSearch (Array A[L..U], KEY)
For i = L to U
If A[i] == KEY
return i
End If
End For
return NULL
7
Working
KEY=33
8
Working
KEY=33
9
Working
KEY=33
10
Working
KEY=33
11
Working
KEY=33
12
Working
KEY=33
13
Working
KEY=33
14
Working
KEY=33
15
Working
KEY=33
1 2 3 4 5 6 7 8 9 10
return 7
16
Analysis of Linear Search
For i = L to U
If A[i] == KEY
If n is the size of the array,
return i
End If S(n) = O(1)
End For
return NULL
17
Analysis of Linear Search
For i = L to U
If A[i] == KEY If n is the size of the array,
the loop will run not more
return i than “n” times.
End If In each iteration, it performs
End For some constant time operations.
return NULL
T(n) = n.c = O(n)
18
Best Case T(n)BestCase = O(1)
KEY=10
1 2 3 4 5 6 7 8 9 10
return 1
19
Worst Case - 1 T(n)WorstCase = O(n)
KEY=44
1 2 3 4 5 6 7 8 9 10
return 10
20
Worst Case - 2 T(n)WorstCase = O(n)
KEY=100
1 2 3 4 5 6 7 8 9 10
return NULL
21
Average Case
■ In the Average Case, the value can be somewhere in the array.
■ Average Case Time Complexity is the average of Time
Complexities of all the possible inputs.
■ Suppose we have an array of n records.

■ If we search for the first record, then it requires 1 comparison; if
the second, then 2 comparisons, and so on.
■ The average of all these searches is:
(1+2+3+4+5+6+7+8+9+…+n)/n = (n+1)/2 = O(n)
22
Advantages of Linear Search
■ The linear search is simple.
■ It is easier to understand & implement.
■ It does not require the data to be stored in any

particular order.
23
Disadvantages of Linear Search
■ It has a very poor efficiency as it takes lots of
comparisons to find a particular record in big
files.
■ It fails in very large databases especially in

real-time systems.
24
Scenario 1
■ Suppose you’re searching for a person in the
phone book. His/Her name starts with K.
■ You could start at the beginning and keep
flipping pages until you get to the Ks.
■ But you’re more likely to start at a page in the
middle, because you know the Ks are going to
be near the middle of the phone book.
25
Scenario 2
■ Suppose you’re searching for a word in a

dictionary, and it starts with O.
■ Again, you’ll start near the middle.
26
Scenario 3
■ Suppose you log on to Facebook. When you do,
Facebook has to verify that you have an account on
the site.
■ So, it needs to search for your username in its
database.
■ Let your username is LARAIB – for instance -
Facebook could start from the As and search for your
name.
2.7 billion monthly active users
■ But it makes more sense for it to begin somewhere in
the middle.
27
Binary Search
■ All these cases use the same algorithm to solve the problem:
BINARY SEARCH.
28
Binary Search
■ Binary search is a searching algorithm.
■ Its input is a sorted list of elements.
■ If an element you’re looking for is in that list, binary search

returns the position where it’s located.
■ Otherwise, binary search returns NULL.
29
Comparison with Linear Search
■ I’m thinking of a number between 1 and 100.
■ You have to try to guess my number in the fewest tries possible.

■ With every guess, I’ll tell you if your guess is too low, too high, or
correct.
30
■ Suppose you start guessing like this:
1, 2, 3, 4 ….
■ Here’s how it would go.
31
■ Suppose you start guessing like this:
1, 2, 3, 4 ….
■ Here’s how it would go.
32
■ This is the sequential search.
■ With each guess, you’re eliminating only one number.
■ If my number was 99, it could take you 99 guesses to get there!
33
■ Here’s a better technique.
■ Start with 50.
34
■ You just eliminated half the numbers!
■ Now you know that 1–50 are all too low.
35
■ Next guess: 75.
■ Too high, but again you cut down half the remaining numbers!
■ With binary search, you guess the middle number and eliminate
half the remaining numbers every time.
36
■ Next is 63 (halfway between 50 and 75).
37
■ This is binary search.
■ Here’s how many numbers you can eliminate every time.
38
■ Whatever number I’m thinking of, you can guess in a maximum of

seven guesses—because you eliminate so many numbers with
every guess!
39
■ Suppose you’re looking for a word in the

dictionary. The dictionary has 240,000
words.
■ In the worst case, how many steps do you

think each search will take?
40
■ Simple search could take 240,000 steps if the word you’re
looking for is the very last one in the book.
■ With each step of binary search, you cut the number of words
in half until you’re left with only one word.
41
Working
KEY=31
42
Working
KEY=31
43
Working
KEY=31
44
Working
KEY=31
45
Working
KEY=31
46
Working
KEY=31
return 5
47
Binary Search (Iterative Version)
Algorithm BinarySearch (Array A[L..U], KEY)
While L<=U, do
mid = (L+U)/2
If A[mid] == KEY
return mid
ElseIf A[mid] < KEY
L = mid + 1
Else
U = mid - 1
End If
End While
return NULL
48
Binary Search (Recursive Version)
Algorithm BinSearch (Array A, KEY, U, L)
If L>U
return NULL
End If
mid = (L+U)/2
If A[mid] == KEY
return mid
ElseIf A[mid] < KEY
return BinSearch(A, KEY, mid + 1, U)
Else
return BinSearch(A, KEY, L, mid - 1)
End If
49
Binary Search – Side by Side
Algorithm BinarySearch (Array A[L..U], KEY) Algorithm BinSearch (Array A, KEY, U, L)
While L<=U, do If L>U
mid = (L+U)/2 return NULL
If A[mid] == KEY End If
return mid mid = (L+U)/2
ElseIf A[mid] < KEY If A[mid] == KEY
L = mid + 1 return mid
Else ElseIf A[mid] < KEY
U = mid - 1 return BinSearch(A, KEY, mid + 1, U)
End If Else
End While return BinSearch(A, KEY, L, mid - 1)
return NULL End If
50
Space Complexity S(n)Iterative = O(1)

End If Else
return NULL End If
51
Space Complexity S(n)Recursive = O(log2n)

End If Else
return NULL End If
52
A(32)
A(0) 1 call
Space Complexity A(1)

A(2)
2 calls
3 calls A(16) c
Algorithm BinSearch (Array A, KEY, U, L) A(4) 4 calls

If L>U A(8) 5 calls A(8) c
A(16) 6 calls
return NULL
A(32) 7 calls
End If A(4) c
mid = (L+U)/2 A(n) x calls
If A[mid] == KEY A(2) c
return mid
ElseIf A[mid] < KEY A(1) c
Else A(0) c

End If c
53
A(32)
A(0) 1 call
Space Complexity A(1)

A(2)
log21+2=2 calls
log22+2=3 calls A(16) c
Algorithm BinSearch (Array A, KEY, U, L) A(4) log24+2=4 calls

If L>U A(8) log28+2=5 calls A(8) c
A(16) log216+2=6 calls
return NULL
A(32) log232+2=7 calls
End If A(4) c
mid = (L+U)/2 A(n) log2n+2=x calls
If A[mid] == KEY A(2) c
return mid
ElseIf A[mid] < KEY A(1) c
Else A(0) c

End If c
54
If each block of stack
Space Complexity consists of k memory
If L>U
space required is k(log2n+2)
return NULL
End If
mid = (L+U)/2
If A[mid] == KEY
return mid
ElseIf A[mid] < KEY
log2n+2
Else
blocks
End If
55
Space Complexity S(n)Recursive = O(log2n)
If L>U
return NULL
End If
mid = (L+U)/2
If A[mid] == KEY
return mid
ElseIf A[mid] < KEY
log2n+2
Else
blocks
End If
56
Time Complexity
If L>U T(n) = T( n/2 ) + c
return NULL T(1) = T(0) + c
End If T(0) = c
mid = (L+U)/2
If A[mid] == KEY
return mid
ElseIf A[mid] < KEY T(n) = T( n/2 ) + c
return BinSearch(A, KEY, mid + 1, U) T(1) = T(0) + c
Else T(0) = c
End If
57
T(n) =T(n/2)+c, for n>1
T(1) =T(0)+c
-------------------------------------------------------------------------
T(n) =T(n/2)+c …eq(i)
T(n/2) =T(n/2/2)+c
=T(n/22)+c …eq(ii)
T(n/22) =T(n/22/2)+c
=T(n/23)+c …eq(iii) 58
Putting eq(ii) in eq(i): T(n) =T(n/2)+c …eq(i)
T(n/2) =T(n/22)+c …eq(ii)
T(n) =T(n/2)+c T(n/22) =T(n/23)+c …eq(iii)
=T(n/22)+c+c
=T(n/22)+2c …eq(iv)
T(n) = T(n/23)+c+2c
=T(n/23)+3c
Suppose we reach T(1) after k calls:
T(n) =T(n/2k)+kc ..eq(v)
59
For what value of k will T(n/2k) become T(1).
n/2k=1  n=2k  k=log2n
T(n) = T(n/2k)+kc
= T(1)+kc
= T(0) + c + kc
= c+c+kc
= 2c+clog2n
= O(log2n)
60
T(n) =T(n/2)+c, for n>1
T(1) =T(0)+c
-------------------------------------------------------------------------
Note that here, at each step, only 1 branch will be drawn;
decreasing the input size by a factor of 2 at each step.
And a total of c work will be done at each branching.
61
A(n) c
A(n/2) c
A(n/22) c
For what value of 2k will T(n/2k) become T(1). A(n/23) c

n/2k=1  n=2k  k=log2n
T(n) = c(k+2)
= c(log2n+2)
A(n/2k) = T(1) c
= O(log2n)
A(0) c
62
Master Theorem
T(n) =T(n/2)+c, for n>1

Here, a=1, b=2, c=0, and d=0
Now, bc=20=1
Here a=bc, so second possibility is our selection.
As d=0, so solution will be:
T(n) = (nlog21log0+1n) = (n0log1n) = (log n)
63
Practice Questions
64
ONLINE QUIZ #1
■ Time Slot: 13:30 pm to 16:30 pm
■ Duration: 50 minutes
65
66
ALGORITHMS
Lecture # 25
INTRODUCTION TO DesIgN Of
sORTINg ALgORITHMs
Sorting
■ In this lesson and in this series of lessons we are going to
talk about sorting algorithms.
■ Sorting as a concept is deeply embedded in a lot of things

that we do.
■ It's quite often that we like to arrange things or data in

certain order.
2
Sorting
■ Sometimes to improve the readability of that data, at others to
be able to search or extract some information quickly out of
that data.
■ For example:
We keep currency notes sorted.
3
Sorting
■ Now, have a look at this, when we go to an online marketplace
such as OLX.
■ The site gives us options of sorting the items.
4
Sorting
■ So, sorting is a really helpful feature here and there are so many
places where we like to keep our data sorted.
■ Be it the language dictionary where we want to keep the words
sorted so that searching a word in the dictionary is easy.
5
Sorting
■ If we want to define sorting formally, then sorting is arranging
the elements in a list or collection in increasing or decreasing
order of some property.
■ The list should be homogeneous, that is all the elements in the

list should be of same type.
■ To study sorting algorithms, most of the times we use a list of

integers, and typically we sort the list of integers in increasing
order of value.
6
Sorting
■ For example, if we have the following list:
12 21 6 8 5
■ Then sorting in increasing order of values will mean rearranging

the above values as:
5 6 8 12 21
7
Sorting
12 21 6 8 5
■ Similarly, sorting in decreasing order of values will result as

following permutation of the above list:
21 12 8 6 5
8
Sorting
12 21 6 8 5
■ But we can sort on any property of data, it is not just the

magnitude of values, like increasing number of factors.
9
Sorting
12 21 6 8 5
1,2,3,4, 1,3,7,21 1,2,3,6 1,2,4,8 1,5
6,12
■ But we can sort on any property of data, it is not just the

magnitude of values, like increasing number of factors.
5 21 6 8 12
10
Sorting
■ A sorted list is the permutation of the original list, When we sort a
list, we just rearrange the elements.
■ Now this was a list of integers. We may have a list of any data
type. We may want to sort a list of strings or words in
lexicographical order, the order in which they will occur in
dictionary.
11
Sorting
“fork” “knife” “screen” “mouse” “key”
■ A list of strings like above in lexicographical order will be arranged

as below:
“fork” “key” “knife” “mouse” “screen”
12
Sorting
■ And we may have a list of complex data type as well.
■ A car object in the OLX website, that we had seen earlier, was a
complex type.
■ Cars will have many properties, like the following, and the list can
be sorted on any of these properties.
– Price
– Relevancy
– Date of inclusion in the database
– Etc.
13
Sorting
■ Sorted data is good, not just for presentation or manual retrieval
of information, but even when we are using computational power
of machines, sorted data is really helpful.
■ If a list is stored in computer's memory as unsorted, then to
search something in this list, we will have to run Linear Search.
(We cannot do binary search on unsorted data)
■ In linear search, we start with the first element of the list and
keep scanning the whole list, until we get the element that we are
looking for, so, in the worst case, like, when an element will not be
there in the list, we will compare it with all the elements in the list,
we will scan it with whole list.
14
Sorting
■ So, if there are n elements in the list, we will make n comparisons
in the worst case.
■ Just, think about the kind and amount of data, modern day
computers deal with.
■ What if this n is very very large! If we take n=2128 items and
imagine that 1 comparison takes 1 millisecond.
■ Then how much time is required to search a value in the worst
case?
Time = 2128 milliseconds

15
Sorting
Time = 10790283070806000000000000000 years
■ But, if data was sorted, by using binary search, it would take only
128 milliseconds.
Time = log22128 = 128 milliseconds
Time = 0.128 second
16
Sorting
■ Now, you can imagine how important sorting as a problem is.
■ Sorting as a problem is well studied and a great deal of research

has gone into devising efficient algorithms for sorting.
■ In this series of lessons we are going to study, analyze, and

compare these sorting algorithms.
17
Sorting
■ Some of the sorting algorithms that we will be talking about, that
we will be analyzing about are:
18
Sorting
■ We have so many algorithms for sorting that have been designed over
a period of time.
■ We often classify sorting algorithms based on some parameters.
■ The first parameter that we want to classify upon is time complexity

which is the measure of rate of growth of time taken by an algorithm
with respect to input size.
■ Some algorithms will be relatively faster than others.

19
Sorting w.r.t Time Complexity
Sorting Algorithms
Polynomial Time n.Logrithmic Time Linear Time
Bubble Sort Quick Sort Counting Sort

Selection Sort Merge Sort Radix Sort
Insertion Sort Heap Sort Bucket Sort
20
Sorting
■ The second parameter that we use for classification is space
complexity or memory usage.
■ Some sorting algorithms are in-place, they use constant i.e. O(1)
or logarithmic i.e. O(log n) amount of extra memory to rearrange
the elements in the list.
■ While some sorting algorithms use extra memory to temporarily

store data and the memory usage grows with input size. These
are called out-place sorting algorithms. Here, the S(n) > (log n)
21
Sorting w.r.t Space Complexity
Sorting Algorithms
In-Place Out-Place
Bubble Sort Merge Sort

Selection Sort Counting Sort
Insertion Sort
Quick Sort
22
Sorting
■ The third parameter that we talk about is stability.
■ A stable sorting algorithm guarantees that in the case of duplicate
values, the relative order of elements is preserved.
Unsorted List: 12 21 6 8 6
After stable sorting: 6 6 8 12 21
After unstable sorting: 6 6 8 12 21

23
Sorting w.r.t Stability
Sorting Algorithms
Stable Unstable
Bubble Sort Quick Sort

Merge Sort Heap Sort
Insertion Sort Selection Sort
24
Sorting
■ The next parameter of classification is whether a sort is internal
sort or external sort.
■ When all the records that need to be sorted in the main memory
or RAM, then such sort is internal sort.
■ And if the records are on auxiliary storage like disk and tapes,
quite often because it's not possible to get all of them in the main
memory in one go, then we call such a sort external sort.
25
Sorting w.r.t Location of Values
Sorting Algorithms
Internal External
Bubble Sort Merge Sort (Hybrid version)

Quick Sort Distribution Sort
Insertion Sort
26
Sorting
■ The next parameter upon which we may classify sorting algorithms
is whether algorithm is recursive or non recursive in nature.
27
Sorting w.r.t Design
Sorting Algorithms
Iterative Recursive
Bubble Sort Merge Sort

Selection Sort Quick Sort
Insertion Sort
28
Sorting
■ There is one more parameter upon which we may classify sorting
algorithms, and it is whether algorithm compares the values or
does sorting in some other way.
■ A comparison sort algorithm sorts items by comparing values

between each other. It can be applied to any input.
■ A non-comparison sort algorithm uses the internal character of

the values to be sorted. It can only be applied to some particular
cases, and requires particular values.
29
Sorting w.r.t Comparison
Sorting Algorithms
Comparison Sorting Non-Comparison Sorting
Bubble Sort Counting Sort

Selection Sort Radix Sort
Insertion Sort Bucket Sort
30
31
ALGORITHMS
Lecture # 26 & 27
DESIGN OF SElEctION, INSErtION,
BuBBlE & ShEll SOrt
Basic Sorting Block
3
■ Before starting to learn sorting
algorithms, lets start with a very
basic sorting block. 4 4
■ It takes two numbers as an input

and tells which one is smaller and 3
vice versa.
2
Basic Sorting Block
num1
If num1 <= num2
Pop_Down num1
Pop_Right num2 num2 larger
Else
Pop_Down num2
Pop_Right num1
smaller
We will combine these sorting block in a triangular way and try to sort a list.
3
5
Constructing Our
4 5 1st Sorting Algorithm
4
After each step, we
3 4 5 get/select the minimum
element of the (remaining)
3 4 list.
1 3 4 5
1 3 4
2 2 3 4
1 2 3 4 5 4
1 st Sorting Algorithm
L U
Algorithm ColumnWiseSorting(List[L…U]) 12 21 6 8 5
For i = L to U-1
Min=i
For j = i+1 to U
If List[ j ] < List[Min]
Min=j
Swap List[i]↔List[Min]
5
L U
For i = L to U-1 i
Min=i
For j = i+1 to U
Min=j
6
L U
For i = L to U-1 i
Min
Min=i
For j = i+1 to U
Min=j
7
L U
For i = L to U-1 i j
Min
Min=i
For j = i+1 to U
Min=j
8
L U
Min
Min=i
For j = i+1 to U
Min=j
9
L U
Min
Min=i
For j = i+1 to U
Min=j
10
L U
Min
Min=i
For j = i+1 to U
Min=j
11
L U
Min
Min=i
For j = i+1 to U
Min=j
12
L U
Min
Min=i
For j = i+1 to U
Min=j
13
L U
Min
Min=i
For j = i+1 to U
Min=j
14
L U
Min
Min=i
For j = i+1 to U
Min=j
15
L U
Min
Min=i
For j = i+1 to U
Min=j
16
L U
Min
Min=i
For j = i+1 to U
Min=j
17
L U
Min
Min=i
For j = i+1 to U
Min=j
18
L U
Min
Min=i
For j = i+1 to U
Min=j
19
L U
Min
Min=i
For j = i+1 to U
Min=j
20
L U
Min
Min=i
For j = i+1 to U
Min=j
21
L U
Min
Min=i
For j = i+1 to U
Min=j
22
L U
Min
Min=i
For j = i+1 to U
Min=j
23
L U
Min
Min=i
For j = i+1 to U
Min=j
24
L U
Min
Min=i
For j = i+1 to U
Min=j
25
L U
Min
Min=i
For j = i+1 to U
Min=j
26
L U
Min
Min=i
For j = i+1 to U
Min=j
27
L U
Min
Min=i
For j = i+1 to U
Min=j
28
L U
Min
Min=i
For j = i+1 to U
Min=j
29
L U
Min
Min=i
For j = i+1 to U
Min=j
30
L U
Min
Min=i
For j = i+1 to U
Min=j
31
L U
Min
Min=i
For j = i+1 to U
Min=j
32
L U
Min
Min=i
For j = i+1 to U
Min=j
33
L U
Min
Min=i
For j = i+1 to U
Min=j
34
L U
Min
Min=i
For j = i+1 to U
Min=j
35
L U
Min
Min=i
For j = i+1 to U
Min=j
36
L U
Min
Min=i
For j = i+1 to U
Min=j
37
L U
Min
Min=i
For j = i+1 to U
Min=j
38
L U
Min
Min=i
For j = i+1 to U
Min=j
39
L U
Min
Min=i
For j = i+1 to U
Min=j
40
L U
Min
Min=i
For j = i+1 to U
Min=j
41
L U
Min
Min=i
For j = i+1 to U
Min=j
42
L U
Min
Min=i
For j = i+1 to U
Min=j
43
L U
Min
Min=i
For j = i+1 to U
Min=j
44
L U
Min
Min=i
For j = i+1 to U
Min=j
45
L U
Min
Min=i
For j = i+1 to U
Min=j
46
L U
Min
Min=i
For j = i+1 to U
Min=j
47
L U
Min
Min=i
For j = i+1 to U
Min=j
48
5
5
Constructing Our
4 2nd Sorting Algorithm
4 5
After each step, we insert
3 4 the next element in its
correct position in the
3 4 5 sorted sub-list.
1 3 4
1 3 4 5
2 2 3 4
1 2 3 4 5 49
2 nd Sorting Algorithm
L U
Algorithm RowWiseSorting(List[L…U]) 12 21 6 8 5
For i = L+1 to U
value=List[i]
For j = i-1 to L
If List[ j ] < value value
break
List[ j+1 ] = List[ j ]
List[ j+1 ] = value
50
L U
For i = L+1 to U i
value=List[i]
For j = i-1 to L
break
List[ j+1 ] = value
51
L U
For i = L+1 to U i
value=List[i]
For j = i-1 to L
break 21

List[ j+1 ] = value
52
L U
For i = L+1 to U j i
value=List[i]
For j = i-1 to L
break 21

List[ j+1 ] = value
53
L U
value=List[i]
For j = i-1 to L
break 21

List[ j+1 ] = value
54
L U
value=List[i]
For j = i-1 to L
break 21

List[ j+1 ] = value
55
L U
value=List[i]
For j = i-1 to L
break 21

List[ j+1 ] = value
56
L U
value=List[i]
For j = i-1 to L
break 21

List[ j+1 ] = value
57
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
58
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
59
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
60
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
61
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
62
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
63
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
64
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
65
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
66
L U
value=List[i]
For j = i-1 to L
break 6

List[ j+1 ] = value
67
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
68
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
69
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
70
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
71
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
72
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
73
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
74
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
75
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
76
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
77
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
78
L U
value=List[i]
For j = i-1 to L
break 8

List[ j+1 ] = value
79
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
80
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
81
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
82
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
83
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
84
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
85
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
86
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
87
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
88
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
89
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
90
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
91
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
92
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
93
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
94
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
95
L U
value=List[i]
For j = i-1 to L
break 5

List[ j+1 ] = value
96
Comparing Both Algorithms
After each step, we get / SELECT After each step, we INSERT
the minimum element of the the next element in its correct
(remaining) list. position in the sorted sub-list.
ColumnWiseSorting(List[L…U]) RowWiseSorting(List[L…U])
For i = L to U-1 For i = L+1 to U
Min=i value=List[i]
For j = i+1 to U For j = i-1 to L
If List[ j ] < List[Min] If List[ j ] < value
Min=j break
Swap List[i]↔List[Min] List[ j+1 ] = List[ j ]
List[ j+1 ] = value
97
Comparing Both Algorithms
SELECTION SORT INSERTION SORT
ColumnWiseSorting(List[L…U]) RowWiseSorting(List[L…U])
For i = L to U-1 For i = L+1 to U
Min=i value=List[i]
For j = i+1 to U For j = i-1 to L
If List[ j ] < List[Min] If List[ j ] < value
Min=j break
Swap List[i]↔List[Min] List[ j+1 ] = List[ j ]
List[ j+1 ] = value
98
Another Basic Sorting Block
■ Before starting to learn out third 4 3

sorting algorithm, lets have a look at
another very basic sorting block.
■ It also takes two numbers in a

circle/bubble and tells which one is 3 4
smaller and vice versa.
99
3 rd Sorting Algorithm : Bubble Sort
Algorithm BubbleSort(List[L…U])
For i = U-1 to L
For j = L to i
If List[ j ] > List[ j+1 ]
Swap List[ j ] = List[ j+1 ]
L U
12 21 6 8 5
100
For i = U-1 to L
For j = L to i
L U
12 21 6 8 5
i 101
For i = U-1 to L
For j = L to i
L U
12 21 6 8 5
j
i 102
For i = U-1 to L
For j = L to i
L U
12 21 6 8 5
j
i 103
For i = U-1 to L
For j = L to i
L U
12 21 6 8 5
j
i 104
For i = U-1 to L
For j = L to i
L U
12 21 6 8 5
j
i 105
For i = U-1 to L
For j = L to i
L U
12 6 21 8 5
j
i 106
For i = U-1 to L
For j = L to i
L U
12 6 21 8 5
j
i 107
For i = U-1 to L
For j = L to i
L U
12 6 21 8 5
j
i 108
For i = U-1 to L
For j = L to i
L U
12 6 8 21 5
j
i 109
For i = U-1 to L
For j = L to i
L U
12 6 8 21 5
j
i 110
For i = U-1 to L
For j = L to i
L U
12 6 8 21 5
j
i 111
For i = U-1 to L
For j = L to i
L U
12 6 8 5 21
j
i 112
For i = U-1 to L
For j = L to i
L U
12 6 8 5 21
j
i 113
For i = U-1 to L
For j = L to i
L U
12 6 8 5 21
j
i 114
For i = U-1 to L
For j = L to i
L U
12 6 8 5 21
j
i 115
For i = U-1 to L
For j = L to i
L U
12 6 8 5 21
j
i 116
For i = U-1 to L
For j = L to i
L U
6 12 8 5 21
j
i 117
For i = U-1 to L
For j = L to i
L U
6 12 8 5 21
j
i 118
For i = U-1 to L
For j = L to i
L U
6 12 8 5 21
j
i 119
For i = U-1 to L
For j = L to i
L U
6 8 12 5 21
j
i 120
For i = U-1 to L
For j = L to i
L U
6 8 12 5 21
j
i 121
For i = U-1 to L
For j = L to i
L U
6 8 12 5 21
j
i 122
For i = U-1 to L
For j = L to i
L U
6 8 5 12 21
j
i 123
For i = U-1 to L
For j = L to i
L U
6 8 5 12 21
j
i 124
For i = U-1 to L
For j = L to i
L U
6 8 5 12 21
j
i 125
For i = U-1 to L
For j = L to i
L U
6 8 5 12 21
j
i 126
For i = U-1 to L
For j = L to i
L U
6 8 5 12 21
j
i 127
For i = U-1 to L
For j = L to i
L U
6 8 5 12 21
j
i 128
For i = U-1 to L
For j = L to i
L U
6 8 5 12 21
j
i 129
For i = U-1 to L
For j = L to i
L U
6 5 8 12 21
j
i 130
For i = U-1 to L
For j = L to i
L U
6 5 8 12 21
j
i 131
For i = U-1 to L
For j = L to i
L U
6 5 8 12 21
j
i 132
For i = U-1 to L
For j = L to i
L U
6 5 8 12 21
j
i 133
For i = U-1 to L
For j = L to i
L U
6 5 8 12 21
j
i 134
For i = U-1 to L
For j = L to i
L U
5 6 8 12 21
j
i 135
For i = U-1 to L
For j = L to i
L U
5 6 8 12 21
j
i 136
For i = U-1 to L
For j = L to i
L U
5 6 8 12 21
j
i 137
For i = U-1 to L
For j = L to i
L U
5 6 8 12 21
j
i 138
Shell Sort
■ In the insertion sort, we took an element out and we
compared it to the elements before it.
■ The elements before it was a sorted sub-list.
■ If the value at the far right was the smallest one, it took
about (n-1) shifts to place it in the correct position.
■ The idea behind the shell sort is to avoid such large shifts.
139
Shell Sort
Algorithm ShellSort(List[L…U])
For gap = U/2 to 1: gap=gap/2
For j = gap+1 to U
For i = j – gap to L: i = i - gap
If List[ i+gap ] > List[ i ]
break
Else
Swap List[ i+gap ] ↔ List[ i ]
140
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
For j = gap+1 to U
break
Else
List[ i+gap ] ↔ List[ i ]
141
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
For j = gap+1 to U
break
Else
142
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
5
For j = gap+1 to U
break
Else
143
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
5
For j = gap+1 to U
break
Else
144
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
5
For j = gap+1 to U
break
Else
145
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
6
For j = gap+1 to U
break
Else
146
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
6
For j = gap+1 to U
break
Else
147
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
6
For j = gap+1 to U
break
Else
148
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
6
For j = gap+1 to U
break
Else
149
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
150
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
151
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
152
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
153
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
8
For j = gap+1 to U
break
Else
154
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
8
For j = gap+1 to U
break
Else
155
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
8
For j = gap+1 to U
break
Else
156
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
8
For j = gap+1 to U
break
Else
157
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
9
For j = gap+1 to U
break
Else
158
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
9
For j = gap+1 to U
break
Else
159
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
9
For j = gap+1 to U
break
Else
160
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
9
For j = gap+1 to U
break
Else
161
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 4
2 7 9 5 23 29 15 9 31
j
i i+gap
10
For j = gap+1 to U
break
Else
162
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 7 9 5 23 29 15 9 31
j
i i+gap
10
For j = gap+1 to U
break
Else
163
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 7 9 5 23 29 15 9 31
j
i i+gap
3
For j = gap+1 to U
break
Else
164
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 7 9 5 23 29 15 9 31
j
i i+gap
3
For j = gap+1 to U
break
Else
165
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 7 9 5 23 29 15 9 31
j
i i+gap
3
For j = gap+1 to U
break
Else
166
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 7 9 5 23 29 15 9 31
j
i i+gap
3
For j = gap+1 to U
break
Else
167
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 7 9 5 23 29 15 9 31
j
i i+gap
4
For j = gap+1 to U
break
Else
168
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 7 9 5 23 29 15 9 31
j
i i+gap
4
For j = gap+1 to U
break
Else
169
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 7 9 5 23 29 15 9 31
j
i i+gap
4
For j = gap+1 to U
break
Else
170
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
4
For j = gap+1 to U
break
Else
171
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
4
For j = gap+1 to U
break
Else
172
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
5
For j = gap+1 to U
break
Else
173
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
5
For j = gap+1 to U
break
Else
174
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
5
For j = gap+1 to U
break
Else
175
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
5
For j = gap+1 to U
break
Else
176
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
6
For j = gap+1 to U
break
Else
177
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
6
For j = gap+1 to U
break
Else
178
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
6
For j = gap+1 to U
break
Else
179
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
6
For j = gap+1 to U
break
Else
180
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
181
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
182
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 23 29 15 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
183
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 29 23 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
184
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 29 23 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
185
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 29 23 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
186
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 29 23 9 31
j
i i+gap
7
For j = gap+1 to U
break
Else
187
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 29 23 9 31
j
i i+gap
8
For j = gap+1 to U
break
Else
188
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 29 23 9 31
j
i i+gap
8
For j = gap+1 to U
break
Else
189
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 29 23 9 31
j
i i+gap
8
For j = gap+1 to U
break
Else
190
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 9 23 29 31
j
i i+gap
8
For j = gap+1 to U
break
Else
191
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 9 23 29 31
j
i i+gap
8
For j = gap+1 to U
break
Else
192
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 9 23 29 31
j
i i+gap
8
For j = gap+1 to U
break
Else
193
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 9 23 29 31
j
i i+gap
8
For j = gap+1 to U
break
Else
194
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 9 23 29 31
j
i i+gap
9
For j = gap+1 to U
break
Else
195
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 9 23 29 31
j
i i+gap
9
For j = gap+1 to U
break
Else
196
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 9 23 29 31
j
i i+gap
9
For j = gap+1 to U
break
Else
197
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 9 23 29 31
j
i i+gap
9
For j = gap+1 to U
break
Else
198
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 2
2 5 9 7 15 9 23 29 31
j
i i+gap
10
For j = gap+1 to U
break
Else
199
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
10
For j = gap+1 to U
break
Else
200
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
2
For j = gap+1 to U
break
Else
201
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
2
For j = gap+1 to U
break
Else
202
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
2
For j = gap+1 to U
break
Else
203
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
2
For j = gap+1 to U
break
Else
204
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
3
For j = gap+1 to U
break
Else
205
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
3
For j = gap+1 to U
break
Else
206
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
3
For j = gap+1 to U
break
Else
207
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
3
For j = gap+1 to U
break
Else
208
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
4
For j = gap+1 to U
break
Else
209
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
4
For j = gap+1 to U
break
Else
210
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 9 7 15 9 23 29 31
j
i i+gap
4
For j = gap+1 to U
break
Else
211
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
4
For j = gap+1 to U
break
Else
212
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
4
For j = gap+1 to U
break
Else
213
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
4
For j = gap+1 to U
break
Else
214
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
4
For j = gap+1 to U
break
Else
215
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
5
For j = gap+1 to U
break
Else
216
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
5
For j = gap+1 to U
break
Else
217
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
5
For j = gap+1 to U
break
Else
218
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
5
For j = gap+1 to U
break
Else
219
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
6
For j = gap+1 to U
break
Else
220
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
6
For j = gap+1 to U
break
Else
221
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 15 9 23 29 31
j
i i+gap
6
For j = gap+1 to U
break
Else
222
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
6
For j = gap+1 to U
break
Else
223
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
6
For j = gap+1 to U
break
Else
224
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
6
For j = gap+1 to U
break
Else
225
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
6
For j = gap+1 to U
break
Else
226
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
7
For j = gap+1 to U
break
Else
227
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
7
For j = gap+1 to U
break
Else
228
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
7
For j = gap+1 to U
break
Else
229
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
8
For j = gap+1 to U
break
Else
230
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
8
For j = gap+1 to U
break
Else
231
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
8
For j = gap+1 to U
break
Else
232
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
9
For j = gap+1 to U
break
Else
233
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
9
For j = gap+1 to U
break
Else
234
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
9
For j = gap+1 to U
break
Else
235
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 1
2 5 7 9 9 15 23 29 31
j
i i+gap
10
For j = gap+1 to U
break
Else
236
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 0
2 5 7 9 9 15 23 29 31
j
i i+gap
10
For j = gap+1 to U
break
Else
237
gap L U
1 2 3 4 5 6 7 8 9
Shell Sort 0
2 5 7 9 9 15 23 29 31
j
i i+gap
10
For j = gap+1 to U
break
Else
238
Time & Space Complexity of Shell Sort
S(n) = O(1)
T(n) = O(n2) Worst Case

T(n) = O(n.logn) Best Case
T(n) = O(n.logn) Average Case
239
Assignment #3
■ Find the Time & Space Complexity of:
– Bubble Sort
– Selection Sort
– Insertion Sort
■ Give Best, Average & Worst Case for

Time Complexity
240
Assignment #3
■ Instructions:
– Single .pdf file to be uploaded on LMS.
– Groups are allowed but not mandatory.
– Maximum group size is 5.
■ Deadline: Sunday, 07 January, 2021; before 23:55 HRS
241
242
ALGORITHMS
Lecture # 29 & 30
Quick Sort, Merge Sort,
Linear tiMe Sorting
(counting Sort, raDiX Sort)
Sorting Algorithms

Shell Sort 2
Sorting Algorithms

Shell Sort 3
Sorting Algorithms

Shell Sort 4
Divide & Conquer : Merge Sort
5
Sorting While Merging
6
7
8
9
Merge Sort
■ Merge sort is a sorting technique based on divide and

conquer technique.
■ Merge sort first divides the array into equal halves and then
combines them in a sorted manner.
– MERGE procedure
10
MERGE Function
Algorithm Merge(A[],p,q,r)
New Array Temp[1…r-p+1]
i=p,j=q+1
for k=1 to (r-p+1)
if i>q, Temp[k]=A[j++]
elseif j>r, Temp[k]=A[i++]
elseif A[i] <= A[j], Temp[k]=A[i++]
else Temp[k]=A[j++]
Copy Temp[1…k] to A[p…r]
11
MERGE Function
i=p,j=q+1
p q for k=1 to (r-p+1)

1 2 9 20 25 elseif j>r, Temp[k]=A[i++]
10 11 12 13 14
else Temp[k]=A[j++]
q+1 r Copy Temp[1…k] to A[p…r]
3 12 27 28
15 16 17 18
p q q+1 r
A 1 2 3 20 25 3 12 27 28
10 11 12 13 14 15 16 17 18 12
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
15 16 17 18
13
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
15 16 17 18
Temp
1 2 3 4 5 6 7 8 9
14
MERGE Function
i=p,j=q+1

i elseif A[i] <= A[j], Temp[k]=A[i++]
10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp
1 2 3 4 5 6 7 8 9
15
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp k
1 2 3 4 5 6 7 8 9
16
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp k
1 2 3 4 5 6 7 8 9
17
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp k
1 2 3 4 5 6 7 8 9
18
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1
k
1 2 3 4 5 6 7 8 9
19
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1
k
1 2 3 4 5 6 7 8 9
20
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1
k
1 2 3 4 5 6 7 8 9
21
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1
k
1 2 3 4 5 6 7 8 9
22
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2
k
1 2 3 4 5 6 7 8 9
23
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2
k
1 2 3 4 5 6 7 8 9
24
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2
k
1 2 3 4 5 6 7 8 9
25
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2
k
1 2 3 4 5 6 7 8 9
26
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3
k
1 2 3 4 5 6 7 8 9
27
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3
k
1 2 3 4 5 6 7 8 9
28
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3
k
1 2 3 4 5 6 7 8 9
29
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9
k
1 2 3 4 5 6 7 8 9
30
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9
k
1 2 3 4 5 6 7 8 9
31
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9
k
1 2 3 4 5 6 7 8 9
32
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9
k
1 2 3 4 5 6 7 8 9
33
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12
k
1 2 3 4 5 6 7 8 9
34
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12
k
1 2 3 4 5 6 7 8 9
35
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12
k
1 2 3 4 5 6 7 8 9
36
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20
k
1 2 3 4 5 6 7 8 9
37
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20
k
1 2 3 4 5 6 7 8 9
38
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20
k
1 2 3 4 5 6 7 8 9
39
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20 25
k
1 2 3 4 5 6 7 8 9
40
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20 25
k
1 2 3 4 5 6 7 8 9
41
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20 25 27
k
1 2 3 4 5 6 7 8 9
42
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20 25 27
k
1 2 3 4 5 6 7 8 9
43
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20 25 27 28
k
1 2 3 4 5 6 7 8 9
44
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
3 12 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20 25 27 28
k
1 2 3 4 5 6 7 8 9
45
MERGE Function
i=p,j=q+1

10 11 12 13 14
A q+1 r
else Temp[k]=A[j++]
20 25 27 28
j
15 16 17 18
Temp 1 2 3 9 12 20 25 27 28
k
1 2 3 4 5 6 7 8 9
46
Analysis of “MERGE”
S(n) = n+c = O(n)
i=p,j=q+1 T(n) = c+n+n(c)
for k=1 to (r-p+1)
= O(n)
elseif j>r, Temp[k]=A[i++]
else Temp[k]=A[j++]
Merge Sort: Divide & Conquer
48
MERGE SORT
Algorithm MergeSort(A[],p,r)
if p<r
q=(p+r)/2
MergeSort(A,p,q)
MergeSort(A,q+1,r)
Merge(A,p,q,r)
49
Merge Sort: Divide & Conquer
Sorted
MS(1,9)
MS(1,5)
Sorted + MS(6,9)
Sorted
MS(1,3)
Sorted + MS(4,5)
Sorted MS(6,7)
Sorted + MS(8,9)
Sorted
MS(1,2)
Sorted + MS
(3,3)
MS
(4,4) + MS
(5,5)
MS
(6,6) + MS
(7,7)
MS
(8,8) + MS
(9,9)
MS
(1,1) + MS
(2,2) 50
Analysis of MERGE SORT
Algorithm MergeSort(A[],p,r)
S(n) = O(n) + O(log2n)
if p<r = O(n)
q=(p+r)/2
MergeSort(A,p,q) T(n) = 2T(n/2)+O(n)
MergeSort(A,q+1,r)
T(1) = c = O(1)
Merge(A,p,q,r) By Master’s Theorem
T(n) = O(n.log2n)
51
Quick Sort
■ Quick sort is also based on divide and conquer technique.
■ Quick sort takes a value from array as a pivot element and

divides the array in such a way that all the elements to the
left of pivot are smaller that the value of this pivot element
and vice versa.
■ Hence, we place the pivot element in its correct sorted
position and we get two sub-lists; one on the left and other on
the left side of the pivot element.
– PLACE&DIVIDE procedure
52
Divide & Conquer : Quick Sort
53
PLACE&DIVIDE Function
Algorithm Place&Divide(A[],p,r)
x=A[r]
i=p-1
for j=p to (r-1)
if A[j]<=x
i++
if(i≠j), A[i]↔A[j]
A[i+1]↔A[r]
return i+1 as index of division
54
PLACE&DIVIDE Function x=A[r]

i=p-1
for j=p to (r-1)
if A[j]<=x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
55

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
5 if(i≠j), A[i]↔A[j]
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
56

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i
57

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
58

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
59

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
ij
60

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
ij
61

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
62

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
63

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
ij
64

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
ij
65

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
66

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
67

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
ij
68

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
ij
69

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
70

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
71

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
72

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
73

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
74

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
75

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 8 7 4 6 9 5
10 11 12 13 14 15 16 17 18
i j
76

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 4 7 8 6 9 5
10 11 12 13 14 15 16 17 18
i j
77

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 4 7 8 6 9 5
10 11 12 13 14 15 16 17 18
i j
78

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 4 7 8 6 9 5
10 11 12 13 14 15 16 17 18
i j
79

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 4 7 8 6 9 5
10 11 12 13 14 15 16 17 18
i j
80

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 4 7 8 6 9 5
10 11 12 13 14 15 16 17 18
i j
81

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 4 7 8 6 9 5
10 11 12 13 14 15 16 17 18
i j
82

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 4 5 8 6 9 7
10 11 12 13 14 15 16 17 18
i j
83

i=p-1
for j=p to (r-1)
if A[j]<=x
x
i++
A[i+1]↔A[r]
p r
A 2 1 3 4 5 8 6 9 7
10 11 12 13 14 15 16 17 18
i j
84
QUICK SORT
Algorithm QuickSort(A[],p,r)
if p<r
q=Place&Divide(A,p,r)
QuickSort(A,p,q-1)
QuickSort(A,q+1,r)
p q-1 q q+1 r
A 2 1 3 4 5 8 6 9 7
85
Analysis of Place&Divide
x=A[r] S(n) = c
i=p-1
for j=p to (r-1)
= O(1)
if A[j]<=x
i++ T(n) = c+(n-1)c
= O(n)
A[i+1]↔A[r]
86
Analysis of QuickSort
Best Case
Algorithm QuickSort(A[],p,r) S(n) = O(1) + O(log2n)
if p<r = O(log2n)
QuickSort(A,p,q-1) T(n) = 2T(n/2) + O(n)
QuickSort(A,q+1,r) T(1) = c = O(1)
By Master’s Theorem
T(n) = O(n.log2n)
87
Analysis of QuickSort
Worst Case
Algorithm QuickSort(A[],p,r) S(n) = O(1) + O(n)
if p<r = O(n)
QuickSort(A,p,q-1) T(n) = T(n-1) + O(n)
QuickSort(A,q+1,r) T(1) = c = O(1)
By Back-Substitution Method
T(n) = O(n2)
88
Practice Question
■ Run Quick Sort Algorithm on the following Lists.
1 2 3 4 5 6 7 8 9 10
5 5 5 5 5 5 5 5 5 5
■ Remember that both the lists are already sorted.
89
Linear Time Sorting
■ The lower bound for Comparison based sorting algorithm (Merge
Sort, Heap Sort, Quick-Sort, etc.) is Ω(n.log n), i.e., they cannot do
better than n.log n.
■ However, we do have some non-comparison based algorithms,
that run faster than O(n log n) time but they require special
assumptions about the input sequence to be sort.
■ Examples of sorting algorithms that run in linear time are:
– Counting sort,
– Radix sort and
– Bucket sort
90
Counting Sort
■ Counting sort is a sorting technique that sorts the elements of an
array by counting the number of occurrences of each unique
element in the array.
■ It is very efficient when the range of values is very small.
■ For larger range of values, it is not so efficient.
91
Counting Sort
Algorithm CountingSort(A[L…U])
Range=Max(A[])-Min(A[])+1
New Array Temp[1…Range]
Adjust=1-Min(A[])
for i=L to U
Temp[A[i]+Adjust]++
i=L
for j=1 to Range
for k=1 to Temp[j]
A[i++]=j-Adjust
92
Counting Sort New Array Temp[1…Range]

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
for k=1 to Temp[j]
A[i++]=j-Adjust
93

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
for k=1 to Temp[j]
A[i++]=j-Adjust
Range
5
94

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Range
Temp 0 0 0 0 0
5
95

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 0 0 0 0 0
2 5
96

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 0 0 0 0 0
2 5
97

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 0 0 1 0 0
2 5
98

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 0 0 1 0 0
2 5
99

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 0 1 1 0 0
2 5
100

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 0 1 1 0 0
2 5
101

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 0 1 1 0 1
2 5
102

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 0 1 1 0 1
2 5
103

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 0 1
2 5
104

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 0 1
2 5
105

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 0 2
2 5
106

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 0 2
2 5
107

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
108

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
109

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
110

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
111

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
112

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
113

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
114

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
115

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
116

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
117

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 3 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
118

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 1 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
119

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 1 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
120

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 1 ―1 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
121

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 1 2 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
122

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 1 2 3 2
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
123

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 1 2 3 3
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
124

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 1 2 3 3
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
125

Adjust=1-Min(A[])
L U for i=L to U
A
Temp[A[i]+Adjust]++
―1 0 1 2 3 3
i=L
for j=1 to Range
i
for k=1 to Temp[j]
A[i++]=j-Adjust
1 2 3 4 5
Adjust Range
Temp 1 1 1 1 2
2 5
j
126
Analysis of Counting Sort
S(n) = k + c
New Array Temp[1…Range] = O(k)
Adjust=1-Min(A[])
for i=L to U
Temp[A[i]+Adjust]++
T(n) = c+nc+(k+n)+n
i=L = O(n+k)
for j=1 to Range
for k=1 to Temp[j]
A[i++]=j-Adjust
127
Radix Sort
■ Counting sort is a linear time sorting algorithm that sort in O(n+k)
time when elements are in range from 1 to k.
■ What if the elements are in range from 1 to n2?
■ We can’t use counting sort because counting sort will take O(n2)
which is worse than comparison based sorting algorithms.
■ Can we sort such an array in linear time?
■ Radix Sort will do so. The idea of Radix Sort is to do digit by digit
sort starting from least significant digit to most significant digit.
■ Radix sort uses counting sort as a subroutine to sort.
128
Radix Sort
182 182 005 005
523 192 523 083
005 523 233 182
233 233 182 192
192 083 192 233
083 005 083 523
129
Radix Sort
Algorithm RadixSort(A[L…U],d)
For d=LSD to MSD S(n) = 10 + c
CountingSort(A[]) w.r.t <d> = O(1)
T(n) = d * O(n+k)
= d * O(n+10)
= O(n.d)
= O(n)
130
131
ALGORITHMS
Lecture # 31, 32 & 33

TREES ADT,
BinARy SEARch TREE,
hEAp, AnALySiS OF hEAp SORT
Trees
■ A tree is a nonlinear abstract data type that stores elements in a
hierarchy.
■ Examples in real life:
– Family tree
– Table of contents of a book
– Class inheritance hierarchy in OOP
– Computer file system (folders and subfolders)
– Decision trees
2
Example: Computer File System
Root directory of C drive
Documents and Settings Program Files My Music
Desktop Favorites Start Menu Adobe Microsoft Office
3
Tree Definition
■ Tree: a set of elements that either
– it is empty
– or, it has a distinguished element called the root and

zero or more trees (called subtrees of the root)
4
Tree Definition
Root
Subtrees of
the root
5
Tree Terminology
■ Nodes: the elements in the tree
■ Edges: connections between nodes
■ Root: the distinguished element that is the origin of the tree

– There is only one root node in a tree
■ Empty tree has no nodes and no edges
6
Tree Terminology
edges Root
nodes
7
Tree Terminology
■ Parent or predecessor: the node directly above another node in

the hierarchy
– A node can have only one parent
■ Child or successor: a node directly below another node in the

hierarchy
■ Siblings: nodes that have the same parent
8
Tree Terminology
■ Ancestors of a node: its parent, the parent of its parent, etc.
■ Descendants of a node: its children, the children of its children,

etc.
■ Leaf node: a node without children
■ Internal node: a node that is not a leaf node
9
Tree Terminology
Interior or
internal Root
nodes
Leaf
nodes 10
Height of a Tree
■ A path is a sequence of edges leading from one node to another
■ Length of a path: number of edges on the path
■ Height of a (non-empty) tree : length of the longest path from the

root to a leaf
– What is the height of a tree that has only a root node?
– By convention, the height of an empty tree is -1
11
Level of a Node
■ Level of a node: number of edges between root and the node
■ It can be defined recursively:

– Level of root node is 0
– Level of a node that is not the root node is level of its parent + 1
12
Level of a Node
Level 0
Level 1
Level 2
Level 3
13
Subtrees
■ Subtree of a node: consists of a child node and all its
descendants:
– A subtree is itself a tree
– A node may have many subtrees
14
Subtrees
Subtrees of the
node labeled E
15
More Tree Terminology
■ Degree or arity of a node:

– the number of children it has
■ Degree or arity of a tree:

– the maximum of the degrees of the tree’s nodes
16
Binary Trees
■ General tree: a tree each of whose nodes may have any number
of children
■ n-ary tree: a tree each of whose nodes may have no more than n
children
■ Binary tree: a tree each of whose nodes may have no more than 2
children
– i.e. a binary tree is a tree with degree (arity) 2
■ The children (if present) are called the left child and right child
17
Binary Tree
A
B C
D E F G
H I
18
Array Based Implementation of BT
A
LeftChild(k)=k*2
RightChild(k)=k*2+1
B C
Parent(k)=k/2
D E F G
H I
Root
1 2 3 4 5 6 7 8 9 10 11 12
A B C D E F G Ø H Ø Ø I
19
A
LeftChild(k)=k*2
RightChild(k)=k*2+1
B C
Parent(k)=k/2
D E F G
H I
Root
1 2 3 4 5 6 7 8 9 10 11 12
A B C D Ø F G Ø H Ø Ø I
20
A
LeftChild(k)=k*2
RightChild(k)=k*2+1
B C
Parent(k)=k/2
D F G
H I
1 2 3 4 5 6 7 8 9 10 15
A B C D Ø F G Ø H Ø I
21
Linked List Based Implementation of BT
NULL Root
A Parent
Right
Left
A
B C
Parent Parent
E
Right
Right
Left
Left
B C
NULL NULL
NULL
Parent
Right
Left
E
NULL NULL
22
Tree Traversals
■ A traversal of a tree requires that each node of the tree be visited
once
– Example: a typical reason to traverse a tree is to display the data stored
at each node of the tree
■ Standard traversal orderings:
– Pre-order
– In-order
– Post-order
– Level-order
23
Traversals
A
B C
D E F G
H I
We’ll trace the different traversals using this tree; recursive calls,
returns, and “visits” will be numbered in the order they occur
24
Pre-order Traversal
■ Start at the root
■ Visit each node, followed by its children; we will choose to visit left
child before right
■ Recursive algorithm for preorder traversal:

– If tree is not empty,
■ Visit root node of tree
■ Perform preorder traversal of its left subtree
■ Perform preorder traversal of its right subtree
25
Pre-order Traversal
1: visit A
2 24
23 45
3: visit B 25: visit C
4 16 26 38
15 22 37 44
5: visit D 17: visit E 27: visit F 39: visit G
. . . . .
6 8 18 28
21 40
19 34 36 41 43
7 14 20 35 42
9: visit H 29: visit I
10
. .
11 12
13
30
. .
31 32
33
Nodes are visited in the order ABDHECFIG

26
In-order Traversal
■ Visit the left child of each node, then the node, then any
remaining nodes
■ Recursive algorithm for inorder traversal

■ Perform inorder traversal of left subtree of root
■ Perform inorder traversal of its right subtree
27
In-order Traversal
23: visit A
1 24
22 45
2 15 25 38
13 21 36 44
. . . . .
3 6 16 26
20 39
17 32 35 40 43
4 12 19 34 42
7
. .
8 10
11
27
. .
28 30
31
Nodes are visited in the order DHBEAIFCG

28
Post-order Traversal
■ Visit the children of each node, then the node
■ Recursive algorithm for postorder traversal

■ Perform postorder traversal of left subtree of root
■ Perform postorder traversal of right subtree of root
29
Post-order Traversal
45: visit A
1 23
22 44
2 14 24 36
13 20 35 42
. . . . .
3 5 15 25
18 37
16 31 33 38 40
4 11 17 32 39
6
. .
7 8
9
26
. .
27 28
29
Nodes are visited in the order HDEBIFGCA

30
Level Order Traversal
■ Visit the nodes at each level, from left to right
31
Level Order Traversal
A
B C
D E F G
H I
Nodes will be visited in the order ABCDEFGHI
32
Tree Traversals
■ Pre-order
Nodes are visited in the order ABDHECFIG
■ In-order
Nodes are visited in the order DHBEAIFCG
■ Post-order
Nodes are visited in the order HDEBIFGCA
■ Level-order
Nodes are visited in the order ABCDEFGHI
33
Traversal Analysis
■ Consider a binary tree with n nodes.
■ How many recursive calls are there at most?

– For each node, 2 recursive calls at most
– So, 2*n recursive calls at most
■ So, time taken for a complete traversal is O(n).

34
Binary Search Tree
■ A Binary Search Tree (BST) is a tree in
which all the nodes follow the below-
mentioned properties:
– The value of left child is less than the

value of its parent node.
– The value of right child is greater than or

equal to the value of its parent node.
35
Basic Operations
■ Following are the basic operations on a BST:
– Search − Searches an element in a BS tree.
– Insert − Inserts an element in a BS tree.
– Delete − Deletes an element in a BS tree.
– Traversal − Traverses a BS tree.
36
key=31
Search in BST
Root
37
key=31
Search in BST
Root
Current
38
key=31
Search in BST
Root
Current
39
key=31
Search in BST
Root
Current
40
key=15
Search in BST
Root
41
key=15
Search in BST
Root
Current
42
key=15
Search in BST
Root
Current
43
key=15
Search in BST
Root
Current
44
key=15
Search in BST
Root
Current=NULL
45
Insertion in BST
46
Insertion in BST
47
Insertion in BST
48
Insertion in BST
49
Insertion in BST
50
Insertion in BST
51
Insertion in BST
52
Insertion in BST
53
Insertion in BST
54
Insertion in BST
55
Insertion in BST
56
Insertion in BST
57
Insertion in BST
58
Insertion in BST
59
Insertion in BST
60
Insertion in BST
61
Insertion in BST
62
Insertion in BST
63
Insertion in BST
64
Insertion in BST
65
Insertion in BST
66
Insertion in BST
67
Insertion in BST
68
Insertion in BST
69
Insertion in BST
70
Insertion in BST
71
Insertion in BST
72
Insertion in BST
73
Insertion in BST
74
Insertion in BST
75
Insertion in BST
76
Insertion in BST
77
Insertion in BST
78
Insertion in BST
79
Insertion in BST
80
Insertion in BST
81
Insertion in BST
82
Insertion in BST
83
Insertion in BST
84
Insertion in BST
85
Insertion in BST
86
Insertion in BST
87
Insertion in BST
88
Insertion in BST
89
Insertion in BST
90
Insertion in BST
91
Insertion in BST
92
Insertion in BST
93
Insertion in BST
94
Deletion in BST
95
Deletion in BST
96
Deletion in BST
97
Deletion in BST
98
Deletion in BST
99
Deletion in BST
100
Deletion in BST
101
Deletion in BST
102
Deletion in BST
103
Deletion in BST
104
Deletion in BST
105
Deletion in BST
106
Deletion in BST
107
Deletion in BST
108
Deletion in BST
109
Deletion in BST
110
Deletion in BST
111
Deletion in BST
112
Deletion in BST
113
Deletion in BST
114
Deletion in BST
115
1
Unbalanced BSTs
5
■ Suppose we have the list:

7
10
■ After insertion in BST, we get →
■ To search 20, we will have to make “n” comparisons.
14
■ Height of the tree is (n-1), so T(n)=O(n)
■ Thus, it will be equivalent to linear search. 20
116
Balanced BSTs 10
7 14
■ If we re-arrange the same list:
8 20
1
■ Now, after insertion in BST, we get →

■ To search any value, we will have to make “log2n” comparisons.
■ Height of the tree is ≈log2n, so T(n)=O(log2n)
■ Thus, it will be equivalent to binary search.
117
Types of Balanced Trees
■ Now we understand why binary search trees need to be balanced
& what happens when we have a tree that’s not balanced?
■ There are a number of algorithms that can be used to remedy this
issue. Some of the most popular algorithms are:
– Red black trees
– AVL trees
■ Both of these force a BST to remain balanced, and therefore can
guarantee search performance.
118
Complexity in BST
119
Heap
■ Heap is an almost complete binary tree data structure where the
root-node key is compared with its children and arranged
accordingly.
■ As the value of parent is greater than that of child, this property
generates Max Heap. Based on this criteria, a heap can be of two
types:
– Min Heap
– Max Heap
■ For the sake on convenience, we will implement heap using arrays.
120
Complete vs. Almost Complete BT
121
Incomplete vs. Almost Complete BT
122
Min-Heap
For Input → 35 33 42 10 14 19 27 44 26 31
The value of any parent node is always less than or equal to either of
its children.
123
Max-Heap
For Input → 35 33 42 10 14 19 27 44 26 31
The value of any parent node is always greater than or equal to either
of its children.
124
Some Properties of Heap
■ As heap is a complete or almost complete BT, here are some
properties for your convenience
– Max number of nodes in a CBT of height, h: n = 2h+1-1
– Height, h, of CBT/ACBT with n nodes: h = log2n
– Number of Leaf nodes in CBT/ACBT: nleaf= n/2+1 to n
– Max. no. of nodes at height, h, in a CBT/ACBT: nh = n/2h+1
125
MAXHEAPIFY Function
Algorithm MaxHeapify(A[],i)
l=2*i, r=2*i+1
if l<=A.HeapSize & A[l]>A[i]
largest=l
else largest=i
if r<=A.HeapSize & A[r]>A[largest]
largest=r
if max≠i
A[i]↔A[largest]
MaxHeapify(A[],largest) 126
l=2*i, r=2*i+1
MAXHEAPIFY Function if l<=A.HeapSize & A[l]>A[i]
max=l
else max=i
if r<=A.HeapSize & A[r]>A[max]
max=r
if max≠i
A[i]↔A[max]
MaxHeapify(A[],max)
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
127
l=2*i, r=2*i+1
i max=l
1
else max=i
5 9 max=r
if max≠i
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
128
l=2*i, r=2*i+1
i max=l
1
else max=i
l r if r<=A.HeapSize & A[r]>A[max]
5 9 max=r
if max≠i
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
129
l=2*i, r=2*i+1
i max=l
1
else max=i
5 9 max=r
if max≠i
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
130
l=2*i, r=2*i+1
i max=l
1
else max=i
max
5 9 max=r
if max≠i
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
131
l=2*i, r=2*i+1
i max=l
1
else max=i
max
5 9 max=r
if max≠i
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
132
l=2*i, r=2*i+1
i max=l
1
else max=i
l r max if r<=A.HeapSize & A[r]>A[max]
5 9 max=r
if max≠i
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
133
l=2*i, r=2*i+1
i max=l
1
else max=i
5 9 max=r
if max≠i
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
134
l=2*i, r=2*i+1
i max=l
9
else max=i
5 1 max=r
if max≠i
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 9 5 1 2 8 7 3 4
1 2 3 4 5 6 7 8
135
l=2*i, r=2*i+1
max=l
9
else max=i
5 i 1 max=r
if max≠i
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 9 5 1 2 8 7 3 4
1 2 3 4 5 6 7 8
136
l=2*i, r=2*i+1
max=l
9
else max=i
max if r<=A.HeapSize & A[r]>A[max]
5 i 1 max=r
if max≠i
l r
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 9 5 1 2 8 7 3 4
1 2 3 4 5 6 7 8
137
l=2*i, r=2*i+1
max=l
9
else max=i
max if r<=A.HeapSize & A[r]>A[max]
5 i 1 max=r
if max≠i
l r
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
A 9 5 1 2 8 7 3 4
1 2 3 4 5 6 7 8
138
l=2*i, r=2*i+1
max=l
9
else max=i
5 i 1 max=r
if max≠i
l r
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
A 9 5 1 2 8 7 3 4
1 2 3 4 5 6 7 8
139
l=2*i, r=2*i+1
max=l
9
else max=i
5 i 1 max=r
if max≠i
l r
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
A 9 5 1 2 8 7 3 4
1 2 3 4 5 6 7 8
140
l=2*i, r=2*i+1
max=l
9
else max=i
5 i 1 max=r
if max≠i
l r
2 8 7 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
A 9 5 1 2 8 7 3 4
1 2 3 4 5 6 7 8
141
l=2*i, r=2*i+1
max=l
9
else max=i
5 i 7 max=r
if max≠i
l r
2 8 1 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
A 9 5 7 2 8 1 3 4
1 2 3 4 5 6 7 8
142
l=2*i, r=2*i+1
max=l
9
else max=i
5 7 max=r
if max≠i
l i r
2 8 1 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
A 7 5 7 2 8 1 3 4
1 2 3 4 5 6 7 8
143
l=2*i, r=2*i+1
max=l
9
else max=i
5 7 max=r
i if max≠i
2 8 1 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
l r
A 9 5 7 2 8 1 3 4
1 2 3 4 5 6 7 8
144
l=2*i, r=2*i+1
max=l
9
else max=i
5 7 max=r
i if max≠i
2 8 1 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
l r
A 9 5 7 2 8 1 3 4
1 2 3 4 5 6 7 8
145
l=2*i, r=2*i+1
max=l
9
else max=i
5 7 max=r
i if max≠i
2 8 1 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
l r
A 9 5 7 2 8 1 3 4
1 2 3 4 5 6 7 8
146
l=2*i, r=2*i+1
max=l
9
else max=i
5 7 max=r
i if max≠i
2 8 1 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
l r
A 9 5 7 2 8 1 3 4
1 2 3 4 5 6 7 8
147
l=2*i, r=2*i+1
max=l
9
else max=i
5 7 max=r
i if max≠i
2 8 1 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
l r
A 9 5 7 2 8 1 3 4
1 2 3 4 5 6 7 8
148
l=2*i, r=2*i+1
max=l
9
else max=i
5 7 max=r
i if max≠i
2 8 1 3 A[i]↔A[max]
MaxHeapify(A[],max)
max
4
l r
A 9 5 7 2 8 1 3 4
1 2 3 4 5 6 7 8
149
BUILDMAXHEAP Function
■ If we call MaxHeapify() for each node, we can build our Max Heap
from ordinary array.
■ But there is no need to call it on the leaf nodes, as each leaf node
already satisfies the property of Max Heap.
■ We also know that,
■ So, we just need to call MaxHeapify() for all the non-leaf nodes,
i.e. from 1 to n/2.
■ And we will call this function from n/2 to 1,so that as we move
above, all the lower nodes already satisfy the Max Heap property.
150
Algorithm BuildMaxHeap(A[])
for i=n/2 to 1 9
MaxHeapify(A[],i)
5 7
2 8 1 3
151
for i=n/2 to 1
MaxHeapify(A[],i)
1
5 9
2 8 7 3
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
152
MaxHeapify(A[],i)
1
5 9
i
2 8 7 3
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
i
153
MaxHeapify(A[],i)
1
5 9
i
4 8 7 3
A 1 5 9 4 8 7 3 2
1 2 3 4 5 6 7 8
i
154
MaxHeapify(A[],i)
1
i
5 9
4 8 7 3
A 1 5 9 4 8 7 3 2
1 2 3 4 5 6 7 8
i
155
MaxHeapify(A[],i)
1
i
5 9
4 8 7 3
A 1 5 9 4 8 7 3 2
1 2 3 4 5 6 7 8
i
156
MaxHeapify(A[],i)
1
i
5 9
4 8 7 3
A 1 5 9 4 8 7 3 2
1 2 3 4 5 6 7 8
i
157
MaxHeapify(A[],i)
1
i
8 9
4 5 7 3
A 1 8 9 4 5 7 3 2
1 2 3 4 5 6 7 8
i
158
MaxHeapify(A[],i)
1
i
8 9
4 5 7 3
A 1 8 9 4 5 7 3 2
1 2 3 4 5 6 7 8
i
159
MaxHeapify(A[],i)
9
i
8 1
4 5 7 3
A 9 8 1 4 5 7 3 2
1 2 3 4 5 6 7 8
i
160
MaxHeapify(A[],i)
9
i
8 1
4 5 7 3
A 9 8 1 4 5 7 3 2
1 2 3 4 5 6 7 8
i
161
MaxHeapify(A[],i)
9
i
8 7
4 5 1 3
A 9 8 7 4 5 1 3 2
1 2 3 4 5 6 7 8
i
162
EXTRACTMAXFROMHEAP Function
Algorithm ExtractMaxFromHeap(A[])
max=A[1] 9
A[1]=A[A.HeapSize--]
MaxHeapify(A[],1) 8 7
return max
4 5 1 3
163
9 max=A[1]
A[1]=A[A.HeapSize - -]
8 7 MaxHeapify(A[],1)
return max
4 5 1 3
A 9 8 7 4 5 1 3 2
1 2 3 4 5 6 7 8
164
9 max=A[1]
return max
4 5 1 3
max 9 A.HeapSize = 8
A 9 8 7 4 5 1 3 2
1 2 3 4 5 6 7 8
165
2 max=A[1]
return max
4 5 1 3
A 2 8 7 4 5 1 3 2
1 2 3 4 5 6 7 8
166
2 max=A[1]
return max
4 5 1 3
A 2 8 7 4 5 1 3
1 2 3 4 5 6 7
167
2 max=A[1]
return max
4 5 1 3
A 2 8 7 4 5 1 3
1 2 3 4 5 6 7
168
2 max=A[1]
return max
4 5 1 3
A 2 8 7 4 5 1 3
1 2 3 4 5 6 7
169
8 max=A[1]
return max
4 5 1 3
A 8 2 7 4 5 1 3
1 2 3 4 5 6 7
170
8 max=A[1]
return max
4 5 1 3
A 8 2 7 4 5 1 3
1 2 3 4 5 6 7
171
8 max=A[1]
return max
4 2 1 3
A 8 5 7 4 2 1 3
1 2 3 4 5 6 7
172
8 max=A[1]
return max
4 2 1 3
A 8 5 7 4 2 1 3
1 2 3 4 5 6 7
173
8 max=A[1]
return max
4 2 1 3
A 8 5 7 4 2 1 3
1 2 3 4 5 6 7
174
8 max=A[1]
return max
4 2 1 3
A 8 5 7 4 2 1 3
1 2 3 4 5 6 7
175
HeapSort
Algorithm HeapSort(A[])
BuildMaxHeap(A[])
New Array Temp[1…A.Size]
for i=A.Size to 1
Temp[i]=ExtractMaxFromHeap(A[])
Copy Temp[] to A[]
176
BuildMaxHeap(A[])
HeapSort New Array Temp[1…A.Size]
for i=A.Size to 1
1
Copy Temp[] to A[]
5 9
2 8 7 3
A 1 5 9 2 8 7 3 4
1 2 3 4 5 6 7 8
177
BuildMaxHeap(A[])
for i=A.Size to 1
9
Copy Temp[] to A[]
8 7
4 5 1 3
A 9 8 7 4 5 1 3 2
1 2 3 4 5 6 7 8
178
BuildMaxHeap(A[])
for i=A.Size to 1
9
Copy Temp[] to A[]
8 7
4 5 1 3
2 A 9 8 7 4 5 1 3 2
1 2 3 4 5 6 7 8
Temp
179
BuildMaxHeap(A[])
for i=A.Size to 1
9
Copy Temp[] to A[]
8 7
4 5 1 3
2 A 9 8 7 4 5 1 3 2
1 2 3 4 5 6 7 8
Temp
i
180
BuildMaxHeap(A[])
for i=A.Size to 1
8
Copy Temp[] to A[]
5 7
4 2 1 3
A 8 5 7 4 2 1 3 2
1 2 3 4 5 6 7 8
Temp 9
i
181
BuildMaxHeap(A[])
for i=A.Size to 1
8
Copy Temp[] to A[]
5 7
4 2 1 3
A 8 5 7 4 2 1 3 2
1 2 3 4 5 6 7 8
Temp 9
i
182
BuildMaxHeap(A[])
for i=A.Size to 1
7
Copy Temp[] to A[]
5 3
4 2 1
A 7 5 3 4 2 1 3 2
1 2 3 4 5 6 7 8
Temp 8 9
i
183
BuildMaxHeap(A[])
for i=A.Size to 1
7
Copy Temp[] to A[]
5 3
4 2 1
A 7 5 3 4 2 1 3 2
1 2 3 4 5 6 7 8
Temp 8 9
i
184
BuildMaxHeap(A[])
for i=A.Size to 1
5
Copy Temp[] to A[]
4 3
1 2
A 5 4 3 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 7 8 9
i
185
BuildMaxHeap(A[])
for i=A.Size to 1
5
Copy Temp[] to A[]
4 3
1 2
A 5 4 3 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 7 8 9
i
186
BuildMaxHeap(A[])
for i=A.Size to 1
4
Copy Temp[] to A[]
2 3
A 4 2 3 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 5 7 8 9
i
187
BuildMaxHeap(A[])
for i=A.Size to 1
4
Copy Temp[] to A[]
2 3
A 4 2 3 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 5 7 8 9
i
188
BuildMaxHeap(A[])
for i=A.Size to 1
3
Copy Temp[] to A[]
2 1
A 3 2 1 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 4 5 7 8 9
i
189
BuildMaxHeap(A[])
for i=A.Size to 1
3
Copy Temp[] to A[]
2 1
A 3 2 1 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 4 5 7 8 9
i
190
BuildMaxHeap(A[])
for i=A.Size to 1
2
Copy Temp[] to A[]
1
A 2 1 1 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 3 4 5 7 8 9
i
191
BuildMaxHeap(A[])
for i=A.Size to 1
2
Copy Temp[] to A[]
1
A 2 1 1 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 3 4 5 7 8 9
i
192
BuildMaxHeap(A[])
for i=A.Size to 1
1
Copy Temp[] to A[]
A 1 1 1 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 2 3 4 5 7 8 9
i
193
BuildMaxHeap(A[])
for i=A.Size to 1
1
Copy Temp[] to A[]
A 1 1 1 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 2 3 4 5 7 8 9
i
194
BuildMaxHeap(A[])
for i=A.Size to 1
Copy Temp[] to A[]
A 1 1 1 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 1 2 3 4 5 7 8 9
i
195
BuildMaxHeap(A[])
for i=A.Size to 1
Copy Temp[] to A[]
A 1 1 1 1 2 1 3 2
1 2 3 4 5 6 7 8
Temp 1 2 3 4 5 7 8 9
i
196
BuildMaxHeap(A[])
for i=A.Size to 1
Copy Temp[] to A[]
A 1 2 3 4 5 7 8 9
1 2 3 4 5 6 7 8
Temp 1 2 3 4 5 7 8 9
i
197
BuildMaxHeap(A[])
Analysis of Heap Sort New Array Temp[1…A.Size]
for i=A.Size to 1
Copy Temp[] to A[]
S(n) = SBuildMaxHeap + n + 1 + SExtractMaxFromHeap
T(n) = TBuildMaxHeap + n(c+TExtractMaxFromHeap) + O(n)
198
max=A[1]
Analysis of A[1]=A[A.HeapSize - -]
ExtractMaxFromHeap() MaxHeapify(A[],1)
return max
SExtractMaxFromHeap(n) = c + SMaxHeapify
TExtractMaxFromHeap(n) = c + TMaxHeapify
199
Analysis of MaxHeapify(A[],i)
BuildMaxHeap()
SBuildMaxHeap(n) = c + SMaxHeapify
TBuildMaxHeap(n) = c + (TMaxHeapify)*(n/2)
200
Some Properties of Heap
■ As heap is a complete or almost complete BT, here are some
properties for your convenience
– Max number of nodes in a CBT of height, h: n = 2h+1-1
– Height, h, of CBT/ACBT with n nodes: h = log2n
– Max. no. of nodes at height, h, in a CBT/ACBT: nh = n/2h+1
201
l=2*i, r=2*i+1
Analysis of if l<=A.HeapSize & A[l]>A[i]
MaxHeapify() max=l
else max=i
max=r
SMaxHeapify(n) = c + k.log2n if max≠i
= O(log2n) A[i]↔A[max]
MaxHeapify(A[],max)
TMaxHeapify(n) = c.log2n
= O(log2n)
202
Analysis of MaxHeapify(A[],i)
BuildMaxHeap()
SBuildMaxHeap(n) = c + O(log2n)
= O(log2n)
TBuildMaxHeap(n)*= c + O(log2n)*(n/2)
= O(n.log2n)
*Here actually, T(n) = O(n). But, lets consider a higher upper bound as it will not affect the final answer. 203
max=A[1]
Analysis of A[1]=A[A.HeapSize - -]
ExtractMaxFromHeap() MaxHeapify(A[],1)
return max
SExtractMaxFromHeap(n) = c + O(log2n)
= O(log2n)
TExtractMaxFromHeap(n) = c + O(log2n)
= O(log2n)
204
BuildMaxHeap(A[])
Analysis of Heap Sort New Array Temp[1…A.Size]
for i=A.Size to 1
Copy Temp[] to A[]
S(n) = O(log2n) + n + 1 + O(log2n)

= O(n)
T(n) = O(n.log2n) + n(c+O(log2n)) + O(n)

= O(n.log2n)
205
Online Quiz #2
■ Graded Quiz
■ 50 Minutes Quiz
■ Lecture 19 to 33
■ Date: 14 Jan, 2021

■ Time: Within Class hours
206
207
ALGORITHMS
Lecture # 34 & 35
BASIC GRAPH ALGORITHMS
& SPANNING TREES
The Graph ADT
■ The Graph Abstract Data Type

– Graph Representations
■ Elementary Graph Operations
– Depth First and Breadth First Search
– Spanning Tree
■ Minimum Cost Spanning Trees
– Kruskal’s, Prim’s and Sollin’s Algorithm
2
The Graph ADT
■ Definitions
– A graph G consists of two sets
■ a finite, nonempty set of vertices V(G)
■ a finite, possible empty set of edges E(G)
– G(V,E) represents a graph
– An undirected graph is one in which the pair of vertices
in an edge is unordered, (v0, v1) = (v1,v0)
– A directed graph is one in which each edge is a
directed pair of vertices, <v0, v1> != <v1, v0>
tail head
3
The Graph ADT
■ Examples for Graph
– complete undirected graph: n(n-1)/2 edges
– complete directed graph: n(n-1) edges
0
0 0 0
1 2 1
1 2 1 2
3 3 4 5 6 2
3
G1 G2 G3
complete graph incomplete graph
V(G1)={0,1,2,3} E(G1)={(0,1),(0,2),(0,3),(1,2),(1,3),(2,3)}
V(G2)={0,1,2,3,4,5,6} E(G2)={(0,1),(0,2),(1,3),(1,4),(2,5),(2,6)}
V(G3)={0,1,2} E(G3)={<0,1>,<1,0>,<1,2>}
4
The Graph ADT
■ Restrictions on graphs
– A graph may not have an edge from a vertex, i, back to
itself. Such edges are known as self loops
– A graph may not have multiple occurrences of the
same edge. If we remove this restriction, we obtain a
data referred to as a multigraph
feedback
loops multigraph
5
The Graph ADT
■ Adjacent and Incident
■ If (v0, v1) is an edge in an undirected graph,
– v0 and v1 are adjacent
– The edge (v0, v1) is incident on vertices v0 and v1
V0 V1
■ If <v0, v1> is an edge in a directed graph

– v0 is adjacent to v1, and v1 is adjacent from v0
– The edge <v0, v1> is incident from v0 on v1
V0 V1
6
The Graph ADT
■ A subgraph of G is a graph G’ such that V(G’)  V(G)
and E(G’)  E(G).
0
1 2
3
G1 0
2
G3 7
The Graph ADT
■ Path
– A path from vertex vp to vertex vq in a graph G, is a
sequence of vertices, vp, vi1, vi2, ..., vin, vq, such that (vp,
vi1), (vi1, vi2), ..., (vin, vq) are edges in an undirected graph.
■ A path such as (0, 2), (2, 1), (1, 3) is also written as 0, 2, 1, 3
– The length of a path is the number of edges on it
0 0 0
1 2 1 2 1 2
3 3 3
8
The Graph ADT
■ Simple path and cycle
– simple path (simple directed path): a path in which all
vertices, except possibly the first and the last, are
distinct.
– A cycle is a simple path in which any vertex is repeated.
1 2
3
9
The Graph ADT
■ Connected graph
– In an undirected graph G, two vertices, v and v , are
0 1
connected if there is a path in G from v to v

0 1
– An undirected graph is connected if, for every pair of

distinct vertices v , v , there is a path from v to v
i j i j
■ Connected component
– A connected component of an undirected graph is a
maximal connected
subgraph.
– A tree is a graph
that is connected
and acyclic (i.e,
has no cycle).
10
The Graph ADT
■ Strongly Connected Graph/Component
– A directed graph is strongly connected if there is a
directed path from vi to vj and also from vj to vi
– A strongly connected component is a maximal
subgraph that is strongly connected
0 strongly connected component

(maximal strongly connected subgraphs)
0
1
2
G3 not strongly connected 1 2
11
The Graph ADT
■ Degree (undirected graph)
– The degree of a vertex is the number of edges incident to
that vertex.
■ For directed graph
– in-degree (v) : the number of edges that have v as the head
– out-degree (v) : the number of edges that have v as the tail
■ If di is the degree of a vertex i in a graph G with n
vertices and e edges, the number of edges is
n 1
e  ( di ) / 2
0
12
The Graph ADT
■ We shall refer to a directed graph as a digraph.
When we us the term graph, we assume that it is
an undirected graph
digraph
undirected graph in-degree & out-degree
degree 2
3
0 in:1, out: 1
0
0 3 3
1 2 1 in: 1, out: 2
3 1 2 3
1 1 1 1
3 3 4 5 6 2 in: 1, out: 0
3
G1 G2 G3 13
The Graph ADT
14
Graph Representations
■ Adjacency Matrix
15
Graph Representations 4
■ Adjacency Matrix 5
– Let G = (V,E) be a graph with n vertices. 6
– The adjacency matrix of G is a two-dimensional
7
n x n array, say adj_mat
0
– If the edge (vi, vj) is(not) in E(G), adj_mat[i][j]=(0)
2 1
– The adjacency matrix for an undirected graph is
symmetric; the adjacency matrix for a digraph 3 G4
need not be symmetric
0 1 1 0 0 0 0 0
0 1

0 0 1 0 0 0 0

0 0 1 1 1 1 0 0 1 0 0 0 0
1  
0 1 1  0 1 0
0 1 1 0 0 0 0 0
  1 0 1 
1 2 1 1 0 1 1   0 0 0 0 0 1 0 0
  0 0 0  
3 1 1 1 0 0 0 0 0 1 0 1 0
0 0 0 0 0 1 0 1
G1 2  
G3 0 0 0 0 0 0 1 0 16
0
Graph Representations 1 2
■ Merits of Adjacency Matrix 3
0 1 1 1
– For an undirected graph, the degree of any 1 0 1 1
 
vertex, i, is its row sum: 1 1 0 1
n 1  
1 1 1 0
 adj _ mat[i ][ j ]
j 0
– For a directed graph, the row sum is the out- 0

degree, while the column sum is the in-degree.
 0 1 0
1 0 1 
n 1 n 1 1  
ind (vi )   A[ j , i ] outd ( vi )   A[i , j ] 0 0 0
j 0 j 0
2
17
■ Adjacency Lists
18
■ Adjacency lists
– There is one list for each vertex in G.

The nodes in list i represent the
vertices that are adjacent from vertex i
– For an undirected graph with n

vertices and e edges, this
representation requires n head nodes
and 2e list nodes
19
Graph Representations G3
0 4
0 2 1 5
3 6
0
G2 1 7
1 2
G1 3 2
20
0
Graph Representations 1 2
3
■ Interesting Operations
– degree of a vertex in an undirected graph
■ # of nodes in adjacency list
– # of edges in a graph
■ determined in O(n+e)
0
– out-degree of a vertex in a directed graph
■ # of nodes in its adjacency list
– in-degree of a vertex in a directed graph 1
■ traverse the whole data structure
2
21
■ Vertices in Any Order
Order is of no significance
Head nodes vertex links
0  3  1  2 NULL
0
1  2  0  3 NULL
1 2
2  3  0  1 NULL
3
3  2  1  0 NULL
22
23
■ Weighted edges
– The edges of a graph have weights assigned to them.
– These weights may represent as
■ the distance from one vertex to another
■ cost of going from one vertex to an adjacent vertex.
– adjacency matrix: adj_mat[i][j] would keep the weights.
– adjacency lists: add a weight field to the node structure.
– A graph with weighted edges is called a network.
24
25
Graph Operations
■ Traversal:
Given G=(V,E) and vertex v, find all wV, such that
w connects v
– Depth First Search (DFS): preorder traversal
– Breadth First Search (BFS): level order traversal
■ Spanning Trees
26
Depth First Search
Algorithm DepthFirstSearch(Vertex v)
if v is unvisited
push v
print v
mark v as visited
u=next unvisited adjacent node of v
if u=NULL
pop() & u=peak()
if u≠NULL, DepthFirstSearch(u)
27
Depth First Search if v is unvisited

push v
v 5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
OUTPUT:
28

push v
v 5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5
29

push v
v 5 print v
mark v as visited
u 3 1 9
if u=NULL
pop() & u=peak()
8 2 0
5
OUTPUT: 5
30

push v
v 5 print v
mark v as visited
u 3 1 9
if u=NULL
pop() & u=peak()
8 2 0
5
OUTPUT: 5
31

push v
5 print v
mark v as visited
v u=next unvisited adjacent node of v

u 3 1 9
if u=NULL
pop() & u=peak()
8 2 0
5
OUTPUT: 5
32

push v
5 print v
mark v as visited

u 3 1 9
if u=NULL
pop() & u=peak()
8 2 0
3
5
OUTPUT: 5 3
33

push v
5 print v
mark v as visited

if u=NULL
3 1 9
pop() & u=peak()
u
8 2 0
3
5
OUTPUT: 5 3
34

push v
5 print v
mark v as visited

if u=NULL
3 1 9
pop() & u=peak()
u
8 2 0
3
5
OUTPUT: 5 3
35

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
u v
8 2 0
3
5
OUTPUT: 5 3
36

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 u v
8 2 0
3
5
OUTPUT: 5 3 8
37

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 v
8 2 0
3
u=NULL OUTPUT: 5 3 8
38

push v
5 print v
mark v as visited
u u=next unvisited adjacent node of v

if u=NULL
3 1 9
pop() & u=peak()
v
8 2 0
3
5
OUTPUT: 5 3 8
39

push v
5 print v
mark v as visited

if u=NULL
3 1 9
pop() & u=peak()
v if u≠NULL, DepthFirstSearch(u)
8 2 0
3
5
OUTPUT: 5 3 8
40

push v
5 print v
mark v as visited

if u=NULL
3 1 9
pop() & u=peak()
8 2 0
3
5
OUTPUT: 5 3 8
41

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
3
5
u=NULL OUTPUT: 5 3 8
42

push v
5 u print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8
43
v push v
5 u print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8
44
v push v
5 u print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8
45
v push v
print v
5
mark v as visited
u if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8
46
v push v
print v
5
mark v as visited
u if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8
47

push v
5 print v
mark v as visited
v u if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8
48

push v
5 print v
mark v as visited
v u if u=NULL
3 1 9
pop() & u=peak()
8 2 0
1
5
OUTPUT: 5 3 8 1
49

push v
5 print v
mark v as visited
v if u=NULL
3 1 9
pop() & u=peak()
8 2 0
1
5
u=NULL OUTPUT: 5 3 8 1
50

push v
5
u print v
mark v as visited
v if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8 1
51
v u
push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8 1
52
v u
push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8 1
53
v push v
5 print v
mark v as visited
u if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8 1
54
v push v
5 print v
mark v as visited
u if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8 1
55

push v
5 print v
mark v as visited
u if u=NULL
3 1 v 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8 1
56

push v
5 print v
mark v as visited
u if u=NULL
3 1 v 9
pop() & u=peak()
8 2 0
9
5
OUTPUT: 5 3 8 1 9
57

push v
5 print v
mark v as visited
if u=NULL
3 1 v 9
pop() & u=peak()
9
8 2
u 0
5
OUTPUT: 5 3 8 1 9
58

push v
5 print v
mark v as visited
if u=NULL
3 1 v 9
pop() & u=peak()
9
8 2
u 0
5
OUTPUT: 5 3 8 1 9
59

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
9
8
v 2
u 0
5
OUTPUT: 5 3 8 1 9
60

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
9
8
v 2
u 0
5
OUTPUT: 5 3 8 1 9 2
61

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
9
8
v 2 0
5
u=NULL OUTPUT: 5 3 8 1 9 2
62

push v
5 print v
mark v as visited
u if u=NULL
3 1 9
pop() & u=peak()
9
8
v 2 0
5
OUTPUT: 5 3 8 1 9 2
63

push v
5 print v
mark v as visited
v u if u=NULL
3 1 9
pop() & u=peak()
8 2 0
9
5
OUTPUT: 5 3 8 1 9 2
64

push v
5 print v
mark v as visited
v u if u=NULL
3 1 9
pop() & u=peak()
8 2 0
9
5
OUTPUT: 5 3 8 1 9 2
65

push v
5 print v
mark v as visited
v if u=NULL
3 1 9
pop() & u=peak()
9
8 2 0 u
5
OUTPUT: 5 3 8 1 9 2
66

push v
5 print v
mark v as visited
v if u=NULL
3 1 9
pop() & u=peak()
9
8 2 0 u
5
OUTPUT: 5 3 8 1 9 2
67

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
9
8 2 v 0 u
5
OUTPUT: 5 3 8 1 9 2
68

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
9
8 2 v 0 u
5
OUTPUT: 5 3 8 1 9 2 0
69

push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
9
8 2 v 0
5
u=NULL OUTPUT: 5 3 8 1 9 2 0
70

push v
5 print v
mark v as visited

if u=NULL
3 1 9
pop() & u=peak()
9
8 2 v 0
5
OUTPUT: 5 3 8 1 9 2 0
71

push v
5 print v
mark v as visited
v u u=next unvisited adjacent node of v

if u=NULL
3 1 9
pop() & u=peak()
8 2 0
9
5
OUTPUT: 5 3 8 1 9 2 0
72

push v
5 print v
mark v as visited
v u u=next unvisited adjacent node of v

if u=NULL
3 1 9
pop() & u=peak()
8 2 0
9
5
OUTPUT: 5 3 8 1 9 2 0
73

push v
5 print v
mark v as visited
v if u=NULL
3 1 9
pop() & u=peak()
8 2 0
9
5
u=NULL OUTPUT: 5 3 8 1 9 2 0
74
u push v
print v
5
mark v as visited
v if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8 1 9 2 0
75
v u push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8 1 9 2 0
76
v u push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
OUTPUT: 5 3 8 1 9 2 0
77
v push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
5
u=NULL OUTPUT: 5 3 8 1 9 2 0
78
v push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
u=NULL OUTPUT: 5 3 8 1 9 2 0
79
v push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
u=NULL OUTPUT: 5 3 8 1 9 2 0
80
v push v
5 print v
mark v as visited
if u=NULL
3 1 9
pop() & u=peak()
8 2 0
u=NULL OUTPUT: 5 3 8 1 9 2 0
81
Breadth First Search
Algorithm BreadthFirstSearch(Vertex v)
print v
unvisited
if v is
mark v as visited
enqueue(all unvisited adjacent nodes of v)
u=dequeue()
if u≠NULL, BreadthFirstSearch(u)
82
Breadth First Search print v

mark v as visited
v 5
u=dequeue()
3 1 9
FRONT
QUEUE:
8 2 0
OUTPUT: 5
83

mark v as visited
v 5
u=dequeue()
3 1 9
FRONT
QUEUE:
8 2 0
OUTPUT: 5
84

mark v as visited
v 5
u=dequeue()
3 1 9
FRONT
QUEUE: 3 1 9
8 2 0
OUTPUT: 5
85

mark v as visited
v 5
u=dequeue()
u
3 1 9
FRONT
QUEUE: 1 9
8 2 0
OUTPUT: 5
86

mark v as visited
5
u=dequeue()
u
3 1 9
v FRONT
QUEUE: 1 9
8 2 0
OUTPUT: 5
87

mark v as visited
5
u=dequeue()
u
3 1 9
v FRONT
QUEUE: 1 9
8 2 0
OUTPUT: 5 3
88

mark v as visited
5
u=dequeue()
u
3 1 9
v FRONT
QUEUE: 1 9 8
8 2 0
OUTPUT: 5 3
89

mark v as visited
5
u=dequeue()
u
3 1 9
v FRONT
QUEUE: 9 8
8 2 0
OUTPUT: 5 3
90

mark v as visited
5
u=dequeue()
u
3 1 9
v FRONT
QUEUE: 9 8
8 2 0
OUTPUT: 5 3
91

mark v as visited
5
u=dequeue()
u
3 1 9
v FRONT
QUEUE: 9 8
8 2 0
OUTPUT: 5 3 1
92

mark v as visited
5
u=dequeue()
u
3 1 9
v FRONT
QUEUE: 9 8
8 2 0
OUTPUT: 5 3 1
93

mark v as visited
5
u=dequeue()
u
3 1 9
v FRONT
QUEUE: 8
8 2 0
OUTPUT: 5 3 1
94

mark v as visited
5
u=dequeue()
u
3 1 9 v
FRONT
QUEUE: 8
8 2 0
OUTPUT: 5 3 1
95

mark v as visited
5
u=dequeue()
u
3 1 9 v
FRONT
QUEUE: 8
8 2 0
OUTPUT: 5 3 1 9
96

mark v as visited
5
u=dequeue()
u
3 1 9 v
FRONT
QUEUE: 8 2 0
8 2 0
OUTPUT: 5 3 1 9
97

mark v as visited
5
u=dequeue()
3 1 9 v FRONT
QUEUE: 2 0
u
8 2 0
OUTPUT: 5 3 1 9
98

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE: 2 0
u v
8 2 0
OUTPUT: 5 3 1 9
99

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE: 2 0
u v
8 2 0
OUTPUT: 5 3 1 9 8
100

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE: 2 0
u v
8 2 0
OUTPUT: 5 3 1 9 8
101

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE: 0
v u
8 2 0
OUTPUT: 5 3 1 9 8
102

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE: 0
u v
8 2 0
OUTPUT: 5 3 1 9 8
103

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE: 0
u v
8 2 0
OUTPUT: 5 3 1 9 8 2
104

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE: 0
u v
8 2 0
OUTPUT: 5 3 1 9 8 2
105

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE:
v u
8 2 0
OUTPUT: 5 3 1 9 8 2
106

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE:
u v
8 2 0
OUTPUT: 5 3 1 9 8 2
107

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE:
u v
8 2 0
OUTPUT: 5 3 1 9 8 2 0
108

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE:
u v
8 2 0
OUTPUT: 5 3 1 9 8 2 0
109

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE:
v
8 2 0
u=NULL OUTPUT: 5 3 1 9 8 2 0
110

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE:
v
8 2 0
u=NULL OUTPUT: 5 3 1 9 8 2 0
111

mark v as visited
5
u=dequeue()
3 1 9
FRONT
QUEUE:
v
8 2 0
u=NULL OUTPUT: 5 3 1 9 8 2 0
112
Graph Operations
■ Spanning trees
– Definition: A tree T is said to be a spanning tree of a
connected graph G if T is a subgraph of G and T
contains all vertices of G.
– E(G): T (tree edges) + N (nontree edges)
■ T: set of edges used during search
■ N: set of remaining edges
113
Graph Operations
■ When graph G is connected, a depth first or breadth first
search starting at any vertex will visit all vertices in G
■ We may use DFS or BFS to create a spanning tree
– Depth first spanning tree when DFS is used
– Breadth first spanning tree when BFS is used
114
Breadth First Spanning Tree
Algorithm BreadthFirstSpanningTree(Vertex v)
Graph ST.AddVertices(V)
BreadthFirstTraversal(v)
Algorithm BreadthFirstTraversal(Vertex v)
if v is unvisited
mark v as visited and added
enqueue(all unvisited unadded adjacent nodes of v)
ST.AddEdges(v,all unvisited unadded adjacent nodes of v)
u=dequeue()
if u≠NULL, BreadthFirstTraversal(u) 115
Breadth First
Spanning Tree
G
5
3 1 9
8 2 0
116
Breadth First
Spanning Tree
5 *
G 3 *
5 ST
5
1 *
9 *
8 *
3 1 9
3 1 9
2 *
0 *
ST
8 2 0 8 2 0
117
Breadth First
Spanning Tree
5 *
V
G 3 *
5 ST
5
1 *
9 *
8 *
3 1 9
3 1 9
2 *
0 *
ST
8 2 0 8 2 0
118
FRONT
QUEUE:
V ST
G 5 ST
5
5 *
3 *
1 *
3 1 9 3 1 9
9 *
8 *
2 *
8 2 0
8 2 0
0 * 119
FRONT
QUEUE:
V
G ST ST
5 5
5 *
3 *
1 *
3 1 9 3 1 9
9 *
8 *
2 *
8 2 0
8 2 0
0 * 120
FRONT
QUEUE: 3 1 9
V
G ST ST
5 5
5 *
3 *
1 *
3 1 9 3 1 9
9 *
8 *
2 *
8 2 0
8 2 0
0 * 121
FRONT
QUEUE: 3 1 9
V
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 *
2 *
8 2 0
8 2 0
0 * 122
FRONT
QUEUE: 1 9
V
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 Ø
U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 *
2 *
8 2 0
8 2 0
0 * 123
FRONT
QUEUE: 1 9
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 *
2 *
8 2 0
8 2 0
0 * 124
FRONT
QUEUE: 1 9
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 *
2 *
8 2 0
8 2 0
0 * 125
FRONT
QUEUE: 1 9
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 *
2 *
8 2 0
8 2 0
0 * 126
FRONT
QUEUE: 1 9 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 *
2 *
8 2 0
8 2 0
0 * 127
FRONT
QUEUE: 1 9 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 128
FRONT
QUEUE: 9 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 129
FRONT
QUEUE: 9 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 130
FRONT
QUEUE: 9 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 131
FRONT
QUEUE: 9 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 132
FRONT
QUEUE: 9 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 133
FRONT
QUEUE: 9 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 134
FRONT
QUEUE: 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 135
FRONT
QUEUE: 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 136
FRONT
QUEUE: 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 137
FRONT
QUEUE: 8
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 138
FRONT
QUEUE: 8 2 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 Ø
8 * 3 Ø
2 *
8 2 0
8 2 0
0 * 139
FRONT
QUEUE: 8 2 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V U
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
2 * 9 Ø
8 0
8 2 0
0 * 9 Ø 2
140
FRONT
QUEUE: 2 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
V
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
2 * 9 Ø
U 8 0
8 2 0
0 * 9 Ø 2
141
FRONT
QUEUE: 2 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
V Ø
U 2 * 9
8 0
8 2 0
0 * 9 Ø 2
142
FRONT
QUEUE: 2 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
V Ø
U 2 * 9
8 0
8 2 0
0 * 9 Ø 2
143
FRONT
QUEUE: 2 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
V Ø
U 2 * 9
8 0
8 2 0
0 * 9 Ø 2
144
FRONT
QUEUE: 2 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
V Ø
U 2 * 9
8 0
8 2 0
0 * 9 Ø 2
145
FRONT
QUEUE: 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
V U Ø
2 * 9
8 0
8 2 0
0 * 9 Ø 2
146
FRONT
QUEUE: 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
U Ø
V 2 * 9
8 0
8 2 0
0 * 9 Ø 2
147
FRONT
QUEUE: 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
U Ø
V 2 * 9
8 0
8 2 0
0 * 9 Ø 2
148
FRONT
QUEUE: 0
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
U Ø
V 2 * 9
8 0
8 2 0
0 * 9 Ø 2
149
FRONT
QUEUE:
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
V U 2 * 9 Ø
8 0
8 2 0
0 * 9 Ø 2
150
FRONT
QUEUE:
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
U 2 * 9 Ø
V 8 0
8 2 0
0 * 9 Ø 2
151
FRONT
QUEUE:
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
U 2 * 9 Ø
V 8 0
8 2 0
0 * 9 Ø 2
152
FRONT
QUEUE:
G ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
U 2 * 9 Ø
V 8 0
8 2 0
0 * 9 Ø 2
153
FRONT
QUEUE:
G U=NULL ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
2 * 9 Ø
V 8 0
8 2 0
0 * 9 Ø 2
154
FRONT
QUEUE:
G U=NULL ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
2 * 9 Ø
V 8 0
8 2 0
0 * 9 Ø 2
155
FRONT
QUEUE:
G U=NULL ST ST
5 5
5 * 3 1 9 Ø
3 * 5 8 Ø
1 * 5 Ø
3 1 9 3 1 9
9 * 5 2 0 Ø
8 * 3 Ø
2 * 9 Ø
V 8 0
8 2 0
0 * 9 Ø 2
156
Algorithm BreadthFirstSpanningTree(Vertex v)
Graph ST.AddVertices(V)
BreadthFirstTraversal(v)
return ST
Algorithm BreadthFirstTraversal(Vertex v)
if v is unvisited
mark v as visited and added
enqueue(all unvisited unadded adjacent nodes of v)
ST.AddEdges(v,all unvisited unadded adjacent nodes of v)
u=dequeue()
if u≠NULL, BreadthFirstTraversal(u)
157
158
ALGORITHMS
Lecture # 36, 37 & 38

The Greedy ApproAch
& InTervAl SchedulInG
ONLINE QUIZ #2
■ Conducted on 14TH January, 2021
■ Time Slot: 13:30 pm to 16:30 pm
2
Greedy Algorithms
■ Hard to define exactly but can give general properties

• Solution is built in small steps
• Decisions on how to build the solution are made to maximize
some criterion without looking to the future
■ Want the ‘best’ current partial solution as if the current
step were the last step
■ May be more than one greedy algorithm using different

criteria to solve a given problem.
3
Greedy Strategy
Goal: Given currency denominations: 1, 5, 10, 25, 100,
give change to customer using fewest number of coins.
Ex: Rs. 34
– 25, 5, 1,1,1,1
Cashier's algorithm: At each iteration, give the largest

coin valued ≤ the amount to be paid.
Ex: Rs. 30
– 25, 5
4
Greedy is not always Optimal
Observation: Greedy algorithm is sub-optimal for say postal denominations:
1, 10, 21, 34, 70, 100, 350, 1225, 1500.
Counterexample. Rs. 140.

– Greedy: 100, 34, 1, 1, 1, 1, 1, 1.
– Optimal: 70, 70.
Lesson: Greedy is short-sighted. Always chooses the most attractive choice

at the moment. But this may lead to a dead-end later.
5
Interval Scheduling
6
Compatibility
• Two jobs compatible if they don’t overlap.
7
Greedy Strategy for Interval Scheduling
1. Use a simple rule to select a request i.
2. Reject all requests that are incompatible with i.
3. Repeat until all requests are processed.
■ This is just a template. We can substitute the underlined phrase

with a greedy rule.
■ There are many possible candidates for this greedy rule.
8
Heuristic 1
■ From the given set of intervals, select a request that STARTS
EARLIEST; that is, the one with the least start time (minimum
s(i)).
9
Heuristic 2
■ Accept the request that has the SMALLEST INTERVAL OF TIME,
i.e., the request for which f(i) − s(i) is minimum.
10
Heuristic 3
■ For each request, we count the number of other requests that
are not compatible, and accept the request that has the fewest
number of non-compatible requests.
■ In other words, we select the interval with the FEWEST
’CONFLICTS’.
11
Heuristic 4
■ Consider the request with respect to their NUMERIC ORDER.
12
Heuristic 5
■ Accept first the request that finishes first, that is, the request i
for which f(i) is as small as possible.
■ Consider the request with EARLIEST FINISH TIME.
Not finding a counterexample of

a heuristic doesn’t imply its proof.
13
Is Heuristic 5 Correct i.e. OPTIMAL?
■ In general, to prove that a greedy algorithm always yields an
optimum solution, we compare the solution output by algorithm
and any optimum solution.
■ Let ALG = { i1, i2, … , ik } be the set of requests output by our
algorithm in order.
■ Let OPT = { j1, j2, … , jm } be an ordered OPTIMAL solution in which
all the request are compatible.
■ Our goal is to prove that k = m. i.e. we need to prove that
|ALG| = |OPT|
14
Are requests in ALG COMPATIBLE?
■ Template of ALG:
■ The intervals in the set ALG returned by the algorithm are

compatible. Thus, heuristic 5 guarantees that for all indices r ≤ k:
f(ir) ≤ f(ik)
1 2 3 r … k
15
ALG = { i1, i2, … , ik }
OPT = { j1, j2, … , jm }
Lemma: f(ir) ≤ f(jr), 1 ≤ r ≤ k
■ For each r ≥ 1, the r th accepted request in the algorithm’s
schedule finishes no later than the rth request in the optimal
schedule.
P(r): f(ir) ≤ f(jr), 1 ≤ r ≤ k
i1 i2 i3 i4 … ik
j1 j2 j3 j4 … jm
16
ALG = { i1, i2, … , ik }
OPT = { j1, j2, … , jm }
Proof of Lemma
P(r): f(ir) ≤ f(jr), 1 ≤ r ≤ k i1

■ We shall prove this by mathematical induction on r.
j1
BASE CASE:
P(1): f(i1) ≤ f(j1) is TRUE.
For r = 1, the algorithm starts by selecting the request i1 with
minimum finish time. So, for every other request, f(i1) ≤ f(ip), 2 ≤ p ≤ k.
In particular, in OPT, j1 is either i1 itself (or) some other request say ip,
2 ≤ p ≤ k. Therefore, f(ir) ≤ f(jr) when r = 1. 17
ALG = { i1, i2, … , ik }
OPT = { j1, j2, … , jm }
Proof of Lemma
HYPOTHESIS:
We assume that P(r-1): f(ir-1) ≤ f(jr-1) is TRUE for r>1.
INDUCTION STEP:
P(r-1)P(r)
or f(ir-1) ≤ f(jr-1)  f(ir) ≤ f(jr)
ir-1 ir
jr-1 jr
18
ALG = { i1, i2, … , ik }
OPT = { j1, j2, … , jm }
Proof of Lemma
ir-1 ir
P(r-1)P(r)
or f(ir-1) ≤ f(jr-1)  f(ir) ≤ f(jr) jr-1 jr
Now, suppose f(ir)>f(jr) is TRUE, as shown in the diagram below.

We will prove that our supposition is not possible.
f(ir-1) ≤ f(jr-1) By Induction Hypothesis ir-1 ir
f(jr-1) ≤ s(jr) OPT contains compatible requests

jr-1 jr
 f(ir-1) ≤ s(jr) By Hypothetical Syllogism
Thus if f(jr) < f(ir), then as jr is the interval with shortest finish
time, according to the heuristic, jrALG and irALG. The later is
contrary to GIVEN and hence not possible.
19
ALG = { i1, i2, … , ik }
OPT = { j1, j2, … , jm }
Proof of Lemma
So, by trichotomy property, if f(ir)>f(jr) is FALSE, then f(ir)  f(jr)
must be TRUE.
Therefore, for each r, the r th interval the ALG selects, finishes no

later than the r th interval in OPT.
ir-1 ir
jr-1 jr
PROOF: Heuristic 5 is Correct i.e. Optimal
ALG = { i1, i2, … , ik } ik-1 ik
OPT = { j1, j2, … , jm } jk-1 jm
Statement: The greedy algorithm ALG is optimal. i.e., k = m.

Proof by contradiction: If ALG is not optimal, then k ≠ m. Since,
k>m is not possible as |ALG| cannot be greater than |OPT|, so by
trichotomy property m>k.
This means that there is a request jk+1 in OPT.
ALG = { i1, i2, … , ik } ik-1 ik
OPT = { j1, j2, … , jk, jk+1, … , jm } jk-1 jk jk+1 … jm
21
PROOF: Heuristic 5 is Correct i.e. Optimal
ALG = { i1, i2, … , ik } ik-1 ik
OPT = { j1, j2, … , jk, jk+1, … , jm } jk-1 jk jk+1 … jm
From above diagram and results from previous slides, it is obvious

that:
f(ik)  f(jk)  s(jk+1)  f(jk+1) or simply f(ik)  f(jk+1)
So after deleting all requests that are not compatible with ik, the set
of possible requests, say R, still contains jk+1.
But the greedy algorithm stops with request ik, and it is only
supposed to stop when R is empty.
Therefore, jk+1 cannot exist and our assumption that m > k is wrong.
Hence, m = k.
22
Class Task 6
■ What is the time complexity of solving the Interval Scheduling
Problem if BRUTE FORCE TECHNIQUE would have been used?
– Upload a single .pdf file on LMS

– Deadline: 28 January, 2021 before midnight.
23
Class Task 7
■ What is the time complexity of solving the Interval Scheduling
Problem using GREEDY HEURISTIC 5?
– Upload a single .pdf file on LMS

– Deadline: 28 January, 2021 before midnight.
24
Online Quiz #3
■ Graded Quiz
■ 50 Minutes Quiz
■ Lecture 31 to 39
■ Date: 30 Jan, 2021

■ Time: Will be announced later
25
26
ALGORITHMS
Lecture # 39
MiniMuM spanning trees (Mst),
priM’s & KrusKaL’s aLgOritHM
Minimum Cost Spanning Trees
■ Introduction
– The cost of a spanning tree of a weighted, undirected graph is
the sum of the costs (weights) of the edges in the spanning tree.
– A minimum-cost spanning tree is a spanning tree of least cost.
– Two different algorithms can be used to obtain a minimum cost
spanning tree.
■ Kruskal’s algorithm
■ Prim’s algorithm
– All these algorithms use a design strategy called the greedy
approach.
– MST of a graph may not be unique.
Minimum Cost Spanning Trees
■ Greedy Strategy
– Construct an optimal solution in stages
– At each stage, we make the best decision (selection) possible at
this time.
■ using least-cost criterion for constructing minimum-cost spanning trees
– Make sure that the decision will result in a feasible solution
– A feasible solution works within the constraints specified by the
problem
■ Our solution must satisfy the following constrains
– Must use only edges within the graph.
– Must use exactly n - 1 edges.
– May not use edges that produce a cycle
Kruskal’s Algorithm
Algorithm Kruskal_MST(Graph G)
New Graph MST
MST.AddVertices(G.V[])
Sort G.E[] in ascending order
For min(u,v) in G.E[]
MST.AddEdges((u,v))
If MST.hasCycles()=TRUE
MST.RemoveEdge((u,v))
if |MST.E[]|≠|MST.V[]|-1
return NULL
return MST
4
Kruskal’s Algorithm G
A
4 3
5
B
D
1
C 5 7
1 3 E
10
F
3
G
G.E[ ]
A,B A,C A,D B,E C,D C,E C,F E,F D,G E,G
3 5 4 7 1 5 1 3 10 3
5
Kruskal’s Algorithm G MST
A
4 3
5
B
D
1
C 5 7
1 3 E
10
F
3
G
G.E[ ]
3 5 4 7 1 5 1 3 10 3
6
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
3 5 4 7 1 5 1 3 10 3
7
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
C,D C,F A,B E,F E,G A,D A,C C,E B,E D,G
1 1 3 3 3 4 5 5 7 10
8
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
9
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
10
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
11
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
12
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
13
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
14
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
15
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
16
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
17
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
18
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
19
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
20
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
21
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
22
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
23
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
24
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
25
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
26
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
27
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
28
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
29
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
30
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
31
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
32
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
33
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
34
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
35
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
36
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
37
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
38
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
39
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
40
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
41
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
42
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
43
A A
4 3 4 3
5 5
B B
D D
1 1
C 5 7 C 5 7
1 3 E
1 3 E
10 10
F F
3 3
G G
G.E[ ]
1 1 3 3 3 4 5 5 7 10
44
Prim’s Algorithm
Algorithm Prims_MST(Graph G, Vertex r)
New Set A[], Temp[]=G.V[]
For each v in G.V[]
v.cost= & v.previous=NULL
r.cost=o
While Temp[] is not Empty
u=Min(Temp[]) by cost
Temp[].Remove(u)
if u.previous≠NULL
A[].Add( (u,u.previous) )
for each v in adjacent(u)
if Temp[].Search(v)=True & w(u,v)<v.cost
v.cost=w(u,v) & v.previous=u
45
Prim’s Algorithm r
G
A
4 3
5
B
D
1
C 5 7
1 3 E
10
F
3
G
46
G
A
4 3
5
B
D
1
C 5 7
1 3 E
10
F
3
G
A[ ]={ }
Temp[ ]={A,B,C,D,E,F,G}
47
G ,
A
4 3 ,
, 5
D
, B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
48
G 0,
A
4 3 ,
, 5
D
, B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
49
G 0,
A
4 3 ,
, 5
D
, B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
50
G 0,
u A
4 3 ,
, 5
D
, B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
51
G 0,
u A
4 3 ,
, 5
D
, B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
Temp[ ]={B,C,D,E,F,G}
52
G 0,
u A
4 3 ,
, 5
D
, B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
53
G 0,
u A
4 3 ,
, 5
D
, B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
54
G 0,
u A
4 3 ,
, 5
D
, B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
55
G 0,
u A
4 3 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
56
G 0,
u A
4 3 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
57
G 0,
A
4 3 u 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
58
G 0,
A
4 3 u 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
Temp[ ]={C,D,E,F,G}
59
G 0,
A
4 3 u 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ } ,
Temp[ ]={C,D,E,F,G}
60
G 0,
A
4 3 u 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ AB } ,
Temp[ ]={C,D,E,F,G}
61
G 0,
A
4 3 u 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ AB } ,
Temp[ ]={C,D,E,F,G}
62
G 0,
A
4 3 u 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F ,
3
G
A[ ]={ AB } ,
Temp[ ]={C,D,E,F,G}
63
G 0,
A
4 3 u 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB } ,
Temp[ ]={C,D,E,F,G}
64
G 0,
A
4 3 u 3,A
4,A 5
D
5,A B
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB } ,
Temp[ ]={C,D,E,F,G}
65
G 0,
A
4 3 3,A
4,A 5
5,A B
u D
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB } ,
Temp[ ]={C,D,E,F,G}
66
G 0,
A
4 3 3,A
4,A 5
5,A B
u D
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB } ,
Temp[ ]={C,E,F,G}
67
G 0,
A
4 3 3,A
4,A 5
5,A B
u D
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB } ,
Temp[ ]={C,E,F,G}
68
G 0,
A
4 3 3,A
4,A 5
5,A B
u D
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD } ,
Temp[ ]={C,E,F,G}
69
G 0,
A
4 3 3,A
4,A 5
5,A B
u D
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD } ,
Temp[ ]={C,E,F,G}
70
G 0,
A
4 3 3,A
4,A 5
5,A B
u D
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD } ,
Temp[ ]={C,E,F,G}
71
G 0,
A
4 3 3,A
4,A 5
1,D B
u D
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD } 10,D
Temp[ ]={C,E,F,G}
72
G 0,
A
4 3 3,A
4,A 5
1,D B
u D
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD } 10,D
Temp[ ]={C,E,F,G}
73
G 0,
A
4 3 3,A
4,A 5
D u 1,D B
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD } 10,D
Temp[ ]={C,E,F,G}
74
G 0,
A
4 3 3,A
4,A 5
D u 1,D B
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD } 10,D
Temp[ ]={E,F,G}
75
G 0,
A
4 3 3,A
4,A 5
D u 1,D B
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD } 10,D
Temp[ ]={E,F,G}
76
G 0,
A
4 3 3,A
4,A 5
D u 1,D B
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD,DC } 10,D
Temp[ ]={E,F,G}
77
G 0,
A
4 3 3,A
4,A 5
D u 1,D B
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD,DC } 10,D
Temp[ ]={E,F,G}
78
G 0,
A
4 3 3,A
4,A 5
D u 1,D B
1
C 5 7
1 3
, E
10
F 7,B
3
G
A[ ]={ AB,AD,DC } 10,D
Temp[ ]={E,F,G}
79
G 0,
A
4 3 3,A
4,A 5
D u 1,D B
1
C 5 7
1 3
1,C E
10
F 5,C
3
G
A[ ]={ AB,AD,DC } 10,D
Temp[ ]={E,F,G}
80
G 0,
A
4 3 3,A
4,A 5
D u 1,D B
1
C 5 7
1 3
1,C E
10
F 5,C
3
G
A[ ]={ AB,AD,DC } 10,D
Temp[ ]={E,F,G}
81
G 0,
A
4 3 3,A
4,A 5
D u 1,D B
1
C 5 7
1 3
1,C E
10
F 5,C
3
G
A[ ]={ AB,AD,DC } 10,D
Temp[ ]={E,F,G}
82
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C u E
10
F 5,C
3
G
A[ ]={ AB,AD,DC } 10,D
Temp[ ]={E,F,G}
83
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C u E
10
F 5,C
3
G
A[ ]={ AB,AD,DC } 10,D
Temp[ ]={E,G}
84
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C u E
10
F 5,C
3
G
A[ ]={ AB,AD,DC } 10,D
Temp[ ]={E,G}
85
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C u E
10
F 5,C
3
G
A[ ]={ AB,AD,DC,CF } 10,D
Temp[ ]={E,G}
86
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C u E
10
F 5,C
3
G
Temp[ ]={E,G}
87
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C u E
10
F 5,C
3
G
Temp[ ]={E,G}
88
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C u E
10
F 3,F
3
G
Temp[ ]={E,G}
89
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C u E
10
F 3,F
3
G
Temp[ ]={E,G}
90
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3 u
1,C E
10
F 3,F
3
G
Temp[ ]={E,G}
91
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3 u
1,C E
10
F 3,F
3
G
Temp[ ]={G}
92
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3 u
1,C E
10
F 3,F
3
G
Temp[ ]={G}
93
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3 u
1,C E
10
F 3,F
3
G
A[ ]={ AB,AD,DC,CF,EF } 10,D
Temp[ ]={G}
94
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3 u
1,C E
10
F 3,F
3
G
Temp[ ]={G}
95
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3 u
1,C E
10
F 3,F
3
G
Temp[ ]={G}
96
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3 u
1,C E
10
F 3,F
3
G
A[ ]={ AB,AD,DC,CF,EF } 3,E
Temp[ ]={G}
97
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3 u
1,C E
10
F 3,F
3
G
Temp[ ]={G}
98
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C E
10
F 3,F
3
G
u
Temp[ ]={G}
99
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C E
10
F 3,F
3
G
u
Temp[ ]={ }
100
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C E
10
F 3,F
3
G
u
Temp[ ]={ }
101
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C E
10
F 3,F
3
G
u
A[ ]={ AB,AD,DC,CF,EF,EG } 3,E
Temp[ ]={ }
102
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C E
10
F 3,F
3
G
u
Temp[ ]={ }
103
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C E
10
F 3,F
3
G
u
Temp[ ]={ }
104
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C E
10
F 3,F
3
G
u
Temp[ ]={ }
105
G 0,
A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
1,C E
10
F 3,F
3
G
u
Temp[ ]={ }
106
G 0,
MST A
4 3 3,A
4,A 5
D
1,D B
1
C 5 7
1 3
New Graph MST 1,C E
10
MST.AddVertices(G.V[]) F 3,F
MST.AddEdges(A[]) 3
if |MST.E[]|≠|MST.V[]|-1 G
return NULL u
return MST
3,E
A[ ]={ AB,AD,DC,CF,EF,EG } 107

G 0,
MST A
A 4 3 3,A
4,A 5
D
1,D B
1
D
B C 5 7
C 1 3
New Graph MST 1,C E
10
E MST.AddVertices(G.V[]) F 3,F
F MST.AddEdges(A[]) 3
if |MST.E[]|≠|MST.V[]|-1 G
return NULL u
G
return MST
3,E

G 0,
MST A
A 4 3 3,A
3 4,A 5
4 1,D B
D
1
D
B C 5 7
1
C 1 3
New Graph MST 1,C E
1 3 10
3 if |MST.E[]|≠|MST.V[]|-1 G
return NULL u
G
return MST
3,E

G 0,
MST A
A 4 3 3,A
3 4,A 5
4 1,D B
D
1
D
B C 5 7
1
C 1 3
New Graph MST 1,C E
1 3 10
3 if |MST.E[]|≠|MST.V[]|-1 G
return NULL u
G
return MST
3,E

G 0,
MST A
A 4 3 3,A
3 4,A 5
4 1,D B
D
1
D
B C 5 7
1
C 1 3
New Graph MST 1,C E
1 3 10
3 if |MST.E[]|≠|MST.V[]|-1 G
return NULL u
G
return MST
3,E

112
ALGORITHMS
Lecture # 40, 41 & 42

DYNAMIC PROGRAMMING
& SHORTEST PATH PROBLEM
A(3)
Example 1
A(2) P(3) A(2)
A( n )
{
A(1) P(2) A(1) A(1) P(2) A(1)
If n >= 1
A(n-1);
A(n-1);
A(0) 1 call
} A(1) 3 calls
A(2) 7 calls
A(3) 15 calls
A(n) m calls
2
A(3)
Example 1
A(2) P(3) A(2)
A( n )
{
A(1) P(2) A(1) A(1) P(2) A(1)
If n >= 1
A(n-1);
A(n-1);
A(0) 1 = 20+1 ― 1 call
} A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
S(n) = k(2n+1-1) = O(2n)
A(n) m = 2n+1 ― 1 calls

3
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(n) m = 2n+1 ― 1 calls

4
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
5
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
6
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
7
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
8
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
9
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
10
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
11
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
12
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
13
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
14
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
15
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
16
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
17
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
18
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213
A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
19
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
20
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
21
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
22
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12131
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
23
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12131
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
24
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
12131
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
25
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121312
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
26
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(0) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121312
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
27
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
121312
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
28
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
29
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
30
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
31
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
32
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
33
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) A(0) A(0) A(0)

A(1) A(1) A(1) A(1) A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
1213121
A(2) A(2)
A(3) 15 = 23+1 ― 1 calls
A(3)
A(n) m = 2n+1 ― 1 calls
34
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
You can see that for an A(1) 3 = 21+1 ― 1 calls
Input of size n, we need A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
A(n) m = 2n+1 ― 1 calls
35
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
If each block of stack A(1) 3 = 21+1 ― 1 calls
consists of k memory A(2) 7 = 22+1 ― 1 calls
A(3) 15 = 23+1 ― 1 calls
space required is k(n+1) A(n) m = 2n+1 ― 1 calls
36
A(3) A( n )
{
Example 1 If n >= 1
A(2) P(3) A(2) A(n-1);
print n
A(n-1);
A(1) P(2) A(1) A(1) P(2) A(1) }
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
A(0) 1 = 20+1 ― 1 call
A(1) 3 = 21+1 ― 1 calls
A(2) 7 = 22+1 ― 1 calls
S(n) = k(n+1) = O(n) A(3) 15 = 23+1 ― 1 calls
A(n) m = 2n+1 ― 1 calls

37
A( n )
{
A(n-1);
print n
T(n) = 2T(n-1) + c; n>=1
A(n-1);
T(0) = c  This is the Anchor Condition }
-------------------------------------------------------------------------
T(n) =2T(n-1)+c …eq(i)
T(n-1) =2T(n-1-1)+c
=2T(n-2)+c …eq(ii)
T(n-2) =2T(n-2-1)+c
=2T(n-3)+c …eq(iii)
38
A( n )
{
Putting eq(ii) in eq(i): T(n) =2T(n-1)+c …eq(i) A(n-1);
T(n-1) =2T(n-2)+c …eq(ii) print n
T(n) =2T(n-1)+c T(n-2) =2T(n-3)+c …eq(iii)
A(n-1);
=2[2T(n-2)+c] +c }
=22T(n-2)+2c+c …eq(iv)
T(n) =22[2T(n-3)+c]+2c+c
=23T(n-3)+22c+2c+c
T(n) =2kT(n-k)+2k-1c+…+22c+2c+c ..eq(v)
39
A( n )
{
As we know that T(0) will be the last recursive call, so A(n-1);
for what value of k will T(n-k) become T(0). print n
A(n-1);
n-k=0  n=k
}
T(n) = 2kT(n-k)+2k-1c+…+22c+2c+c
= 2nT(0)+2n-1c+…+22c+2c+c
= 2nc+2n-1c+…+22c+2c+c
= c(2n+2n-1+…+22+2+1)
= c(2n+1 ― 1)
= O(2n) 40
A( n )
{
Our Approach If n >= 1
A(n-1);
S(n) = O(n) A(3)
print n
A(n-1);
T(n) = O(2n)
}
A(2) P(3) A(2)
A(1) P(2) A(1) A(1) P(2) A(1)
A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0) A(0) P(1) A(0)
41
A( n )
{
A(n-1);
print n
A(3) A(n-1);
}
A(2)
A(1) 0 1 2 3
Ø Ø Ø Ø
A(0)
42
A( n )
{
A(n-1);
print n
A(3) A(n-1);
}
A(2)
A(1) 0 1 2 3
<blank> Ø Ø Ø
A(0) P(1) M(0)
43
A( n )
{
A(n-1);
print n
A(3) A(n-1);
}
A(2)
A(1) P(2) M(1) 0 1 2 3
<blank> 1 Ø Ø
A(0) P(1) M(0)
44
A( n )
{
A(n-1);
print n
A(3) A(n-1);
}
A(2) P(3) M(2)
A(1) P(2) M(1) 0 1 2 3
<blank> 1 121 Ø
A(0) P(1) M(0)
45
A( n )
{
A(n-1);
S(n) = O(n) + 2(n+1) print n
= O(n) A(3) A(n-1);
T(n) = (n+1).c }
= O(n) A(2) P(3) M(2)
A(1) P(2) M(1) 0 1 2 3
<blank> 1 121 1213121
A(0) P(1) M(0)
46
Dynamic Programming
■ Dynamic programming is essentially recursion without
repetition.
■ Developing a dynamic programming algorithm generally
involves two separate steps:
– Formulate problem recursively. Write down a formula for the
whole problem as a simple combination of answers to smaller
sub-problems.
– Build solution to recurrence from bottom up. Write an
algorithm that starts with base cases and works its way up to
the final solution.
Dynamic Programming
■ Decompose into sub-problems like divide and conquer.
■ Sub-problems are dependant.
■ Record results of smaller sub-problems. Dynamic
programming algorithms need to store the results of
intermediate sub-problems.
■ Re-use it for further occurrence.
■ Mostly reduces complexity exponential to polynomial.
In our case, it reduced complexity from exponential time to linear time.
Dynamic Programming
■ It's especially good and intended for optimization problems.
■ In optimization problems, you want to minimize, maximize
something, things like shortest paths.
■ In optimization problems, you want to find the best way to do
something.
■ Typically good algorithms to solve them involve dynamic
programming.
49
DP is Careful Brute Force
■ You can also think of dynamic programming as a kind of
exhaustive search. which is usually a bad thing to do because it
leads to exponential time.
■ But if you do it in a clever way, via dynamic programming, you
typically get polynomial time.
■ So according to this perspective, dynamic programming is
approximately ‘careful brute force’.
50
Literal Meaning of DP
■ The notion ‘Dynamic’ means ‘opposite of static’.
■ While the word ‘Programming’ refers to ‘Optimization’ or
‘Planning’
Dynamic Planning to dynamically


Programming optimize something
51
1
Example 2
2 6
V(n) = f ( V(n-3), V(n-4), V(n-5) )

3 7 11
Properties:
1. Recursive in nature 4 8
2. Dividing into subproblems
3. Overlapping subproblems
4. Optimal Substructure 5
52
Example 3
■ Fibonacci Numbers:
– 1 1 2 3 5 8 13 21 34 55 …
■ Formula:
 Fn = Fn-1 + Fn-2
 F 1 = F2 = 1
Or
 F(n) = F(n-1) + F(n-2)

 F(1) = F(2) = 1
53
Example 3:
Naïve Recursive Algorithm
fib(n):
if n<3: f=1
else: f=fib(n-1) + fib(n-2)
return f
■ Time Complexity:
 T(n) = T(n-1) + T(n-2) + (1)

 T(1) = T(2) = (1)
54
Example 3:
Naïve Recursive Algorithm
 T(n) = T(n-1) + T(n-2) + (1)
 T(1) = T(2) = (1)
■ Solution:
T(n) > T(n-2) + T(n-2) + (1)
=>T(n) > 2T(n-2) + (1)
 T(n) > (2n/2)
So, growth is exponential.

55
Example 3:
Memoized DP Algorithm
initiate memo as EMPTY
fib(n):
if n is memo: return memo[n]
if n<3: f=1
else: f=fib(n-1) + fib(n-2)
memo[n] = f
return f
56
Example 3:
fib(n)
fib(n-1) fib(n-2)
fib(n-2) fib(n-3) fib(n-3) fib(n-4)
fib(n-3) fib(n-4) fib(n-4) fib(n-5) fib(n-4) fib(n-5) fib(n-5) fib(n-6)
fib(n-4)
57
Example 3:
fib(n)
fib(n-1) fib(n-2)
fib(n-3) fib(n-4) fib(n-4) fib(n-5) fib(n-4) fib(n-5) fib(n-5) fib(n-6)
fib(n-4)
58
Example 3:
fib(n)
fib(n-1) fib(n-2)
fib(n-3) fib(n-4) fib(n-4) fib(n-5) fib(n-4) fib(n-5)
fib(n-4)
59
Example 3:
fib(n)
fib(n-1) fib(n-2)
fib(n-3) fib(n-4)
fib(n-4)
60
Example 3:
fib(n)
fib(n-1) fib(n-2)  T(n) = T(n-1) + (1)

 T(1) = T(2) = (1)
fib(n-2) fib(n-3)
■ Solution:
T(n) = T(n-1) + (1)
fib(n-3) fib(n-4) = (n)
fib(n-4) So, growth is linear.

61
Example 3:
Bottom-up DP Algorithm
fib(n):
initiate array[1…n] to EMPTY
for i = 1 to n
if i<3: f=1
else: f=array[i-1] + array[i-2]
array[i] = f
return array[n]
62
Example 3:
Bottom-up DP Algorithm
fib(n):
initiate array[1…n] to EMPTY ---> O(n)
for i = 1 to n ---> O(1)+O(n)+O(n) + c = O(n)
if i<3: f=1 ---> 2c = O(1)
else: f=array[i-1] + array[i-2] ---> c.(n-2) = O(n)
array[i] = f ---> cn = O(n)
return array[n] ---> O(1)
63
Class Tasks 8 & 9
8. Write a C++ code to implement the BOTTOM
UP version of the example 1 discussed in the
beginning of lecture.
9. Write the pseudocode of the MEMOIZED

version of the same example.
■ Instructions:
– Upload a single “.CPP” file for Class Task 8 and a
single “.DOCX” file for Class Task 9 by 04 February,
2021 on LMS.
64
Shortest Path Problem (Single Source)
s
■ Here, again, it is an optimization problem. G
A
4 3
■ We can write at least three algorithms for it: 5
B
– Naïve Recursive Algorithm D
1
– Memoized DP Algorithm C 4 7
– Bottom-up DP Algorithm 1 2 E
10
■ I will discuss the necessary basics here, and F
leave the rest on you as an ungraded

assignment. G
65
s
■ NAÏVE Algorithm G
– Tool: GUESSING A
– We will guess the right answer. 4 3

– Concept: 5
B
■ Try guessing the right answer. D
1
■ Don’t just try any guess. C 4 7
■ Try all of them.
1 2 E
■ Take the best one.
10
F
– This is the BRUTE FORCE part.
– In the DP version of this problem, you will
CAREFULLY use it by avoiding unnecessary G
recursions by memoization.
66
■ Applying the GUESSING principle on the s
G
shortest path problem. A
– Look at all the nodes from where we can reach 4 3

‘d’ and them look at the shortest path from ‘s’ 5
B
to there. D
1
– So, there is some hypothetical shortest path C 4 7
from ‘s’ to ‘d’. 1 2 E
– We don’t know from where we can reach ‘d’. It
may be any incoming edge to ‘d’. What to do
10
F
then?
– Answer: We will GUESS. Or more precisely, we d
will try all possible guesses. G
67
■ Applying the GUESSING principle on the s
G
shortest path problem. A
– After that recursively compute the shortest 4 3

path from ‘s’ to ‘d’. 5
B
D
D(s,d) = D(s,u) + w(u,d) 1
C 4 7
– Here, (u,v) is our current GUESS and ‘u’ is an
adjacent node of ‘d’ from where we can reach 1 2 E
here. 10
F
– But may be our guess is wrong!!! So, we will
find the minimum of all possible guesses.
d
D(s,d) = Min[D(s,u) + w(u,d) : (u,d)E] G
68
D(s,d) = Min[w(u,d) + D(s,u) : (u,v)E] s
D(s,s) = 0
G
A
4 3
D(A,G) = Min[ 10 + D(A,D)] 5
B
D(A,D) = Min[ 4 + D(A,A) or 1 + D(A,C)] D
1
D(A,A) = 0 C 4 7
D(A,C) = Min [ 5 + D(A,A)] 1 2 E
D(A,A) = 0 10
F
D(A,C) = 5
D(A,D) = Min[4 or 6] = 4 G
d
D(A,G) = 10 + 4 = 14 69
■ The same will be applied for all nodes. s
G
A
■ So in the worst case, there can be (n-1) 3
incoming edges to each node. 4
5
B
■ Hence, its time complexity will be D
1
exponential. C 4 7
1 2 E
10
NOT PRACTICAL IN F
BIG GRAPHS. G
d
70
s
■ MEMOIZED Algorithm G
A
– To make algorithms better, we learnt to
memorize. 4 3
– Memoized function is the same Brute Force
5
B
D
recursive algorithm: 1
C 4 7
D(s,d) = Min[w(u,d) + D(s,u) : (u,d)E]
1 2 E
D(s,s) = 0
10
– But in the beginning of the function definition, F
we check if the value of D(s,d) is already in the
memo or not.
G
– So, do we have a better algorithm, now???
71
s
■ Limitation of our Algorithm G
A
– Remember, we are only considering Directed
Acyclic Graphs, DAGs here. 4 3
– It will not work for Cyclic Graphs.
5
B
D
1
C 4 7
■ Ungraded Class Tasks 1
– Considering the above statement, show that our
2 E
algorithm is not correct for Directed Cyclic 10

F
Graphs, DCGs. It is rather an infinite algorithm.
72
73
ALGORITHMS
Lecture # 43 & 44
ShorteSt Path Problem,
DijkStra’S algorithm
Shortest Path Problem
■ The shortest path problem is about finding a path
between 2 vertices in a graph such that the total sum of the
edges weights is minimum.
■ Dijkstra’s Algorithm finds the minimal distance from the source

vertex to all other vertices in the graph.
■ This information can be used to get the shortest path from the
source to any other node.
2
Dijkstra’s Algorithm
Algorithm Dijkstra(Graph G, Vertex s)
New Set Temp[]=G.V[]
For each v in G.V[]
v.distance= & v.previous=NULL
s.distance=o
While Temp[] is not Empty
u=Min(Temp[]) by distance
Temp[].Remove(u)
for each v in adjacent(u)
if v.distance > u.distance + w(u,v)
v.distance = u.distance + w(u,v)
v.previous = u
3
Dijkstra’s Algorithm G
A
4 3
5
D s B
1
C 5 7
1 3 E
10
F
3
G
4
Dijkstra’s Algorithm G
A
4 3
5
D s B
1
C 5 7
1 3 E
10
F
3
G
5
Dijkstra’s Algorithm G ,
A
4 3 ,
, 5
D s , B
1
C 5 7
1 3
, E
10
F ,
3
G
,
6
A
4 3 ,
, 5
D s 0, B
1
C 5 7
1 3
, E
10
F ,
3
G
,
7
A
4 3 ,
, 5
D s 0, B
1
C 5 7
1 3
, E
10
F ,
3
G
,
8
A
4 3 ,
, 5
D s 0, B
1u
C 5 7
1 3
, E
10
F ,
3
G
,
9
A
4 3 ,
, 5
D s 0, B
1u
C 5 7
1 3
, E
10
F ,
3
G
,
Temp[ ]={A,B,D,E,F,G}
10
A
4 3 ,
, 5
D s 0, B
1u
C 5 7
1 3
, E
10
F ,
3
G
,
11
A
4 3 ,
, 5
D s 0, B
1u
C 5 7
1 3
, E
10
F ,
3
G
,
12
Dijkstra’s Algorithm G 5,C
A
4 3 ,
1,C 5
D s 0, B
1u
C 5 7
1 3
1,C E
10
F 5,C
3
G
,
13
A
4 3 ,
1,C 5
D s 0, B
1u
C 5 7
1 3
1,C E
10
F 5,C
3
G
,
14
A
4 3 ,
1,C 5
u D s 0, B
1
C 5 7
1 3
1,C E
10
F 5,C
3
G
,
15
A
4 3 ,
1,C 5
u D s 0, B
1
C 5 7
1 3
1,C E
10
F 5,C
3
G
,
Temp[ ]={A,B,E,F,G}
16
A
4 3 ,
1,C 5
u D s 0, B
1
C 5 7
1 3
1,C E
10
F 5,C
3
G
,
Temp[ ]={A,B,E,F,G}
17
A
4 3 ,
1,C 5
u D s 0, B
1
C 5 7
1 3
1,C E
10
F 5,C
3
G
,
Temp[ ]={A,B,E,F,G}
18
A
4 3 ,
1,C 5
u D s 0, B
1
C 5 7
1 3
1,C E
10
F 5,C
3
G
11,D
Temp[ ]={A,B,E,F,G}
19
A
4 3 ,
1,C 5
u D s 0, B
1
C 5 7
1 3
1,C E
10
F 5,C
3
G
11,D
Temp[ ]={A,B,E,F,G}
20
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10 u
F 5,C
3
G
11,D
Temp[ ]={A,B,E,F,G}
21
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10 u
F 5,C
3
G
11,D
Temp[ ]={A,B,E,G}
22
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10 u
F 5,C
3
G
11,D
Temp[ ]={A,B,E,G}
23
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10 u
F 5,C
3
G
11,D
Temp[ ]={A,B,E,G}
24
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10 u
F 4,F
3
G
11,D
Temp[ ]={A,B,E,G}
25
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10 u
F 4,F
3
G
11,D
Temp[ ]={A,B,E,G}
26
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3 u
1,C E
10
F 4,F
3
G
11,D
Temp[ ]={A,B,E,G}
27
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3 u
1,C E
10
F 4,F
3
G
11,D
Temp[ ]={A,B,G}
28
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3 u
1,C E
10
F 4,F
3
G
11,D
Temp[ ]={A,B,G}
29
A
4 3 ,
1,C 5
D s 0, B
1
C 5 7
1 3 u
1,C E
10
F 4,F
3
G
11,D
Temp[ ]={A,B,G}
30
A
4 3 11,E
1,C 5
D s 0, B
1
C 5 7
1 3 u
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={A,B,G}
31
A
4 3 11,E
1,C 5
D s 0, B
1
C 5 7
1 3 u
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={A,B,G}
32
A
u
4 3 11,E
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={A,B,G}
33
A
u
4 3 11,E
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={B,G}
34
A
u
4 3 11,E
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={B,G}
35
A
u
4 3 11,E
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={B,G}
36
A
u
4 3 8,A
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={B,G}
37
A
u
4 3 8,A
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={B,G}
38
A
4 3 8,A
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
u
7,E
Temp[ ]={B,G}
39
A
4 3 8,A
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
u
7,E
Temp[ ]={B}
40
A
4 3 8,A
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
u
7,E
Temp[ ]={B}
41
A
4 3 8,A
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
u
7,E
Temp[ ]={B}
42
A
4 3 8,A
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
u
7,E
Temp[ ]={B}
43
A
4 3 8,A
1,C 5
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
u
7,E
Temp[ ]={B}
44
A
4 3 8,A
1,C 5 u
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={B}
45
A
4 3 8,A
1,C 5 u
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={ }
46
A
4 3 8,A
1,C 5 u
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={ }
47
A
4 3 8,A
1,C 5 u
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={ }
48
A
4 3 8,A
1,C 5 u
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={ }
49
A
4 3 8,A
1,C 5 u
D s 0, B
1
C 5 7
1 3
1,C E
10
F 4,F
3
G
7,E
Temp[ ]={ }
50
A
4 3 8,A
Vertex Distance Previous 1,C 5 u
A 5 C D s 0, B
1
B 8 A C 5 7
C 0  1 3
1,C E
D 1 C 10
F 4,F
E 4 F
3
F 1 C
G
G 7 E
7,E
51
A
4 3 8,A
A 5 C D s 0, B
1
B 8 A C 5 7
C 0  1 3
1,C E
D 1 C 10
F 4,F
E 4 F
3
F 1 C
G
G 7 E
7,E
Distance(C,G):
52
A
4 3 8,A
A 5 C D s 0, B
1
B 8 A C 5 7
C 0  1 3
1,C E
D 1 C 10
F 4,F
E 4 F
3
F 1 C
G
G 7 E
7,E
Distance(C,G): G
53
A
4 3 8,A
A 5 C D s 0, B
1
B 8 A C 5 7
C 0  1 3
1,C E
D 1 C 10
F 4,F
E 4 F
3
F 1 C
G
G 7 E
7,E
Distance(C,G): E→ G
54
A
4 3 8,A
A 5 C D s 0, B
1
B 8 A C 5 7
C 0  1 3
1,C E
D 1 C 10
F 4,F
E 4 F
3
F 1 C
G
G 7 E
7,E
Distance(C,G): F→ E→ G
55
A
4 3 8,A
A 5 C D s 0, B
1
B 8 A C 5 7
C 0  1 3
1,C E
D 1 C 10
F 4,F
E 4 F
3
F 1 C
G
G 7 E
7,E
Distance(C,G): C→ F→ E→ G
56
A
4 3 8,A
A 5 C D s 0, B
1
B 8 A C 5 7
C 0  1 3
1,C E
D 1 C 10
F 4,F
E 4 F
3
F 1 C
G
G 7 E
7,E
57
Dijkstra’s Algorithm
Vertex Distance Previous

A 5 C
B 8 A C
C 0  1 3 E
D 1 C
F
E 4 F
3
F 1 C
G
G 7 E
58
QUIZ #3
■ Time: 03:50 pm to 04:40 pm
59
60
ALGORITHMS
Lecture # 45 & 46
IntroductIon to computatIonal
complEXItY claSSES
Topics: Computational Complexity
■ P, EXP & R Classes
■ Most problems are not computable
■ NP Class
■ P vs. NP?
■ Hardness & Completeness
■ Reductions
2
P, EXP & R Classes R
EXP
P = {problems solvable in << polynomial time} P
nc
EXP = {problems solvable in << exponential time}
2n^c
R = {problems solvable in some finite time}
EXP
Not Solvable
R
P 3
Examples
Tetris
 Negative –weight cycle detection  P &  EXP

 N x N chess  EXP but  P
 Tetris  EXP and ?(,)P
 The Halting Problem  R
4
Most Decision Problems  R
PROOF:
finite
 a program ≈ a binary string ≈ a natural number  n
 a decision problem ≈ a function from input to Yes/No
Input: a binary string ≈ a natural number  n
Output: Yes/No or 0/1
≈ a function from n to {0,1}
0 1 2 3 4 5 6 7 8 …
0 0 1 0 1 1 0 0 1 …
5
PROOF:
 a program ≈ a finite binary string ≈ a natural number  n

 a decision problem ≈ a function from input to Yes/No
Input: a binary string ≈ a natural number  n
Output: Yes/No or 0/1
≈ a function from n to {0,1} ≈ an infinite binary string  r
0 1 2 3 4 5 6 7 8 …
. 0 0 1 0 1 1 0 0 1 …
6
PROOF:
 a decision problem  r
 a program  n
|r| >> |n|
Hence, almost every problem is unsolvable by any program.
7
NP Class
NP = {decision problems solvable in << polynomial time
via a ‘lucky’ algorithm}
A nondeterministic model: Algorithm makes guesses & says Y/N.
Guesses are guaranteed to lead to a ‘Yes’ answer, if possible.
EXP
Not Solvable
P R
Tetris Halting
Problem
NP 8
NP Class
NP = {decision problems solvable in << polynomial time
via a ‘lucky’ algorithm}
A nondeterministic model: Algorithm makes guesses & says Y/N.
Guesses are guaranteed to lead to a ‘Yes’ answer, if possible.
No
No
Yes
9
NP = {decision problems solvable in
<< polynomial time via a ‘lucky’ algorithm}
Tetris  NP
The decision problem “Can I Survive?” is in NP.
10
NP Class
NP = {decision problems with “solutions” that can be “checked”
or “verified” in << polynomial time}
Whenever the answer is ‘Yes’, it can be proved and its proof can
be verified in polynomial time.
EXP
Not Solvable
P R
Tetris Halting
Problem
NP 11
NP = {decision problems with “solutions” that can be
“checked” or “verified” in << polynomial time}
Verify !!??
Consider a following scenario:
Khalid: I am ‘lucky’. I am really good at playing Tetris.

Anwar: I am not ‘convinced’.
Khalid: Give me a problem and I can prove it.

Anwar gives him a really hard Tetris problem.
Anwar: Can you survive these pieces?

Khalid: Yes, I can survive?
12
NP = {decision problems with “solutions” that can be
“checked” or “verified” in << polynomial time}
Verify !!??
So, PROOF is a sequence of steps that Khalid makes.
Anwar can then CHECK/VERIFY that the sequence works or not by

implementing the rules of Tetris in polynomial time.
NP = {decision problems with “solutions” that can be “checked”

or “verified” in << polynomial time}
Whenever the answer is ‘Yes’, it can be proved and its proof can be
verified in polynomial time.
13
P=NP ???
It is just a conjecture. Not proved yet.
General Consensus: R
P≠NP EXP
Some Mathematicians:
NP
P=NP
P
Millennium Prize Problem
$ 1 000 000 14
NP-HARD
If P ≠ NP,
Then Tetris  NP-P
So, Tetris  P
So, it is one of the hardest problems in NP.
Tetris is NP-HARD.
 As hard as every problem in NP.
15
NP-HARD
If P = NP,
Then Tetris  P
But, it is still in the right extreme of P and NP.
So, it is still one of the hardest problems in P and NP.
Thus, Tetris is still NP-HARD.
16
NP-Complete
Tetris is NP-COMPLETE:
 Tetris is NP-HARD.
 Tetris is in NP.
NP  NP-Hard = NP-COMPLETE
17
NP-Complete
EXP-Complete
NP-Complete
NP-Hard
EXP-Hard
P = NP ??? NP=EXP ??? Not Solvable
R
P Halting
Tetris Chess
NP Problem
EXP
18
Reductions Problem A  Problem B
Reductions are a way to design algorithms.

 Graph reductions.
Examples:
 Unweighted shortest path  Dijkstra’s WSP Problem
 <see next slide>
19
Lets consider a game called “Number Scrabble”
P1 P2 P2 P1
First, player 1 will pick a number from 1 to 9.

Then player 2 will pick a number.
The aim of the game is to pick 3 such numbers that add up to 15.
Whoever does this first, wins. If all numbers are picked and no player
has exactly 15, the game is a draw. 20
Let us reframe the game now and arrange them in a square.

Now, these numbers are arranged in a very
specific order.
Every triplet adds to 15.
This is called as a “Magic Square”
This type of arrangement looks very familiar.
Remember the “Tic-Tac-Toe” game.
21
O X X
O X
Similarly, a lot of games can be reduced to the exact same game.

22
O X X
O X
And lot of problems in math and computer science can be reduced

to the exact same problem. 23
‫‪End of COURSE‬‬
‫لی ٰذلِ َ‬
‫ک‬ ‫َ‬
‫ع‬ ‫ِ‬ ‫ﱣ‬ ‫ِ‬ ‫ُ‬
‫د‬ ‫ْ‬
‫م‬ ‫َ‬
‫ح‬ ‫ْ‬
‫ال‬ ‫َ‬
‫ف‬
‫ٰ‬
‫‪24‬‬

Slides

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Slides

Uploaded by

Copyright:

Available Formats

DESIGN & ANALYSIS OF

■ Introduction to The Design & Analysis of

■ Computers and Intractability, Guide to the

■ A computer algorithm is a detailed step-by-step method for solving

■ A well-formulated problem is unambiguous and precise, leaving

■ In other words, algorithm is the method of translating the inputs

■ If algorithm is not of finite length, then we do not have infinite

■ BUT… no matter how much testing

2. Iterative Algorithms 8. Greedy Algorithms

3. Divide & Conquer Algorithms 9. Brute Force Algorithms

■ It is not the actual running time. It is not the standard.

■ Low-level details will not be considered

■ But with all these weaknesses, our model is not so bad

– Comparison not the absolute analysis of any algorithm.

■ Exact, not asymptotic, measure of efficiency can be sometimes

Let X = a universal set, A  X, B  X

■ Permutation: If all elements of a finite sequence are distinct, that

■ Some times, A and B are of same set, e.g.,

Definition: A subset of X x Y is a relation over X x Y

Lemma 1: if |X| = m, |Y| = n then

Algorithm: All binary relations, cardinality of it

Not good algorithms

Note: Submit your answers on LMS by 12 Nov, 2020.

■ May I come in? Not a statement

■ Logic is interesting. A statement

■ It is hot today. A statement

■ Here, AND, OR, NOT are called LOGICAL CONNECTIVES.

Note: Some books used “” symbol for negation.

– p : “Islamabad is the capital of Pakistan”

– p q : “Islamabad is the capital of Pakistan and 17 is divisible by 3”

– p q : “Islamabad is the capital of Pakistan or 17 is divisible by 3”

– ~p : “It is not the case that Islamabad is the capital of Pakistan”

Let p = “It is hot”, and q = “ It is sunny”

■ It has opposite truth value from p i.e., if p is true, then ~ p is false;

■ In other words, we will fill T in the first row of conjunction and F in

■ TASK: Prove That

Note:Submit your answers on LMS by 16 Nov, 2020.

■ 6) Double Negation Law

■ 7) Idempotent (something that remains same) Laws

■ 10) Absorption Laws

■ 11) Negation of t and c

■ p → q is false when p is true and q is false; otherwise it is true.

■ The arrow "→ " is the conditional operator.

■ In p → q, the statement p is called the hypothesis (or antecedent) and

■ It is denoted p↔q. “if and only if” is abbreviated as iff.

■ The double headed arrow " ↔" is the biconditional operator.

■ If you will attend the classes, then you will be allowed

■ If you will attend the classes, you will be allowed to

■ You will be allowed to participate in the exams if you

■ You will be allowed to participate in the exams when

■ You will be allowed to participate in the exams

■ You will be allowed to participate in the exams follows from

■ You will attend the classes implies that you will be

■ You will attend the classes is sufficient to allow you to

■ A sufficient condition for q is p.

■ A sufficient condition to allow you to participate in the

■ To allow you to participate in the exams is necessary if

■ A necessary condition for p is q.

■ A necessary condition to attend the classes is to allow

■ You will attend the classes only if you will be allowed to

■ Propositional language is an artificial language; we only parallel

■ The if-then construction used in many programming languages is

■ It is denoted p↔q. “if and only if” is abbreviated as iff.