Divide-and-Conquer Algorithms

and Recurrence Relations

Divide-and-Conquer Algorithmic Paradigm
Definition: A divide-and-conquer algorithm works by first dividing a
problem into one or more instances of the same problem of smaller
size and then conquering the problem using the solutions of the
smaller problems to find a solution of the original problem.
Examples:
• Binary search
• Merge sort
 Divide-and-Conquer Recurrence Relations

• Suppose that a recursive algorithm divides a problem of size n into a

subproblems.
• Assume each subproblem is of size n/b.
• Suppose g(n) extra operations are needed in the conquer step.
• Then f(n) represents the number of operations to solve a problem of
size n satisisfies the following recurrence relation:
f(n) = af(n/b) + g(n)
• This is called a divide-and-conquer recurrence relation.
Example: Binary Search
• Binary search reduces the search for an element in a sequence of size n to
the search in a sequence of size n/2. Two comparisons are needed to
implement this reduction;
• one to decide whether to search the upper or lower half of the sequence and
• the other to determine if the sequence has elements.
• Hence, if f(n) is the number of comparisons required to search for an
element in a sequence of size n, then

f(n) = f(n/2) + 2
when n is even.
Example: Merge Sort
• The merge sort algorithm splits a list of n (assuming n is even) items
to be sorted into two lists with n/2 items. It uses fewer than n
comparisons to merge the two sorted lists.
• Hence, the number of comparisons required to sort a sequence of
size n, is no more than than M(n) where

M(n) = 2M(n/2) + n.
Example: Fast Multiplication of Integers
• An algorithm for the fast multiplication of two 2n-bit integers (assuming n is even) first splits each of the 2n-bit integers into two blocks, each of n bits.
• Suppose that a and b are integers with binary expansions of length 2n. Let
a = (a2n−1a2n−2 … a1a0)2 and b = (b2n−1b2n−2 … b1b0)2 .
• Let a = 2nA1 + A0, b = 2nB1 + B0 , where
A1 = (a2n−1 … an+1an)2 , A0 = (an−1 … a1a0)2 ,
B1 = (b2n−1 … bn+1bn)2 , B0 = (bn−1 … b1b0)2.
• The algorithm is based on the fact that ab can be rewritten as:
ab = (22n + 2n)A1B1 +2n (A1−A0)(B0 − B1) +(2n + 1)A0B0.
• This identity shows that the multiplication of two 2n-bit integers can be carried out using three multiplications of n-bit integers, together with additions, subtractions, and shifts.
• Hence, if f(n) is the total number of operations needed to multiply two n-bit integers, then

f(2n) = 3f(n) + Cn

where Cn represents the total number of bit operations; the additions, subtractions and shifts that are a constant multiple of n-bit operations.
Estimating the Size of Divide-and-Conquer
Functions
Theorem 1: Let f be an increasing function that
satisfies the recurrence relation
f(n) = af(n/b) + cnd
whenever n is divisible by b, where a≥ 1, b is an
integer greater than 1, and c is a positive real number.
Then

Furthermore, when n = bk and a ≠1, where k is a

positive integer,

where C1 = f(1) + c/(a−1) and C1 = −c/(a−1).

Complexity of Binary Search
Binary Search Example: Give a big-O estimate for the number of
comparisons used by a binary search.
Solution: Since the number of comparisons used by binary search is
f(n) = f(n/2) + 2 where n is even, by Theorem 1, it follows that f(n) is O(log n).
Estimating the Size of Divide-and-conquer
Functions (continued)
Theorem 2. Master Theorem: Let f be an increasing
function that satisfies the recurrence relation
f(n) = af(n/b) + cnd
whenever n = bk, where k is a positive integer
greater than 1, and c and d are real numbers with c
positive and d nonnegative. Then
Complexity of Merge Sort
Merge Sort Example: Give a big-O estimate for the number of
comparisons used by merge sort.
Solution: Since the number of comparisons used by merge sort to
sort a list of n elements is less than M(n) where M(n) = 2M(n/2) + n, by
the master theorem M(n) is O(n log n).
Complexity of Fast Integer Multiplication
Algorithm
Integer Multiplication Example: Give a big-O estimate for the number of bit
operations used needed to multiply two n-bit integers using the fast
multiplication algorithm.
Solution: We have shown that f(n) = 3f(n/2) + Cn, when n is even, where f(n) is
the number of bit operations needed to multiply two n-bit integers. Hence by the
master theorem with a = 3, b = 2, c = C, and d = 0 (so that we have the case
where a > b ), it follows that f(n) is O(nlog 3).
d

Note that log 3 ≈ 1.6. Therefore the fast multiplication algorithm is a substantial
improvement over the conventional algorithm that uses O(n2) bit operations.
 Discrete Mathematics
Advanced Counting Techniques
Discrete Mathematics
 Generating Functions
Inclusion-Exclusion

8.4: Generating Functions

Discrete Mathematics
 Generating Functions
Inclusion-Exclusion

Generating functions
Definition
The generating function for the sequence a0, a1, ...,ak, ... of real
numbers is the infinite series
Σ∞
G (x) = a0 + a1x + · · · + ak x k + · · · = ak x k .
k=0
Discrete Mathematics
 Generating Functions
Inclusion-Exclusion

Generating functions
Definition
The generating function for the sequence a0, a1, ...,ak, ... of real
numbers is the infinite series
Σ∞
G (x) = a0 + a1x + · · · + ak x k + · · · = ak x k .
k=0
Discrete Mathematics

Remark
We can define generating functions for finite sequences of real
numbers by extending a finite sequence a0, a1, . . . , an into an
infinite sequence by setting an+1 = 0, an+2 = 0, and so on. The
generating function G (x ) of this infinite sequence {a n } is a
polynomial of degree n:

G(x) = a0 + a1x + · · · + anxn.

 Generating Functions
Inclusion-Exclusion

Generating functions

Example
Let m be a positive integer. Let ak = C (m, k) for
k = 0, 1, 2,. . . , m. What is the generating function for the
sequence a0, a1, ...,am?
Prof. Steven Evans Discrete Mathematics
 Generating functions
• Example
• Let m be a positive integer. Let ak = C (m, k)for
• k = 0, 1, 2,. . . , m. What is the generating function
for the sequence a0, a1, ...,am?
• The generating function for this sequence is
Prof. Steven Evans Discrete Mathematics

. Σ . Σ . Σ . Σ
m m m 2 m m
G(x) = + x+ x +· · · + x
0 1 2 m
 Generating functions
• Example
• Let m be a positive integer. Let ak = C (m, k)for
• k = 0, 1, 2,. . . , m. What is the generating function for the sequence
a0, a1, ...,am?
• The generating function for this sequence is
Prof. Steven Evans Discrete Mathematics

. Σ . Σ . Σ . Σ
m m m 2 m m
G(x) = + x+ x +· · · + x = (1 + x) m.
0 1 2 m
 Useful facts about power series

Example
The function f (x ) = 1/(1 − ax ) is the generating function of the
sequence 1, a, a2, a3, . . . , because
Prof. Steven Evans Discrete Mathematics

1
= 1 + ax + a2x 2 + · · ·
1 − ax
when |ax|< 1, or equivalently, for |x|< 1/|a|for a ƒ= 0.
Useful facts about power series

Theorem

Discrete Mathematics
Useful facts about power series

Example
Let f (x) = 1/(1 − x)2. Find the coefficients for its Taylor
expansion.
Discrete Mathematics
Useful facts about power series

Example
Let f (x) = 1/(1 − x)2. Find the coefficients for its Taylor
expansion.
We know that 1/(1 − x) = 1 + x + x 2 + · · · . Using the product
Discrete Mathematics

expansion theorem, we then have

∞ k ∞
2
1/(1 − x) = Σ Σ 1 xk = Σ (k + 1)xk.
k=0 j =0 k=0
Useful facts about power series

Definition

Discrete Mathematics

.
 Useful facts about power series
Example
When u = − n is a negative integer, the extended binomial
coefficient can be expressed in terms of an ordinary one:

−n (−n)(−n − 1) · · · (−n − r + 1)
. Σ =
r r!
Prof. Steven Evans Discrete Mathematics
 Useful facts about power series
Example
When u = − n is a negative integer, the extended binomial
coefficient can be expressed in terms of an ordinary one:
. Σ
−n (−n)(−n − 1) · · · (−n − r + 1)
=
r r!
(−1) r n(n + 1) · · · (n + r − 1)
Prof. Steven Evans Discrete Mathematics

=
r!
 Useful facts about power series
Example
When u = − n is a negative integer, the extended binomial
coefficient can be expressed in terms of an ordinary one:
. Σ
−n (−n)(−n − 1) · · · (−n − r + 1)
=
r r!
(−1) r n(n + 1) · · · (n + r − 1)
Prof. Steven Evans Discrete Mathematics

=
r!
(−1) r (n + r − 1)(n + r − 2) · · · n
=
r!
 Useful facts about power series
Example
When u = − n is a negative integer, the extended binomial
coefficient can be expressed in terms of an ordinary one:
. Σ
−n (−n)(−n − 1) · · · (−n − r + 1)
=
r r!
(−1) r n(n + 1) · · · (n + r − 1)
Prof. Steven Evans Discrete Mathematics

=
r!
(−1) r (n + r − 1)(n + r − 2) · · · n
=
r!
(−1) r (n + r − 1)!
=
r!(n − 1)!
 Useful facts about power series
Example
When u = − n is a negative integer, the extended binomial
coefficient can be expressed in terms of an ordinary one:
. Σ
−n (−n)(−n − 1) · · · (−n − r + 1)
=
r r!
(−1) r n(n + 1) · · · (n + r − 1)
Prof. Steven Evans Discrete Mathematics

=
r!
(−1) r (n + r − 1)(n + r − 2) · · · n
=
r!
(−1) r (n + r − 1)!
=
r!(n − 1)!
. Σ
n + r− 1
= (−1) r
r
 Useful facts about power series
Theorem
Let x be a real number with |x|< 1 and let u be a real number.
Then: ∞
Σ Σu k
(1 + x ) =
u . x .
Prof. Steven Evans Discrete Mathematics
k
k=0
 Useful facts about power series

Theorem
Let x be a real number with |x|< 1 and let u be a real number.
Then:
Σ∞ . uΣ
(1 + x )u = xk.
Prof. Steven Evans Discrete Mathematics
k
k=0

Example
Find the generating function for (1 + x ) −n and (1 − x ) −n , where n
is a positive integer.
 Useful facts about power series
Solution
Using the extended binomial theorem, we have
∞
−n k
(1 + x) −n
= Σ. Σ x .
k
k=0

Prof. Steven Evans Discrete Mathematics

 Useful facts about power series
Solution
Using the extended binomial theorem, we have
∞
−n k
(1 + x) −n
= Σ. Σ x .
k
k=0

Prof. Steven Evans There is a nice expression for these coefficients:

Discrete Mathematics

∞ .
n+ k − 1 k
(1 + x)− n = Σ (− 1)k Σk x .
k=0
 Useful facts about power series
Solution
Using the extended binomial theorem, we have
Σ∞ . − nΣ
(1 + x)− n = xk.
k
k=0

Prof. Steven Evans There is a nice expression for these coefficients:

Discrete Mathematics

Σ∞ . Σ
n+ k − 1 k
(1 + x)− n = (− 1)k x .
k
k=0

Replacing x by − x , we also find that

Σ∞ . n + k − 1 Σ
(1 − x)− n = x k.
k
k=0
 Useful facts about power series
Example
Find the number of solutions of e1 + e2 + e3 = 17, where e1, e2,
and e3 are nonnegative integers with 1 + i ≤ ei ≤ 4 + i .

Prof. Steven Evans Discrete Mathematics

 Useful facts about power series
Example
Find the number of solutions of e1 + e2 + e3 = 17, where e1, e2,
and e3 are nonnegative integers with 1 + i ≤ ei ≤ 4 + i .
The number of solutions with the indicated constraints is the
Prof. Steven Evans
coefficient of x 17 in the expansion of
Discrete Mathematics

(x 2 + x 3 + x 4 + x 5)(x 3 + x 4 + x 5 + x 6)(x 4 + x 5 + x 6 + x 7).

 Useful facts about power series
Example
Find the number of solutions of e1 + e2 + e3 = 17, where e1, e2,
and e3 are nonnegative integers with 1 + i ≤ ei ≤ 4 + i .
The number of solutions with the indicated constraints is the
Prof. Steven Evans
coefficient of x 17 in the expansion of
Discrete Mathematics

(x 2 + x 3 + x 4 + x 5)(x 3 + x 4 + x 5 + x 6)(x 4 + x 5 + x 6 + x 7).

Explicit expansion shows this coefficient to be 3. (Note that

performing the expansion is about as much work as an explicit
enumeration of the solutions.)
 Using generating functions to solve
recurrence relations
Example
Solve the recurrence relation ak = 3ak−1 and initial condition
a0 = 2.

Prof. Steven Evans Discrete Mathematics

 Using generating functions to solve
recurrence relations

Prof. Steven Evans Discrete Mathematics

 Using generating functions to solve
recurrence relations
Example
Solve the recurrence relation ak = 3ak−1 and initial condition
a0 = 2.
Let G (x ) be the associated generating function. Note that xG (x )
is the generating function for a shift of this sequence, and hence
Prof. Steven Evans
the recurrence relation becomes
Discrete Mathematics

Σ∞ Σ∞
G (x) − 3xG(x) = ak x k − 3 ak− 1 x k
k=0 k=1
Σ∞
= a0 + (ak − 3ak− 1 )x k= 2.
k=1

Thus, G(x) − 3xG(x) = 2.

 Using generating functions to solve
recurrence relations
Example
Suppose that a valid codeword is an n-digit number in decimal
notation containing an even number of 0s, and let an denote the
number of valid codewords of length n. We’ve previously shown
Prof. Steven Evans Discrete Mathematics

this satisfies the recurrence an = 8an−1 + 10n−1 with initial

condition a1 = 9. Use generating functions to find an explicit
formula for an.
 Using generating functions to solve
recurrence relations
Solution
To make life simpler, we set a0 = 1, which is consistent with our
recurrence. Multiplying the recurrence by xn, we get
anxn = 8an−1xn + 10n−1xn. Writing G (x ) for the generating
function for an and summing this equation over n, we find
Prof. Steven Evans Discrete Mathematics

Σ∞
G (x) − 1 = (8an− 1 xn + 10n−1xn)
n=1
Σ∞ Σ∞
= 8x an− 1x n−1 + x 10n−1 xn−1
n=1 n=1
= 8xG(x) + x/(1 − 10x).
 Using generating functions to solve
recurrence relations
Solution, continued
G(x) − 1 = 8xG(x) + 1/(1 − 10x).
Solving for G (x) gives
. Σ
1 − 9x 1 1 1
G(x) = = + .
(1 − 8x)(1 − 10x) 2 1 − 8x 1 − 10x
Prof. Steven Evans Discrete Mathematics
 Using generating functions to solve
recurrence relations
Solution, continued
G(x) − 1 = 8xG(x) + 1/(1 − 10x).
Solving for G (x) gives
. Σ
1 − 9x 1 1 1
G(x) = = + .
(1 − 8x)(1 − 10x) 2 1 − 8x 1 − 10x
Prof. Steven Evans Discrete Mathematics

Expanding this last expression using series, we find

. ∞ ∞
Σ ∞
1 Σ Σ Σ 1 n
G(x) = 8n x n + 10n xn = (8 + 10n)xn.
2 2
n=0 n=0 n=0

Consequently, an = 1 2(8n + 10n).

Solving Linear Homogeneous
Recurrence Relations
Linear Homogeneous Recurrence Relations
Definition: A linear homogeneous recurrence relation of degree k with constant
coefficients is a recurrence relation of the form an = c1an−1 + c2an−2 + ….. + ck an−k ,
where c1, c2, ….,ck are real numbers, and ck ≠ 0

• it is linear because the right-hand side is a sum of the previous terms of the sequence each
multiplied by a function of n.
• it is homogeneous because no terms occur that are not multiples of the ajs. Each coefficient is a
constant.
• the degree is k because an is expressed in terms of the previous k terms of the sequence.

By strong induction, a sequence satisfying such a recurrence relation is uniquely determined by

the recurrence relation and the k initial conditions a0 = C1, a0 = C1 ,… , ak−1 = Ck−1.
Examples of Linear Homogeneous Recurrence
Relations
• Pn = (1.11)Pn-1 linear homogeneous recurrence relation of degree
one
• fn = fn-1 + fn-2 linear homogeneous recurrence relation of degree two
• not linear
• Hn = 2Hn−1 + 1 not homogeneous
• Bn = nBn−1 coefficients are not constants
Solving Linear Homogeneous Recurrence
Relations
• The basic approach is to look for solutions of the form an = rn,
where r is a constant.
• Note that an = rn is a solution to the recurrence relation an = c1an−1 +
c2an−2 + ⋯ + ck an−k if and only if n
r = c1r + c2r + ⋯ + ck rn−k .
n − 1 n − 2

• Algebraic manipulation yields the characteristic equation:

rk − c1rk−1 − c2rk−2 − ⋯ − ck−1r − ck = 0
• The sequence {an} with an = rn is a solution if and only if r is a solution to the
characteristic equation.
• The solutions to the characteristic equation are called the characteristic roots of
the recurrence relation. The roots are used to give an explicit formula for all the
solutions of the recurrence relation.
Solving Linear Homogeneous Recurrence Relations of
Degree Two
Theorem 1: Let c1 and c2 be real numbers. Suppose that r2 – c1r – c2
= 0 has two distinct roots r1 and r2. Then the sequence {an} is a
solution to the recurrence relation an = c1an−1 + c2an−2 if and only
if

for n = 0,1,2,… , where α1 and α2 are constants.

Using Theorem 1
Example: What is the solution to the recurrence relation

an = an−1 + 2an−2 with a0 = 2 and a1 = 7?

Solution: The characteristic equation is r2 − r − 2 = 0.

Its roots are r = 2 and r = −1 . Therefore, {an} is a solution to the recurrence relation if and
only if an = α12n + α2(−1)n, for some constants α1 and α2.

To find the constants α1 and α2, note that

a0 = 2 = α1 + α2 and a1 = 7 = α12 + α2(−1).

Solving these equations, we find that α1 = 3 and α2 = −1.

Hence, the solution is the sequence {an} with an = 3∙2n − (−1)n.

 An Explicit Formula for the Fibonacci Numbers

We can use Theorem 1 to find an explicit formula for the Fibonacci

numbers. The sequence of Fibonacci numbers satisfies the recurrence
relation fn = fn−1 + fn−2 with the initial conditions: f0 = 0 and f1 = 1.

Solution: The roots of the characteristic equation r2 –

r – 1 = 0 are
Fibonacci Numbers (continued)

Therefore by Theorem 1

for some constants α1 and α2.

Using the initial conditions f0 = 0 and f1 = 1 , we have

,
Solving, we obtain .
Hence,
The Solution when there is a Repeated Root

Theorem 2: Let c1 and c2 be real numbers with c2 ≠ 0. Suppose that

r2 – c1r – c2 = 0 has one repeated root r0. Then the sequence {an} is a
solution to the recurrence relation an = c1an−1 + c2an−2 if and only if

for n = 0,1,2,… , where α1 and α2 are constants.

Using Theorem 2
Example: What is the solution to the recurrence relation an = 6an−1 − 9an−2 with a0 = 1 and a1 = 6?

Solution: The characteristic equation is r2 − 6r + 9 = 0.

The only root is r = 3. Therefore, {an} is a solution to the recurrence relation if and only if
an = α13n + α2n(3)n
where α1 and α2 are constants.

To find the constants α1 and α2, note that

a0 = 1 = α1 and a1 = 6 = α1 ∙ 3 + α2 ∙3.

Solving, we find that α1 = 1 and α2 = 1 .

Hence,
an = 3n + n3n .
Solving Linear Homogeneous Recurrence Relations
of Arbitrary Degree
This theorem can be used to solve linear homogeneous recurrence relations with
constant coefficients of any degree when the characteristic equation has distinct roots.

Theorem 3: Let c1, c2 ,…, ck be real numbers. Suppose that the characteristic equation
rk – c1rk−1 –⋯ – ck = 0
has k distinct roots r1, r2, …, rk. Then a sequence {an} is a solution of the recurrence
relation
an = c1an−1 + c2an−2 + ….. + ck an−k
if and only if

for n = 0, 1, 2, …, where α1, α2,…, αk are constants.

The General Case with Repeated Roots Allowed

Theorem 4: Let c1, c2 ,…, ck be real numbers. Suppose that the characteristic equation
rk – c1rk−1 –⋯ – ck = 0
has t distinct roots r1, r2, …, rt with multiplicities m1, m2, …, mt, respectively so that mi ≥ 1 for i =
0, 1, 2, …,t and m1 + m2 + … + mt = k. Then a sequence {an} is a solution of the recurrence relation
an = c1an−1 + c2an−2 + ….. + ck an−k
if and only if

for n = 0, 1, 2, …, where αi,j are constants for 1≤ i ≤ t and 0≤ j ≤ mi−1.

 Solving Linear Non-
homogeneous Recurrence
Relations
Linear Non-homogeneous Recurrence Relations
with Constant Coefficients
Definition: A linear non-homogeneous recurrence relation with constant
coefficients is a recurrence relation of the form:
an = c1an−1 + c2an−2 + ….. + ck an−k + F(n) ,
where c1, c2, ….,ck are real numbers, and F(n) is a function not identically zero
depending only on n.
The recurrence relation
an = c1an−1 + c2an−2 + ….. + ck an−k ,
is called the associated homogeneous recurrence relation.
Linear Non-homogeneous Recurrence Relations with
Constant Coefficients (cont.)
The following are linear non homogeneous recurrence relations with constant
coefficients:
an = an−1 + 2n ,
an = an−1 + an−2 + n2 + n + 1,
an = 3an−1 + n3n ,
an = an−1 + an−2 + an−3 + n!
where the following are the associated linear homogeneous recurrence relations,
respectively:
an = an−1 ,
an = an−1 + an−2,
an = 3an−1 ,
an = an−1 + an−2 + an−3
Solving Linear Non-homogeneous Recurrence Relations with Constant
Coefficients

Theorem 5: If {an(p)} is a particular solution of the nonhomogeneous

linear recurrence relation with constant coefficients
an = c1an−1 + c2an−2 + ⋯ + ck an−k + F(n) ,
then every solution is of the form {an(p) + an(h)}, where {an(h)} is a
solution of the associated homogeneous recurrence relation
an = c1an−1 + c2an−2 + ⋯ + ck an−k .
Solving Linear Nonhomogeneous Recurrence Relations with Constant
Coefficients (continued)
Example: Find all solutions of the recurrence relation an = 3an−1 + 2n.
What is the solution with a1 = 3?

Solution: The associated linear homogeneous equation is an = 3an−1.

Its solutions are an(h) = α3n, where α is a constant.

Because F(n)= 2n is a polynomial in n of degree one, to find a particular solution we might try a linear function in n, say pn = cn + d, where c and d are constants. Suppose that pn = cn + d is such a solution.
Then an = 3an−1 + 2n becomes cn + d = 3(c(n− 1) + d)+ 2n.

Simplifying yields (2 + 2c)n + (2d − 3c) = 0. It follows that cn + d is a solution if and only if
2 + 2c = 0 and 2d − 3c = 0. Therefore, cn + d is a solution if and only if c = − 1 and d = − 3/2.
Consequently, an(p) = −n − 3/2 is a particular solution.

By Theorem 5, all solutions are of the form an = an(p) + an(h) = −n − 3/2 + α3n, where α is a constant.

To find the solution with a1 = 3, let n = 1 in the above formula for the general solution.
Then 3 = −1 − 3/2 + 3 α, and α = 11/6. Hence, the solution is an = −n − 3/2 + (11/6)3n.
Relations and Their
Properties
• Relations and Functions
• Properties of Relations
• Reflexive Relations
• Symmetric and Antisymmetric Relations
• Transitive Relations
• Combining Relations
Binary Relations
Definition: A binary relation R from a set A to a set B is a subset R ⊆ A
× B.
Example:
• Let A = {0,1,2} and B = {a,b}
• {(0, a), (0, b), (1,a) , (2, b)} is a relation from A to B.
• We can represent relations from a set A to a set B graphically or using a table:

Relations are more general than

functions. A function is a relation
where exactly one element of B is
related to each element of A.
Binary Relation on a Set

Definition: A binary relation R on a set A is a subset of

A × A or a relation from A to A.
Example:
• Suppose that A = {a,b,c}. Then R = {(a,a),(a,b), (a,c)} is a
relation on A.
• Let A = {1, 2, 3, 4}. The ordered pairs in the relation
R = {(a,b) | a divides b} are
(1,1), (1, 2), (1,3), (1, 4), (2, 2), (2, 4), (3, 3), and (4, 4).
Binary Relation on a Set (cont.)
Question: How many relations are there on a set A?

Solution: Because a relation on A is the same thing as a subset of A ⨉ A, we

count the subsets of A × A. Since A × A has n2 elements when A has n
elements, and a set with m elements has 2m subsets, there are subsets of A
× A. Therefore, there are relations on a set A. 2
2| A|
| A|2
2
Binary Relations on a Set (cont.)
Example: Consider these relations on the set of integers:
R1 = {(a,b) | a ≤ b}, R4 = {(a,b) | a = b},
R2 = {(a,b) | a > b}, R5 = {(a,b) | a = b + 1},
R3 = {(a,b) | a = b or a = −b}, R6 = {(a,b) | a + b ≤ 3}.

Note that these relations are on an infinite set and each of these relations is an infinite
set.
Which of these relations contain each of the pairs

(1,1), (1, 2), (2, 1), (1, −1), and (2, 2)?

Solution: Checking the conditions that define each relation, we see that the pair
(1,1) is in R1, R3, R4 , and R6: (1,2) is in R1 and R6: (2,1) is in R2, R5, and R6: (1,
−1) is in R2, R3, and R6 : (2,2) is in R1, R3, and R4.
Reflexive Relations
Definition: R is reflexive iff (a,a) ∊ R for every element a ∊ A.
Written symbolically, R is reflexive if and only if
∀x[x∊U ⟶ (x,x) ∊ R]
Example: The following relations on the integers are reflexive:
If A = ∅ then the empty relation is
R1 = {(a,b) | a ≤ b},(1, 1) reflexive vacuously. That is the empty
R3 = {(a,b) | a = b or a = −b}, relation on an empty set is reflexive!

R4 = {(a,b) | a = b}.
The following relations are not reflexive:
R2 = {(a,b) | a > b} (note that 3 ≯ 3),
R5 = {(a,b) | a = b + 1} (note that 3 ≠3 + 1),
R6 = {(a,b) | a + b ≤ 3} (note that 4 + 4 ≰ 3).
Symmetric Relations
Definition: R is symmetric iff (b,a) ∊ R whenever (a,b) ∊ R for all a,b ∊
A. Written symbolically, R is symmetric if and only if
∀x∀y [(x,y) ∊R ⟶ (y,x) ∊ R]
Example: The following relations on the integers are symmetric:
R3 = {(a,b) | a = b or a = −b},
R4 = {(a,b) | a = b},
R6 = {(a,b) | a + b ≤ 3}.
The following are not symmetric:
R1 = {(a,b) | a ≤ b} (note that 3 ≤ 4, but 4 ≰ 3),
R2 = {(a,b) | a > b} (note that 4 > 3, but 3 ≯ 4),
R5 = {(a,b) | a = b + 1} (note that 4 = 3 + 1, but 3 ≠4 + 1).
Antisymmetric Relations
Definition:A relation R on a set A such that for all a,b ∊ A if (a,b) ∊ R and
(b,a) ∊ R, then a = b is called antisymmetric. Written symbolically, R is
antisymmetric if and only if
∀x∀y [(x,y) ∊R ∧ (y,x) ∊ R ⟶ x = y]
• Example: The following relations on the integers are antisymmetric:
R1 = {(a,b) | a ≤ b},
R2 = {(a,b) | a > b}, For any integer, if a a ≤ b and a
R4 = {(a,b) | a = b}, ≤ b , then a = b.
R5 = {(a,b) | a = b + 1}.
The following relations are not antisymmetric:
R3 = {(a,b) | a = b or a = −b}
(note that both (1,−1) and (−1,1) belong to R3),
R6 = {(a,b) | a + b ≤ 3} (note that both (1,2) and (2,1) belong to R6).
Transitive Relations
Definition: A relation R on a set A is called transitive if whenever (a,b) ∊ R
and (b,c) ∊ R, then (a,c) ∊ R, for all a,b,c ∊ A. Written symbolically, R is
transitive if and only if
∀x∀y ∀z[(x,y) ∊R ∧ (y,z) ∊ R ⟶ (x,z) ∊ R ]
• Example: The following relations on the integers are transitive:
R1 = {(a,b) | a ≤ b}, For every integer, a ≤ b
and b ≤ c, then a ≤ c.
R2 = {(a,b) | a > b},
R3 = {(a,b) | a = b or a = −b},
R4 = {(a,b) | a = b}.
The following are not transitive:
R5 = {(a,b) | a = b + 1} (note that both (3,2) and (4,3) belong to R5, but not (3,3)),
R6 = {(a,b) | a + b ≤ 3} (note that both (2,1) and (1,2) belong to R6, but not (2,2)).
Combining Relations
• Given two relations R1 and R2, we can combine them using basic set
operations to form new relations such as R1 ∪ R2, R1 ∩ R2, R1 − R2,
and R2 − R1.
• Example: Let A = {1,2,3} and B = {1,2,3,4}. The relations R1 =
{(1,1),(2,2),(3,3)} and R2 = {(1,1),(1,2),(1,3),(1,4)} can
be combined using basic set operations to form new relations:

R1 ∪ R2 ={(1,1),(1,2),(1,3),(1,4),(2,2),(3,3)}
R1 ∩ R2 ={(1,1)} R1 − R2 ={(2,2),(3,3)}
R2 − R1 ={(1,2),(1,3),(1,4)}
Composition
Definition: Suppose
• R1 is a relation from a set A to a set B.
• R2 is a relation from B to a set C.
Then the composition (or composite) of R2 with R1, is a relation from
A to C where
• if (x,y) is a member of R1 and (y,z) is a member of R2, then (x,z) is a member
of R2∘ R1.
Composition
• What is the composite of the relations R and S, where R is the
relation from {1, 2, 3} to {1, 2, 3, 4} with R = {(1, 1), (1, 4), (2,
3), (3, 1), (3, 4)} and S is the relation from {1, 2, 3, 4} to {0, 1, 2} with
• R = {(1, 1), (1, 4), (2, 3), (3, 1), (3, 4)}
• S = {(1, 0), (2, 0), (3, 1), (3, 2), (4, 1)}?

• (1,0)((1,1)(2,1)(2,2)(3,0)(3,1)
Composition
• Solution:- S ◦R is constructed using all ordered pairs in R and
ordered pairs in S, where the second element of the ordered pair in R
agrees with the first element of the ordered pair in S.
• For example, the ordered pairs (2, 3) in R and (3, 1) in S
produce the ordered pair (2, 1) in S ◦R.
• Computing all the ordered pairs in the composite, we find S ◦ R = {(1,
0), (1, 1), (2, 1), (2, 2), (3, 0), (3, 1)}.
Powers of a Relation
Definition: Let R be a binary relation on A. Then the powers Rn of the
relation R can be defined inductively by:
• Basis Step: R1 = R
• Inductive Step: Rn+1 = Rn ∘ R

The powers of a transitive relation are subsets of the

relation. This is established by the following theorem:

Theorem 1: The relation R on a set A is transitive iff Rn ⊆ R for n =

1,2,3 ….
Representing Relations
• Representing Relations using Matrices
• Representing Relations using Digraphs
 Representing Relations Using Matrices

• A relation between finite sets can be represented using a zero-one matrix.

• Suppose R is a relation from A = {a1, a2, …, am} to B = {b1,
b2, …, bn}.
• The elements of the two sets can be listed in any particular arbitrary order. When A =
B, we use the same ordering.
• The relation R is represented by the matrix MR =
[mij], where

• The matrix representing R has a 1 as its (i,j) entry when ai is related to bj

and a 0 if ai is not related to bj.
Examples of Representing Relations Using Matrices

Example 1: Suppose that A = {1,2,3} and B = {1,2}. Let R be the

relation from A to B containing (a,b) if a ∈ A, b ∈ B, and a > b.
What is the matrix representing R (assuming the ordering of elements
is the same as the increasing numerical order)?
Solution: Because R = {(2,1), (3,1),(3,2)}, the matrix is
Examples of Representing Relations Using Matrices (cont.)

Example 2: Let A = {a1,a2, a3} and B = {b1,b2, b3,b4, b5}. Which ordered
pairs are in the relation R represented by the matrix

Solution: Because R consists of those ordered pairs (ai,bj) with mij =

1, it follows that:

R = {(a1, b2), (a2, b1),(a2, b3), (a2, b4),(a3, b1), {(a3, b3), (a3, b5)}.
 Matrices of Relations on Sets

• If R is a reflexive relation, all the elements on the main diagonal of MR

are equal to 1.

• R is a symmetric relation, if and only if mij = 1 whenever mji = 1. R is

an antisymmetric relation, if and only if mij = 0 or mji = 0 when i≠ j.
 Example of a Relation on a Set

Example 3: Suppose that the relation R on a set is represented by the

matrix

Is R reflexive, symmetric, and/or antisymmetric?

Solution: Because all the diagonal elements are equal to 1, R is
reflexive. Because MR is symmetric, R is symmetric and not
antisymmetric because both m1,2 and m2,1 are 1.
 Representing Relations Using Digraphs

Definition: A directed graph, or digraph, consists of a set V of vertices (or nodes)

together with a set E of ordered pairs of elements of V called edges (or arcs). The
vertex a is called the initial vertex of the edge (a,b), and the vertex b is called the
terminal vertex of this edge.
• An edge of the form (a,a) is called a loop.

Example 7: A drawing of the directed graph with vertices a, b, c, and d, and

edges (a, b), (a, d), (b, b), (b, d), (c, a), (c, b), and (d, b) is shown here.
 Examples of Digraphs Representing Relations

Example 8: What are the ordered pairs in the relation

represented by this directed graph?

Solution: The ordered pairs in the relation are

(1, 3), (1, 4), (2, 1), (2, 2), (2, 3), (3, 1), (3, 3), (4, 1), and (4, 3)
Determining which Properties a Relation has from its
Digraph

• Reflexivity: A loop must be present at all vertices in the

graph.
• Symmetry: If (x,y) is an edge, then so is (y,x).
• Antisymmetry: If (x,y) with x ≠ y is an edge, then (y,x) is
not an edge.
• Transitivity: If (x,y) and (y,z) are edges, then so is (x,z).
Determining which Properties a Relation has
from its Digraph

a
b

c d
• Reflexive? No, there are no loops
• Symmetric? No, there is an edge from a to b, but not from b to a
• Antisymmetric? No, there is an edge from d to b and b to d
• Transitive? No, there are edges from a to c and from c to b,
but there is no edge from a to d
Determining which Properties a Relation
has from its Digraph – Example 4

b
a

c d
• Reflexive? No, there are no loops
• Symmetric? No, for example, there is no edge from d to a
• Antisymmetric? Yes, whenever there is an edge from one vertex
to another, there is not one going back
• Transitive? Yes (trivially), there are no two edges where the first
edge ends at the vertex where the second edge begins
Example of the Powers of a Relation

a b a b

d c d c
R R2

a b b
a

d c d c
R4 R3
The pair (x,y) is in Rn if there is a path of length n from x to y in R
(following the direction of the arrows).
Hasse Diagram
• Let A = {1, 2, 3, 9, 18} and consider the “divides” relation
on A: For all a, b  A,
Example:

• Draw the Hasse diagram representing the partial ordering {(a, b) ∣ a

divides b} on {1, 2, 3, 4, 6, 8, 12}.
Solution:
Partially and Totally Ordered Sets
•For instance, the set of real numbers is a partially ordered set with respect
to the “less than or equal to” relation , and a set of sets is partially ordered
with respect to the “subset” relation .

•It is entirely straightforward to show that any subset of a partially ordered

set is partially ordered.

•This, of course, assumes the “same definition” for the relation on the subset
as for the set as a whole. A set A is called a totally ordered set with respect
to a relation if, and only if, A is partially ordered with respect to and
is a total order.
Partially and Totally Ordered Sets
•A maximal element in a partially ordered set is an element that is
greater than or equal to every element to which it is comparable.
(There may be many elements to which it is not comparable.)

•A greatest element in a partially ordered set is an element that is

greater than or equal to every element in the set (so it is comparable to
every element in the set). Minimal and least elements are defined
similarly.
Partially and Totally Ordered Sets
 Example 10 – Maximal, Minimal, Greatest, and Least Elements

•Let A = {a, b, c, d, e, f, g, h, i } have the partial ordering defined by the

following Hasse diagram. Find all maximal, minimal, greatest, and least
elements of A.

•Solution:
•There is just one maximal element, g, which is also the greatest element. The
minimal elements are c, d, and i, and there is no least element.
The least element of the poset with Hasse diagram (a) is a. This poset has no
greatest element. The poset
with Hasse diagram (b) has neither a least nor a greatest element.
The poset with Hasse
diagram (c) has no least element. Its greatest element is d.
The poset with Hasse diagram
(d) has least element a and greatest element d.
• POSET (S, < ).
• If ‘u’ is an element of ‘S’ such that a < u for all elements a ∈ A, then
‘u’ is called an upper bound of A.
• If ‘l’ is an element of ‘S’ such that l < a for all elements a ∈ A, then
‘l’ is called a lower bound of A.
• The element x is called the least upper bound of the subset A if x is an
upper bound that is less than every other upper bound of A.
• Similarly greatest lower bound
• Find the lower and upper bounds of the subsets
{a, b, c}, {j, h}, and {a, c, d, f } in the poset with the
Hasse diagram shown in Figure 7.
• The upper bounds of {a, b, c} are e, f, j, and h, and its only lower
bound is a.

• There are no upper bounds of {j, h}, and its lower bounds are a, b, c,
d, e, and f .

• The upper bounds of {a, c, d, f } are f , h, and j , and its lower bound is
a.
Lattice
• A partially ordered set in which every pair of elements has both a
least upper bound and a greatest lower bound is called a lattice.
Topological Sorting

A total ordering is said to be compatible

with the partial ordering R if
a b whenever a R b. Constructing a total
ordering from a partial ordering is called
topological sorting.

105
 9 6 3 4 8 2 0 5 1 7

0 1 2 3 4

5 6 7 8 9

If there is an edge from v to w, then v

precedes w in the sequential listing.
106
Example

Consider the set A = {2, 3, 4, 6, 18, 24} ordered by

the “divides” relation. The Hasse diagram follows:
24 18

4 6

2 3

The ordinary “less than or equal to” relation on this

set is a topological sorting for it since for positive
integers a and b, if a|b then a b. 
107
Topological Sorting

108
Assemble an Automobile
1) Build Frame
2) Install engine, power train components, gas tank.
3) Install brakes, wheels, tires.
4) Install dashboard, floor, seats.
5) Install electrical lines.
6) Install gas lines.
7) Attach body panels to frame
8) Paint body.

109
Prerequisites
Task Immediately Preceding Time Needed to
Tasks Perform Task

1 7 hours
2 1 6 hours
3 1 3 hours
4 2 6 hours
5 2, 3 3 hours
6 4 1 hour
7 2, 3 1 hour
8 4, 5 2 hours
9 6, 7, 8 5 hours

110
Example – Job Scheduling
What is the total order compatible with it?

Task 4
6 hours Task 6
Task 2
1 hour
6 hours
Task 1
Task 5
7 hours
3 hours

Task 3 Task 9
Task 8
3 hours 5 hour
2 hour
Task 7
1 hour

111
112
113