MTH 307: Discrete Structures II

(Theory of Computation)

Chapter 1: Introduction to The

Theory of Computation

Dr. Maroun Abi Assaf

1
Chapter 1 Introduction to The Theory of Computation
1.1. Introduction
Computer Science stems from two starting points:
Mathematics: What can be computed?
And what cannot be computed?
Electrical Engineering: How can we build computers?
Not in this course.
Computability Theory deals with the profound
mathematical basis for Computer Science.
The question we will try to answer in this course is:
“What can be computed? What Cannot be computed and
where is the line between the two?”

A Computational Model is a mathematical object (Defined

on paper) that enables us to reason about computation and
to study the properties and limitations of computing.
We will deal with Three principal computational models in
increasing order of Computational Power.

3
Computational Models
We will deal with three principal models of computations:
1. Finite Automaton (in short FA).
recognizes Regular Languages .
2. Push Down Automaton (in short PDA).
recognizes Context Free Languages .
3. Turing Machines (in short TM).
recognizes Decidable (Recursive) Languages .

4
temporary memory

input
CPU
output

Program memory
 3
temporary memory f ( x)  x
z  2*2  4
f ( x)  z * 2  8
input
x2
CPU
f ( x)  8
Program memory output
compute xx
2
compute x x
 Automaton
temporary memory

Automaton
input
CPU
output

Program memory
 Automaton
temporary memory

Automaton
input

output
transition

state
 Different Kinds of Automata
Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

 Finite Automaton

temporary memory

input
Finite
Automaton
output

Example: Elevators, Vending Machines

(small computing power)
 Pushdown Automaton
Temp.
memory Stack Push, Pop

Pushdown input

Automaton
output

Example: Compilers for Programming Languages

(medium computing power)
 Turing Machine

Temp.
memory Random Access Memory

input
Turing
Machine
output

Examples: Any Algorithm

(highest computing power)
 Power of Automata
Simple More complex Hardest
problems problems problems

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
Turing Machine is the most powerful
computational model known

Question: Are there computational

problems that a Turing Machine
cannot solve?

Answer: Yes (unsolvable/undecidable/

un-computable
problems)
1.2. Finite Automata - A Short Example
• The control of a washing machine is a very simple example
of a finite automaton.
• The most simple washing machine accepts quarters and
operation does not start until at least 3 quarters were inserted.

q0 25 q25 25 q50 25 q75

25

• Accepts quarters.
• Operation starts after at least 3 quarters were inserted.
• Accepted words: 25,25,25; 25,25,25,25; …
• The second washing machine accepts 50 cents coins as
well.

q0 25 q25 25 q50 25,50 q75

25,50
50 50

• The most complex washing machine accepts $1 coins

too. 100

q0 25 q25 25 q50 25,50,100 q75

50 50,100
25,50,100
1.3. Preliminaries
1.3.1 Sets
Definition 1. A set is a group of objects ． The objects in a set
are called the elements, or members, of the set.

Example 1

The set of positive integers less than 100 can be denoted as1,2,3,...,99.

Example 2
A set can also consists of seemingly unrelated elements:a,2, Fred , New Jersey.

Definition 2. Two sets are equal if and only if they have the
same elements.
Example 3
Set {1,3,3,3,5,5,5,5} is the same as set {1,3,5}.
• A set can be described by using a set builder notation.
Example 3
O  {x | x is an odd positive integer less than 10}

Example 4
N  {x | natural numbers}  {0,1,2,3,...}
Z  {x | integers}  {...,- 2 ,-1,0 ,1,...}
Z  {x | positive integers}  {1,2,3,...}
R  {x | real numbers}
• A set can be described by using a Venn diagram.
Example 5
Draw a Venn diagram that presents V, the set of vowels in English alphabet.

U
a,e,i,o,u
V
• The set that has no elements is called empty set, denoted by .

Definition 3. The set A is said to be a subset of B if and

only if every element of A is also an element of B. We use
the notation A  B to indicate that A is a subset of the set B.

 A  B if and only if x(x  A  x  B)

 For any set P,   P and P  P.
 If A  B and B  A, then A  B.
Example 6
{ , {a}, {b}, {a, b}}  {x| x is a subset of the set {a, b}}.

 If a set A is a subset of set B but that A  B, A is called to be a proper subset of B,

denoted as A  B.

A B
 Definition 4. The power set of a set S is the set of all subsets of S,
denoted by P(S) or 2 S.

Example 7
What is the power set of the set S  {0,1,2}?
Solution : P ({0,1,2}) (or 2S )  { , {0}, {1}, | {2}, {0,1}, {0,2}, {1,2}, {0,1,2}}.

• Set Operations:
Complement A  {x | x  U and x  A}, where set U is called " universal"
that contains all the elements we might ever consider}
Union A  B  {x | x  A or x  B}

Intersection A  B  {x | x  A and x  B}
Difference A  B  {x | x  A and x  B}
 Example 8
{1,3,5} {1,2,3}  {1,2,3,5}.
{1,3,5} {1,2,3}  {1,3}.
{1,3,5} - {1,2,3}  {5}.

U U U
A B B A B

Example 9
Let A  {a,e,i,o,u } and the universal set is the set of the letters of the English
alphabet.Then A  {b, c, d , f , g , h, j , k , l , m, n, p, q, r , s, t , v, w, x, y, z}.

U
A
A
 Cartisian product of A and B, denoted by A  B, is the set of all ordered paris
pairs (a,b),
where a  A and b  B. Hence, A  B  {(a,b)| a  A  b  B}.

Example 10
Let A  { 1,2 } and B  {a,b,c}.
A  B  {( 1,a),( 1,b),( 1,c),( 2 ,a),( 2 ,b),( 2 ,c)}
B  A  {(a,1 ),(a,2 ),(b,1 ),(b,2 ),(c,1 ),(c,2 )}
A B  B  A
 1.3.2 Functions and Relations
Definition 1. Let A and B be sets. A function f from A to B is an assignment of
exactly one element of B to each element of A. We write f(a)=b if b is the
unique element of B assigned by the function f to the element a of A. If f is a
function from A to B, we write f : A  B.

x x
y
z
A function Not a function
Example 1
Let set A={Adams, Chou, Goodfriend, Rodriguez, Stevens} and B={A,B,C,D,F}.
Let G be the function that assigns a grade to a student in our theory of
computation. G
Adames A
Chou B
The domain of G is the set A={Adams,
Goodfriend C Chou, Goodfriend, Rodriguez, Stevens},
Rodriguez D and the range of G is the set {A,B,C,F}.

Stevens F
 Definition 3
Let A and B be the sets. A relation R from A to B is a subset A B
of .
From the definition, relation R is a set of pairs.
If (a, b)  R, we say a has a relation R with b, denoted as aRb.

• Functions can be consider as relations, but relations are

more general than functions.
Example 3
Let A={1,2,3,4}. R={(1,2), (2,3),(3,1),(4,4)} is a relation from A to A,
and it is also a function from A to A.
Example 4
R={(1,2), (2,3),(1,3),(1,4)} is a relation but not a function.
Example 5
Let A be the set of students in the TSU. Let B the set of
courses. The set R that consists of those pairs (a,b), where
a is a student enrolled in course b, is a relation from A to B.
1.3.3 Graphs and Trees
Definition 1
A graph is a construct consisting of two sets, denoted as G  (V,E) ,
whereV  {v1,v2 ,..., vn } is a set of vertices and E  {e1,e2 ,...,em } is a set of edges.
Each edge is a pair from V .

Example 1 Example 2
v3 v3
e1 v2 e3 e1 v2 e3
e7 e7
v1 e4 e10 v7 e8 v1 e4 e10 v7 e8
v5 e5 e6 v8 v5 e5 e6 v8
v9 e2 v4 e9 v9 e2 v4 e9

A directed graph An undirected graph

(digraph)
Definition 3
A tree is a directed graph that has no cycles. There is a one distinct
vertex in tree, called the root.

root
Level 0 v1

Level 1 v2 v4 v3 Parent of v6
Height=3
Level 2 v8 v5 v6 Child of v3

Level 3 v7 v9
1.3.4 Languages
Definition 1
Let  be a nonempty set of symbols, called alphabet,
a string be finite sequence of symbols from , and
* contain all the string (including empty string ε ) from .
A language L is defined as a subset of * .
Example 1
Let Σ  {a,b}. Then Σ *  {ε, , a, b, aa, ab, ba, bb, aaa, aab,...}.
L1  {a, aa, aab} is a finite language on . L2  {a nb n : n  0} is an infinite language on .
Useful concepts for strings
Let w  a1a2  an and v  b1b2 bm be two strings on .
(1) The concatenation of w and v, denoted by wv, is wv  a1a2  an b1b2 bm .
(2) The reverse of the string w, denoted as w R , is w R  an an 1  a1.
(3) The length of the string w, denoted by |w|, is the number of symobls in w.
Therefore,|λε | 0.
(4) Any string of consecutive characters in w is said to be a substring of w.
(5) If w  vu, then the substring v is said to be a prefix of w,
and the substring u is said to be a suffix of w.
 Operations on languages
Let L be a language on Σ .
(1) The complement of L is defined as L  Σ *  L.
(2) The reverse of L is defined as LR  {w R:w  L}.
(3) The concatenation of two languages L1 and L2 on 
is defined as L1 L2  {xy: x  L1 and y  L2 }.
(4) The star - closure of L is defined as L*  L0  L1  L2 , and
the posive closure of L is defined as L  L1  L2 

Example 2
Let L  {a nb n: n  0 }.
Then L2  {a nb n a mb m: n  0 ,m  0 }, and LR  {b n a n:n  0 }.
1.4. Relationship between Languages and Problems
• A programming language can also be seen as a set of strings over an alphabet.
For simplicity, assume this alphabet is ∑ = {0,1}.
– As such, the set of all syntactically correct programs can be seen as a subset of ∑ *
• A decision problem is one whose required answer/output for any given/input
string is either Yes or No.
– The set of all Yes-instances of a decision problem X is a language LX, being a set of
strings.
– Therefore the set of languages is equivalent to (or in 1-1 correspondence with) the set
of decision problems.
– Every decision problem gives rise to a language
– Any language L can be seen as the set of Yes-instance of a problem
– Therefore the set of decision problems is equivalent to the power set of ∑ * (i.e.,
the set of all subsets of ∑*)
• The set of all decision problems is uncountable while the set of all (computer
programs) solutions is countable.
– This implies that, unfortunately, there are unsolvable decision problems.