Theory of Computation

Lecture – 01 : Introduction

Instructor: Trushna Parida (email: trushnaparida@soa.ac.in)

Department of Computer Science and Engineering
Institute of Technical Education & Research, SOA University
Course Structure:
1. Course Number and Name:

CSE 3031, Theory of Computation

2. Credits and Course Format:

• Credits = 4

• 3 Classes/week, 1hr/Class

• 1 Problem Solving Session/Week, 2hr/Problem Solving Session

3. Grading Pattern
• Attendance : 05%

• Assignments / Quizes : 20%

• Mid-Term Examination : 15%

• PSS Examination : 15%

• End-Term Examination : 45%

Text Book and References:

Michael Sipser, Introduction to the Theory of Computation, 3rd Edition, CENGAGE learning.

Course Description:

• focuses on fundamental mathematical models of computation.

• will discuss both the inherent capabilities and limitations of these computational models as well as
their relationships with formal languages.

• Emphasize on Rigorous arguments and proofs of correctness

• Enhances/develops student’s ability to understand and conduct mathematical proofs for computation
and algorithms.
Topics to be covered:

❖ Finite automata, regular languages, and regular grammars.

❖ Deterministic and nondeterministic computations on various automata.

❖ Context free grammars, languages, and pushdown-automata

❖ Turing machines, Church's thesis, and undecidable problems

Course outcomes:

❖ Be able to understand and conduct mathematical proofs for computational algorithms.

❖ Be able to design and analyze finite automata, and equivalent regular expression for describing

regular languages

❖ Be able to design and analyze pushdown automata, and context-free grammars and languages.

❖ Be able to design and analyze Turing machines.

❖ Be able to analyze a language is decidable or undecidable and apply reductions accordingly.

Theory of Computation:
• A branch that deals with whether and how efficiently problems can be solved on a model of
computation, using an algorithm.

• This course covers 3 areas, which make up the theory of computation

❑ Automata and Language Theory

❑ Computability Theory

❑ Computational Complexity Theory

Computational Models:

• Mathematical models used to perform computations

• Finite Automata

• Push Down Automata

▪ Turing Machine
Complexity Theory
• The objective is to classify the problem according to their computational difficulties

• A easy one or a hard one

• Can determine how hard a problem is?

• With the help of time and space complexity of algorithms we can classify problems

• complexity theory helps computer scientists determine the limits of what a computational

machine can and cannot do

Computability Theory
• The objective is to classify the problem as solvable or not solvable.

• Certain problems cannot be solved by computational machines

• That is, there is no algorithmic solutions exist for those.

• With this, We can ask what a computational model can and can’t do

• Closely related to computational complexity theory

• introduces several of the concepts used in complexity theory.

Automata and Languages
• Deals with the definitions and properties of mathematical models of Computation

• Basically, it introduces models of computation

• Computational Models:

• Finite Automata : used in text processing, compilers, and hardware design

• Push Down Automata : programming languages and artificial intelligence.

• Turing Machine : a machine capable of solving all the problems that can be solved
by a computer.

• Each model determines what can be expressed and what can’t be.

• Given a model, we can examine computability and complexity of the problem

Mathematical Notation
• Sets

• Sequences and Tuples

• Functions and Relations

• Graphs

• Strings and Languages

• Boolean Logic
Sets
• A set is a group of objects, order doesn't matter

• The objects are called elements or members

• Examples: (1, 3, 5) or ( 1, 5,3)

• A set is represented using the set builder notation or roster notation.

• Example 1.1: A set of natural numbers equal to or less than 5 can be represented: in

roster notation as {1, 2, 3, 4, 5} and in set builder notation as {x|x 𝟄 X, x ≤5}

• Subset (A ⊆ B or A ⊇ B)

• Cardinality Number elements in set (|A| or n(A))

• Intersection (∩) and Union (∪), Complement (Ac)

Sets II
• Venn Diagrams can be used to visualize sets

A∪B A∩B

• Power Set: All possible subsets of a set A : P(A) or (2A)

• If A = (0, 1) then what is P(A)?

• {∅, {0}, {1}, {0, 1}}.

• In general, the cardinality of P(A) = 2|A|

Sequences and Tuples
• A sequence is an ordered list of elements

• order matters

Example (1, 3, 5) or (5, 3, 1)

• may also have repeated elements in a sequence such as (5, 5, 5) (5,5)

• Similar to sets, sequences may be finite or infinite

• Finite sequences often are often called tuples.

• A sequence with k elements is also called k-tuple

• (1, 3, 5) is a 3-tuple and a k-tuple has k elements

• A 2-tuple is also called an ordered pair.

ex:- (0, 1), (a, b) and (21, 57)

Sequences and Tuples II
• Cartesian product (x) is an operation on sets that yields a set of tuples or sequences

• Example if A = (1, 2 ) and B = (x, y, z)

• A x B = {(1,x), (1,y), (1,z), (2,x), (2,y), (2,z)}

• We can take Cartesian product of a set with itself

• AK represents A x A x A x A where there are k numbers of As.

Functions and Relations
• A function maps an input to a (single) output

Example: f(a) = b, that means f maps a to b

• Functions have three main parts: the input, the relationship and the output.

• The function always maps the input to output using a relationship and hence called a mapping

• The set of possible inputs to a function is the domain and the set of possible outputs of a function
is the range

• f : A -> B, where the set A is called the domain of the function and the set B is called the co-domain
of the function.

• The range of the function is subset of the co-domain set B.

Relations
• A relation in mathematics defines the relationship between two different sets of information

• mathematically we can say, A relation R from a set ‘A’ to a set ‘B’ is any subset of A × B and is
represented as R : A ->B

• Ex. let A={a, b, c} and B={1, 2, 3}

then any subset of A×B becomes the relation from A to B.

As there are 29 subset of A × B is possible, number of relations possible is 29. Some of those relations are R1={(a, 2),
(b, 1)} R2={(a, 3), (b, 2)} , R3={(c, 2), (b, 1)} etc.

• If a relation has k tuples as its domain then it is called a k-ary relation

• The common case of 2−aryrelation is called the binary relation.

• Relations have 3 important properties : reflexive, symmetric and transitive

• A binary relation is an equivalence relation if it satisfies all three properties

Graphs
• An undirected graph, or simply a graph G(V,E), is a set of points with lines connecting some or all of
the points.

• The points are called nodes or vertices(V ), and the lines are called edges(E)

• So, a graph is a set of vertices V and set of edges E

Graphs II
• If the nodes and/or edges of a graph are labeled, then it is called a labeled graph

• A subgraph S of a graph G is a graph whose vertex set V (S) is a subset of the vertex set V (G) that is
V (S) ⊆ V (G), and whose edge set E(S), is a subset of the edge set E(G), that is E(S) ⊆ E(G).
Graphs III
• The degree of a vertex is the number of edges touching it

• A path is a sequence of nodes connected by Some edges

• A simple path is a path that does not repeat nodes of the graph.

• A path is a cycle if it starts and ends at same node

• A simple cycle repeats only first and last node

• A graph is a tree if it is connected and has no simple cycles tree

Graphs IV
• A tree may contain a specially designated node called the root

• The nodes of degree 1 in a tree, other than the root, are called the leaves of the tree.

• Directed Graphs: A directed graph has arrows instead of lines

• The number of arrows pointing from a particular node is the outdegree of that node, and

• the number of arrows pointing into a particular node is the indegree.

Graphs V
• A path in which all the arrows point in the same direction as its steps is called a directed path.

• A directed graph is strongly connected if a directed path connects every two nodes.
Strings and Languages:
• A non empty finite set of symbols is called alphabet .
• Generally capital Greek letters ∑ and Γ are used to designate alphabets and a typewriter font for
symbols.
• Examples of alphabets are ∑1 = {0, 1}, ∑2 = {a, b, c}, etc.

• A string over an alphabet is a finite sequence of symbols from that alphabet, usually written next to

one another and not separated by commas

• If ∑1 = {0,1}, then ”01001” is a string over ∑1.

• If ∑2 = {a, b, c, ..., z}, then ”abracadabra” is a string over ∑2.

Strings and Languages:
• The number of symbols in a string is called its length
• If w is a string then its length is represented by | w |. The string of length zero is called the
empty string and is written as ∊.

• Reverse of a string, is the string obtained by writing a string w in the opposite order and is

represented as WR

• If 01001 is a string , then its reverse is 10010.

• String z is a substring of string w if z appears consecutively within w.

• For example, ”cad” is a substring of string ”dfcadjhgi”

Strings and Languages:
• If we have string x of length m and string y of length n, the concatenation of x and y, written xy,
where xy will be the string obtained by appending y to the end of x, as in X1X2X3……XmY1Y2Y3…..Yn
• If y=xz is a string, then we can say x as the prefix for string y
• Example: 10100 is a string we can say, 1, 10, 101, 1010, 10100 are the prefixes of the given
string
• x will be said to be proper prefix of string y if x ≠ y
• Example: 10100 is a string we can say, 1, 10, 101, 1010are the proper prefixes of the given string
• If y=xz is a string, then we can say z as the suffix for string y
• Example: 10100 is a string we can say, 0, 00, 100, 0100, 10100 are the suffixes of the given
string
• x will be said to be proper suffix of string y if x ≠ y
• Example: 10100 is a string we can say, 0, 00, 100, 0100 are the proper suffixes of the given string
Strings and Languages:
• A language is a set of string all of which are chosen from some ∑*, where ∑ is a particular alphabet

• This means that language L is subset of ∑*.

• If the language takes all possible strings of length 2 over ∑ = {a, b}, then L = { ab, aa, ba, bb }

• A language is prefix-free if no member is a proper prefix of another member

Boolean Logic:
• Boolean logic is a mathematical system built around the two values TRUE and FALSE.
• The values TRUE and FALSE are called the Boolean values and are often represented by the values 1
and 0.
• Boolean values can be manipulated by using Boolean operations

• Negation or NOT (¬)

• Conjunction or AND operation (∧ )

• Disjunction or OR operation ( ∨)

• “Exclusive or” or XOR operation ( ⊕)

• Equality operation (↔ )

• Implication operation (→ )
Boolean Logic II
• The Boolean operations are used for combining simple statements into more complex Boolean
expressions.
• For example,
• if P is the Boolean value representing the truth of the statement “the sun is shining” and
• Q is the Boolean value that represents the truth of the statement “today is Monday”, then
• P ∧ Q , represent the truth value of the statement “the sun is shining and today is Monday”
and
• similarly P ∨ Q represent “the sun is shining or today is Monday”

• . The values P and Q are called the operands of the operation.

Boolean Logic III
• Negation or NOT (¬)
• The negation of a Boolean value is the opposite value.
• Thus ¬0=1 and
• ¬1=0.
• Conjunction or AND operation ( ∧ )
• The conjunction of two Boolean values is 1 if both of those values are 1.
• Thus 0 ^ 0 = 0 and 1 ^ 1 = 1

P Q P∧ Q
0 0 0
0 1 0
1 0 0
1 1 1
Boolean Logic IV
• Disjunction or OR operation ( ∨ )
• The disjunction of two Boolean values is 1 if either of those values is 1.
• Ex: 0 ∨ 1 = 1

P Q P∨ Q
0 0 0
0 1 1
1 0 1
1 1 1
Boolean Logic V
• “Exclusive or” or XOR operation ( ⊕ )
• The Exclusive or” or XOR operation of two Boolean values is 1 if either of those values is 1 but not both
of its two operands are same .
• Ex: 0 ⊕ 0 = 0 and 1 ⊕ 1 = 0

P Q P⊕ Q
0 0 0
0 1 1
1 0 1
1 1 0
Boolean Logic VI
• Equality operation (↔ )
• The equality operation of two Boolean values is 1 if both of its operands have the same value
• Ex: 0 ↔ 0 = 1 and 1 ↔ 1 = 1

P Q P⊕ Q
0 0 1
0 1 0
1 0 0
1 1 1
Boolean Logic VII
• Implication operation (→ )
• The implication operation of two Boolean values is 0 if its first operand is 1 and its second
• operand is 0; otherwise, it is 1.
• Ex: 0 → 0 = 1 and 1 → 1 = 1

P Q P⊕ Q
0 0 1
0 1 1
1 0 0
1 1 1
Boolean Logic VIII
• All Boolean operations can be expressed in terms of the AND & NOT operations

• P∨Q = ¬(¬P ∧ ¬Q)

• P→Q = ¬P ∨ Q
• P↔Q = (P → Q) ∧ (Q → P)
• P⊕Q = ¬(P ↔ Q)
• In the above example, the two expressions in each row are equivalent.
• In each row the operations in the left-hand side are written in terms of AND and NOT operations in its
right-hand side.
Boolean Logic IX
• The distributive law for AND and OR is
• P ∧ (Q ∨ R) equals (P ∧ Q) ∨ (P ∧ R), and its dual
• P ∨ (Q ∧ R) equals (P ∨ Q) ^ (P ∨ R).

• Note: The dual of a Boolean expression is obtained by interchanging sums and products and
interchanging 0 and 1.
• For example, the dual of xy is (x + y)
• The dual of xy +1 is (x + y) · 0
SUMMARY OF MATHEMATICAL TERMS
• Alphabet A finite, nonempty set of objects called symbols
• Argument An input to a function
• Binary relation A relation whose domain is a set of pairs
• Boolean operation An operation on Boolean values
• Boolean value The values TRUE or FALSE, often represented by 1 or 0
• Cartesian product An operation on sets forming a set of all tuples of elements from respective sets
• Complement An operation on a set, forming the set of all elements not present
• Concatenation An operation that joins strings together
• Conjunction Boolean AND operation
• Connected graph A graph with paths connecting every two nodes
SUMMARY OF MATHEMATICAL TERMS
• Cycle A path that starts and ends in the same node
• Directed graph A collection of points and arrows connecting some pairs of points
• Disjunction Boolean OR operation
• Domain The set of possible inputs to a function
• Edge A line in a graph
• Element An object in a set
• Empty set The set with no members
• Empty string The string of length zero
• Equivalence relation A binary relation that is reflexive, symmetric, and transitive
• Function An operation that translates inputs into outputs
• Graph A collection of points and lines connecting some pairs of points
• Intersection An operation on sets forming the set of common elements
SUMMARY OF MATHEMATICAL TERMS
• k-tuple A list of k objects
• Language A set of strings
• Member An object in a set
• Node A point in a graph
• Ordered pair A list of two elements
• Path A sequence of nodes in a graph connected by edges
• Predicate A function whose range is {TRUE, FALSE}
• Property A predicate
• Range The set from which outputs of a function are drawn
• Relation A predicate, most typically when the domain is a set of k-tuples
• Sequence A list of objects
SUMMARY OF MATHEMATICAL TERMS
• Set A group of objects
• Simple path A path without repetition
• Singleton set A set with one member
• String A finite list of symbols from an alphabet
• Symbol A member of an alphabet
• Union An operation on sets combining all elements into a single set
• Unordered pair A set with two members
• Vertex A point in a graph
Exercise:
0.1 Examine the following formal descriptions of sets so that you understand which members they

contain. Write a short informal English description of each set.

a. {1, 3, 5, 7, . . . }

b. { . . . ,−4,−2, 0, 2, 4, . . . }

c. {n| n = 2m for some m in N}

d. {n| n = 2m for some m in N, and n = 3k for some k in N}

e. {w| w is a string of 0s and 1s and w equals the reverse of w}

f. {n| n is an integer and n = n +1}

Exercise:
0.2 Write formal descriptions of the following sets.

a. The set containing the numbers 1, 10, and 100

b. The set containing all integers that are greater than 5

c. The set containing all natural numbers that are less than 5

d. The set containing the string ‘aba’

e. The set containing the empty string

f. The set containing nothing at all

Exercise:
0.3 Let A be the set {x, y, z} and B be the set {x, y}.

a. Is A a subset of B?

b. Is B a subset of A?

c. What is A ∪ B?

d. What is A ∩ B?

e. What is A × B?

f. What is the power set of B?

Exercise:
0.4 If A has a elements and B has b elements, how many elements are in A × B?

Explain your answer.

0.5 If C is a set with c elements, how many elements are in the power set of C?

Explain your answer.

Exercise:
Exercise:
0.7 For each part, give a relation that satisfies the condition.

a. Reflexive and symmetric but not transitive

b. Reflexive and transitive but not symmetric

c. Symmetric and transitive but not reflexive

0.8 Consider the undirected graph G=(V,E) where V , the set of nodes, is {1, 2, 3, 4} and E, the set of edges,

is {{1, 2}, {2, 3}, {1, 3}, {2, 4}, {1, 4}}. Draw the graph G. What are the degrees of each node?

a. Indicate all the path from node 3 to node 4 on your drawing of G.