CSE2013 - Theory of Computation

Gunavathi C
gunavathi.cm@vit.ac.in
Theory of computation
• Theory of computation (TOC) is a branch of Computer Science that is
concerned with how problems can be solved using algorithms and how
efficiently they can be solved.

• To understand the way an algorithm is meant to work

• To actually prove it works through analysing problems that may arise

with the technique used and finding solutions to these problems.

2
Computational model
• What is Computation?
Program running in computer

Input x Algorithm f(x) Output

YES
Input w M
NO
3
•What is computation?
 Execution of algorithms.
 Step by step procedure that produce a result from given input.
 Computation is simply a sequence of steps that can be performed
by computer.

•What is Theory?
• A theory explains known facts.
• It also allows the scientists to make predictions of what they should
observe if a theory is true.

4
Purpose of Theory of Computation

Develop formal mathematical models of computation that reflect

real-world computers.

Models of computation can be classified in three categories:

• Sequential models
• Functional models
• Concurrent models.

5
Models of Computation
• Sequential models:
FSM, PDA, Random Access Machine, TM
• Functional models:
Lambda Calculus, General Recursive functions, Combinatory logic
• Concurrent models:
Petri nets, DFG (Data Flow Graph), Interaction nets, Actor model

6
Questions
• Are there problems that cannot be solved on any computing device?
• How does one determine if a given problem can or cannot be
computationally solved?
• If we place bounds on the resources (time and space) available to a
computer, then what can be said about which problems can and
which problems cannot be solved on a computer?
• How does the power of a computer change, if it has access to random
bits?

7
Answers
• Computation models: Finite State Automata, Turing machines (TM).
• Regular Expressions and Regular Grammars.
• Context-free Grammars and Pushdown Automata.
• Computability Theory and reducibility
• Computational complexity
• Determinism and Non-determinism
• Time and Space Complexity
• NP completeness

8
 Motivation
1. What is computable,and what is not ?
2. How to analyze an algorithm ?
3. Whether a program/code is executable or not?
4. Whether a program/code is erroneous or not?
5. Is it possible to make a program more efficient?
6. What is easy,and what is difficult, to compute?
7. What is easy,and what is hard for computers to do?

9
 Fields Related to TOC

Web search: theory of pattern matching.

Sequential circuits: theory of finite state automata.
Compilers: theory of context free grammars.
Cryptography: theory of computational complexity.
Neural networks : Automata theory
Computational biology: DNA and proteins are strings.
Artificial intelligence: the undecidability of first-order logic.(FL and Automata)
Natural languages: mostly context-free. Speech understanding systems use
probabilistic FSMs
Algorithm Analysis: Computational complexity

10
Module -1
Introduction to Languages and Grammars

 Recall on Proof techniques in Mathematics

 Overview of a Computational Models
 Languages and Grammars
 Alphabets
 Strings
 Operations on Languages
 Overview on Automata

11
Introduction to TOC
⦁ TO C suggests various abstract models of computation, which are represented
mathematically.
⦁ The machines which performs computation are not actual computers.
⦁ They are abstract machines.

⦁ Some examples of abstract machine are :

1. Turing machines - Universal model.
2. FiniteAutomata- Restricted Machine
3. Push DownAutomata- Restricted but powerful than FA

⦁ Theory of computation lays foundation for various fields of computer science,

such as:
Compiler Design, Robotics, Cryptography, Artificial Intelligence
12
Chomsky Hierarchy
Language Grammar Machine Example

Regular grammars
Finite-state
Type 3 Regular languages • Right-linear grammars a*
automata
• Left-linear grammars

Push-down
Type 2 Context-free languages Context-free grammars anbn
automata
Linear-bound n n n
Type 1 Context-sensitive languages Context-sensitive grammars abc
automata

Recursive languages any

Turing
Type 0 Recursively enumerable Unrestricted grammars computable
machines
languages function

13
Strings and Languages (1)
• Alphabet: Finite, nonempty set of symbols
• Examples:
•  = {0, 1}: binary alphabet
•  = {a, b, c, …, z}: the set of all lower case letters
• The set of all ASCII characters

• String: Finite sequence of symbols from an alphabet 

• Examples:
• 01101 where  = {0, 1}
• abracadabra where  = {a, b, c, …, z}

14
Strings and Languages (2)
• Empty String: The string with zero occurrences of symbols from  and is
denoted e or 

• Length of String: Number of symbols in the string

• The length of a string w is usually written |w|
• |1010| = 4
• |e| = 0
• |uv| = |u| + |v|

• Reverse : wR
• If w = abc, wR = cba

15
Strings and Languages (3)
• Concatenation: if x and y are strings, then xy is the string obtained by
placing a copy of y immediately after a copy of x
• x = a1a2 …ai, y = b1b2 …bj
• xy = a1a2 …aib1b2 …bj
• Example: x = 01101, y = 110, xy = 01101110
• xe = ex = x

16
Strings and Languages (4)
• Power of an Alphabet: k = the set of strings of length k with symbols from 
• Example:  = {0, 1}
• 1 =  = {0, 1}
• 2 = {00, 01, 10, 11}
• 0 = {e}

• Question: How many strings are there in 3?

• The set of all strings over  is denoted *

• * = 0  1  2  3  …
• Also
• + = 1  2  3  …
• * = +  {e}
• + = * - {e}

17
Strings and Languages (5)
• Substring: any string of consecutive characters in some string w
• If w = abc
• e, a, ab, abc are substrings of w

• Prefix and suffix:

• if w = vu
• v is a prefix of w
• u is a suffix of w
• Example
• If w = abc
• a, ab , abc are prefixes of w
• c, bc, abc are suffixes of w

18
Strings and Languages (6)
• Suppose: S is the string banana
• Prefix : ban, banana
• Suffix : ana, banana
• Substring : nan, ban, ana, banana

19
Strings and Languages (7)
• Language: set of strings chosen from some alphabet
• A language is a subset of *
• Example of languages:
• The set of valid Arabic words
• The set of strings consisting of n 0’s followed by n 1’s
• {e, 01, 0011, 000111, …}
• The set of strings with equal number of 0’s and 1’s
• {e, 01, 10, 0011, 0101, 1010, 1001, 1100, …}

• Empty language:  = { }
• The language {e} consisting of the empty string
• Note:   {e}
20
Strings and Languages (8)
• Can concatenate languages
• L1L2 = {xy | x  L1, y  L2}
• Ln = L concatenated with itself n times
• L0 = {e}; L1 = L

• Star-closure
• L* = L0  L1  L2  ...
• L+ = L * - L0

• Languages can be finite or infinite

• L = {a, aba, bba}
• L = {an | n > 0}

21
Strings and Languages (9)

OPERATION DEFINITION
union of L and M L  M = {s | s is in L or s is in M}
written L  M
concatenation of L LM = {st | s is in L and t is in M}
and M written LM
Kleene closure of L L* =  Li , i=0,.., 
written L*
L* denotes “zero or more concatenations of “ L
positive closure of L
written L+ L+ =  Li, i=1,.., 
L+ denotes “one or more concatenations of “ L
L+ = LL*
22
Strings and Languages (10)
L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9}
L  D = the set of letters and digits
LD = all strings consisting of a letter followed by a digit
L2 = the set of all two-letter strings
L4 = L2 L2 = the set of all four-letter strings
L* = { All possible strings of L plus e }, L+ = L* - e
D+ = set of strings of one or more digits
L (L  D ) = set of all strings consisting of a letter
followed by a a letter or a digit
L (L  D )* = set of all strings consisting of letters and
digits beginning with a letter 23
Strings and Languages (11)
• The language L consists of strings over {a,b} in which each string
begins with an a should have an even length

• aa, ab  L
• aaaa,aaab,aaba,aabb,abaa,abab,abba,abbb  L

• baa  L
• aL

24
Strings and Languages (12)
• The language L consists of strings over {a,b} in which each occurring
of b is immediately preceded by an a

• eL
• aL
• abaab  L

• bb  L
• bab  L
• abb  L

25
Strings and Languages (13)
• Let X = {a,b,c} and Y = {abb, ba}. Then
• XY = {aabb, babb, cabb, aba, bba, cba}
• X0 = {e}
• X1 = X = {a,b,c}
• X2 = XX = {aa,ab,ac,ba,bb,bc,ca,cb,cc}
• X3 = XXX =
{aaa,aab,aac,aba,abb,abc,aca,acb,acc,baa,bab,bac,bba,bbb,bbc,bca,bcb,bcc,c
aa,cab,cac,cba,cbb,cbc,cca,ccb,ccc}

26
Strings and Languages (14)
• The language L = {a,b}*{bb}{a,b}* = *{bb}*
• consists of the strings over {a,b} that contain the substring bb
• bb  L
• abb  L
• bbb  L
• aabb  L
• bbaaa  L
• bbabba  L
• abab  L
• bab  L
• bL

27
Strings and Languages (15)
• Let L be the language that consists of all strings that begin with
aa or end with bb
• L1 = {aa}{a,b}*
• L2 = {a,b}*{bb}
• L = L1  L2 = {aa}{a,b}*  {a,b}*{bb}
• bb  L
• abb  L
• bbb  L
• aabb  L
• bbaaa  L
• bbabba  L
• abab  L
• bab  L
• ba  L
28
Strings and Languages (16)
• Let L1 = {bb} and L2 = {e, bb, bbbb} over b

• The languages L1* and L2* both contain precisely the strings
consisting of an even number of b’s.

• e , with length zero, is an element of L1* and L2*

29
Strings and Languages (17)
• What is the language of all even-length strings over {a,b}
• L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)*

• What is the language of all odd-length strings over {a,b}

• L = {a,b}* - {aa, bb, ab, ba}* or
• L = {a,b}{aa, bb, ab, ba}* or
• L = {aa, bb, ab, ba}* {a,b}

30