BCSE304L

Theory of Computation

Dr. S K Somasundaram
Assistant Professor Senior Grade 2
School of Computer Science and Engineering,
Vellore Institute of Technology,
Vellore – 632014
Phone No: +91 9843665115
Mail ID: somasundaram.sk@vit.ac.in
Location: PRP Block – 218D
 Module: 1
Introduction to Languages and Grammars
Contents
1. Recall on Proof techniques in Mathematics
a. Introduction to Formal Proof
b. Additional Forms of Proof
c. Proof by Contradiction
d. Inductive Proofs

2. Overview of a Computational Models

3. Languages and Grammars
4. Alphabets, Strings and Operations on Languages
5. Overview on Automata
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Theory of Computing
• Most fundamental course of Computer Science

• Will help you understand how people have taught about Computer Science as
Science in the past 50-60 years.

• To know the foundations and principles of computer science

• Branch that deals with how efficiently problems can be solved on a model of
computation, using an algorithm

• Branches:
• Automata Theory
• Computability Theory
• Computational Complexity Theory
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Example
1. Design a machine that accepts all binary strings that ends in 0
• Ex: 1101010, 100001

2. Design a machine that accepts all valid Java codes.

• Ex: Compilers

3. Design a machine that accepts all valid Java codes and does
not go into an infinite loop.

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Applications
1. Traffic Lights

2. Lifts and elevators

3. Cryptography

4. Design and Analysis of Algorithm

5. Quantum Calculation

6. Logic within Computer Science

7. Randomness within Calculation

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Cont…

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Automata Theory
• Study of abstract computing machine as well as computational
problems solved with them
• Study of mathematical properties of automata
• Ex: FA modeling of on/off switch
Push

Start off on
Push

• Need for Automata Theory

• Essential for the study of the limits of computation
• Pattern searching in websites
• Designing and checking the behaviour of the digital circuits
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Overview of a computational model
• Computation
Program running in computer

Input x Algorithm f(x) Output

Y
Input w M
N
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Central Concepts of Automata Theory
• Alphabets ()
• A finite, non-empty set of symbols
• Ex:
a)  =   the binary alphabet
b)  = {a, b, … , z}, the set of all lower-case letter
c) The set of all ASCII characters

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Central Concepts of Automata Theory – cont..

• Strings
• A string (or word) is a finite sequence of symbols
chosen from some alphabet
• Example
• 1011 is a string from alphabet  = {0, 1}
• Empty string,  → a string with zero occurrences of symbols
• Length |w| of string w → the number of positions for symbols in w
• Examples:|0111|= 4, ||=0, …
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
 Central Concepts of Automata Theory – cont..
• Power of an alphabet, k • Concatenation of two strings
• A set of all strings of length k x and y is xy
• Examples • Example
• Given  =   we have • If x = 01101, y = 110,
 = { 2 = {00, 01, 10, 11} • then xy = 01101110, and xx =
• Set of all strings over  → denoted as  x = 0110101101, …

 =  ∪  ∪  ∪ …
• Set of nonempty strings from  → + •  is the identity for
+ =  −  concatenation
+ =  ∪  ∪  ∪ … since w = w = w
 = + ∪
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Central Concepts of Automata Theory – cont..
• Languages
• A language is a set of strings all chosen from some *
• If  is an alphabet, and L*, then L is a language over .
• Examples
• The set of all legal English words is a language ➔ {a, b, c, …., z}
• The set of all strings of n 0’s followed by n 1’s for n 0 ➔ {, 01, 0011, 000111, …}
• The set of strings of equal no. of 0’s and 1’s ➔ {, 01, 10, 0011, 0101,…}
• * is an infinite language for any alphabet 
•  = the empty language (no string or not the empty string, ) is a language over
any alphabet
•  ≠ {}
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Languages – cont…
• Σ* denotes the set of all finite strings over Σ
• A formal language L over the alphabet Σ is a subset of Σ*
• That is L is a subset of Σ*
• Let Σ = {0, 1} be the alphabet.
• L = {wΣ* | the number of 1’s in w is even}.
• ε, 0, 00, 11, 000, 110, 101, 011, 0000, 1100, 1010, 1001, 0110,
0101, 0011, … are in L

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Properties of languages

1. Concatenation

2. Substring

3. Reversal

4. Kleene Closure and Positive Closure

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

 Concatenation of languages
• The concatenation of two languages L1 and L2 is given by,
L = L1 L2 = {w   w = xy for some x  L1 and y  L2}

• Concatenation of two strings w1 and w2 is defined as the sequence of

symbols in w1 followed by sequence of symbols in w2
• Let w1= abcd and w2 = fgh The concatenation of string x for n times,
where n ≥ 0, is denoted by xn
Then
x0 = ε
w1w2 = abcdfgh x1 = x
w2w1 = fghabcd x2 = xx
x3 = xxx
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
…
Substring
Let x and y be strings over an alphabet, Σ
The string x is a substring of y if there exist strings w and z
over Σ such that y = w x z.
• ε is a substring of every string.
• For every string x, x is a substring of x itself.
Example
• ε, comp and computation are substrings of computation.

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Reversal

The reversal of a string is a string with same symbols in reverse

order, denoted by wR
Example
Let w = abcdfgh
Then
wR = hgfdcba

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

 Kleen Star
• The Kleen star of a language L is denoted by L* which is the set of all strings
obtained by concatenating zero or more strings from L
L* = {w * ; w = w1…wk for k>=0 and w1, w2, …, wk  L}
Let ∑ = {a, b} i.e. ∑1 = {a, b}
∑0 ={ε}
∑2 = ∑1 . ∑1 = {a, b}. {a, b} = {aa, ab, ba, bb} The Kleene closure of ∑, denoted by
∑3= ∑1 . ∑1 . ∑1 = {a, b}. {a, b}. {a, b} ∑* is defined as follows:
∑* = ∑0 U ∑1 U ∑2 U ∑3 …….. = {ε, a, b,
= {aa, ab, ba, bb}. {a, b} aa, ab, ba, bb, aaa,….}
= {aaa, aab, aba, abb, baa, bab, bba, bbb}
… etc The Positive closure of , denoted by
∑+ is defined as follows:
∑+ = ∑* - { ε }

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Operations on Languages
• Complementation
• Union
• Intersection
• Concatenation
• Reversal
• Closure

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Complementation
Let L be a language over an alphabet, Σ.
The complementation of L, denoted by L ( or L’), is Σ*– L.
Example:
Let Σ = {0, 1} be the alphabet.
L = {Σ* | the number of 1’s in  is even}.
L’ = {Σ* | the number of 1’s in  is not even}.

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Union
Let L1 and L2 be languages over an alphabet Σ.
The union of L1 and L2, denoted by L1L2, is
L1L2 = {x | x is in L1 or L2}.
Example:
{x {0,1}* | x begins with 0 }  {x{0,1}*|x ends with 0}
= {x  {0,1}*| x begins or ends with 0}

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Intersection
Let L1 and L2 be languages over an alphabet Σ.
The intersection of L1 and L2, denoted by L1L2, is
{ x | x is in L1 and L2}.
Example:
{ x{0,1}* | x begins with 0}  { x{0,1}*| x ends with 0}
= { x{0,1}*| x begins and ends with 0}

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Concatenation
Let L1 and L2 be languages over an alphabet Σ.
The concatenation of L1 and L2, denoted by L1L2, is
{w1w2| w1 is in L1 and w2 is in L2}.
Example
L1 = {x  {0,1}*| x begins with 0}
L2 = {x  {0,1}*| x ends with 0}
L1L2 = { x  {0,1}*| x begins and ends with 0 and length(x)  2}

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Reversal
Let L be a language over an alphabet Σ.
The reversal of L, denoted by LR, is {wR | w is in L}.
Example
L = {x  {0,1}*| x begins with 0}
LR = {x  {0,1}*| x ends with 0}

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Kleene’s closure
Let L be a language over an alphabet Σ.
The Kleene’s closure of L, denoted by L*, is {x | for an integer
n  0, x = x1 x2 … xn and x1, x2 , …, xn are in L}.
That is, L* = i=0Li

Example: Let Σ = {0,1} and

Le* = {Σ* | the number of 1’s in  is even}
(Le)* = {Σ*| the number of 1’s in  is odd}

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Central Concepts of Automata Theory – cont…
• Grammar
• A grammar of a language, G is defined as G = (V, T, S, P)
• Where,
V or N➔ Finite set of objects called Variables / Non-terminals
T ➔ Finite set of objects called Terminals
SV ➔ Start Symbol
P ➔ Finite set of Productions

Ex: G = {(S, T), (a, b), S, P}

Note: N  T = 

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Grammar (Examples)
G = ({S, A}, {a, b}, P, S), where
P: S → aA
A→ b
S

a A

L(G) = { ab }

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Grammar
G = ({S}, {a, b}, P, S), where
P: S → aS
S→b S S S S

a S a S
b a S
a a S
S
b
a S
b
L(G) = { b, ab, a2b, a3b,. . . } b

L(G) = { anb / n ≥ 0 }

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Example problems on grammar
1. Given  = a, b} obtain 
• Give an example of a finite language in 
• Given L = {anbn : n>=0}, check if strings aabb, abb are in language, L
• Soln:
•  =  a, b, ab, ba, ….}
• {a, aa, aab} → Some example of a language in 
• String: ‘aabb’
L = { ab, aabb, aaabbb, …}
When n = 2, L = a2 b2 therefore, it is a string in L
• String: ‘abb’
This is not a string in L. Since, there is no equal number of a’s and b’s

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Problem 2
• Given u = a2ba3b2; v = ba. Find (i) uv (ii) uv2 (iii)|uv|
• Soln:
(i) uv = (a2ba3b2).(ba) = a2ba3b3a

(ii) uv2 = (a2ba3b2).(ba).(ba) = a2ba3b3aba

(iii) |uv| = |a2ba3b3a| = |aabaaabbba| = 10

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Exercise
• Consider the following two languages:
L1 = {aa, bb} where  = {a, b}
L2 = {ppp, qqq} where  = {p, q}
Compute the following:
Solution:
a) L3 = (L1)2 . L2
L3 = {aaaa,aabb, bbaa, bbbb} . {ppp, qqq}
b) L4 = L2 . 
L4 = {ppp, qqq} .  = L2
c) L5 = L2 – L3
L5 = L2 = {ppp, qqq}
d) L6 = L2  L4 L6 = {ppp, qqq}
e) L7 = (L5)* L7 = {, ppp, qqq, pppppp, qqqqqq, ….}

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Problem – Construct Language from Grammar
• Given a grammar, G = [{S}, {a, b}, S, P] where Production, P is given by S → a S b,
S → . Obtain language L(G).
Soln:
S ==> a S b
==> a a S b b
==> a a a S b b b
==>….
==> an S bn
==> an bn

L (G) = {an bn |where n0} or

L(G) = {set of strings with n a’s followed by n b’s |where n0}
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Problem 1
• Given a grammar, G = [{A, S}, {a, b}, A, P] where Production, P is
given by A → a S b, S → S b |. Obtain language L(G).
Soln:
A ==> a S b
==> a S b b
==> a S b b b
==> ….
==> a S bn
==> a bn

L(G) = { a bn | where n1} or

L(G) = { All strings with exactly one ‘a’ and n b’s}
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Problem 2
G= ({S, B}, {a, b, c }, P, S), where
P: S → aSBc (rule 1)
S → abc
S → abc (rule 2)
cB → Bc (rule 3) S → aSBc → aabcBc → aabBcc → aabbcc
bB →bb (rule 4)
S → aSBc → aaSBcBc → aaabcBcBc →
aaabBccBc → aaabbccBc → aaabbcBcc →
aaabbBccc → aaabbbccc

L(G) = { anbncn/ n ≥ 1 }

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Problem 3
G= ({S}, {a, b, c }, P, S), where
P: S → aSa (rule 1)
S → bSb (rule 2)
S → c (rule 3) S →c
S → aSa → aca
S → bSb → bcb
S → aSa → abSba → abcba
S → bSb → baSab → bacab

L(G) = { wcwR/ w ∈ {a, b}* }

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Construction of grammar
1) L(G) = { ancan / n ≥ 0 }
G= ({S}, {a, c}, P, S), where
P: S → aSa
S→c
2) L(G) = { anb2n / n ≥ 0 }
G= ({S}, {a, b}, P, S), where
P: S → aSbb OR P: S → aSA, A → bb
S→  S→ 
3) L(G) = { an+2bn / n ≥ 1 }
G= ({S}, {a, b}, P, S), where
P: S → aSb
S → aaab
Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT
Problem 4
• Find the grammar for  = {0, 1} that generate the set of
all strings containing more 0’s than 1’s where 0’s
followed by 1’s.
• Soln:
A→ 0 S
S→0S1|

G=({A,S},{0,1},A,P)

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Practice problems

1)L(G) = { anbn-3 / n ≥ 3 }

2)L(G) = { anbm / n ≥ 0, m ˃ n }

3)L(G) = { anbn cm / n, m ≥ 1 }

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Solution to practice problems
1) L(G) = { anbn-3 / n ≥ 3 }
3) L(G) = { anbncm / n, m ≥ 1 }
G= ({S}, {a, b }, P, S), where G= ({S, A, B}, {a, b, c}, P, S), where
P: S → aSb P: S → AB
S → aaa A → aAb / ab
B → cB / c
2) L(G) = { anbm / n ≥ 0, m ˃ n }
G= ({S, A}, {a, b }, P, S), where
P: S → aSb / A
A → bA / b

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Chomsky Hierarchy – Types of grammar

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT

Chomsky Hierarchy
Language Grammar Machine

Regular grammars
Finite-state
Type 3 Regular languages • Right-linear grammars
automata
• Left-linear grammars

Push-down
Type 2 Context-free languages Context-free grammars
automata

Linear-bound
Type 1 Context-sensitive languages Context-sensitive grammars
automata

Recursive languages
Type 0 Unrestricted grammars Turing machines
Recursively enumerable languages

Winter 2023-24 Dr. Somasundaram S K, Dept. of IS, SCOPE, VIT