Theory of Languages and

Automata

Chapter 1: An Introduction to
the Theory of Computation (2)

Lecturer: Reza Entezari-Maleki

entezari@iust.ac.ir
Department of Computer Engineering
Iran University of Science and Technology
 Outlines

• Models of Computation

• Automata

• Languages

• Grammars

2
Models of Computation

3
Computation

CPU memory

4
temporary memory

input memory
CPU
output memory

Program memory

5
 3
Example: f ( x) = x

temporary memory

input memory
CPU
output memory
Program memory
compute x∗x
2
compute x ∗x
6
 3
f ( x) = x

temporary memory
input memory
x=2
CPU
output memory
Program memory
compute x∗x
2
compute x ∗x
7
 3
temporary memory f ( x) = x
z = 2*2 = 4
f ( x) = z * 2 = 8
input memory
x=2
CPU
output memory
Program memory
compute x∗x
2
compute x ∗x
8
 3
temporary memory f ( x) = x
z = 2*2 = 4
f ( x) = z * 2 = 8
input memory
x=2
CPU
f ( x) = 8
Program memory output memory
compute x∗x
2
compute x ∗x
9
Automata

10
 Automaton
temporary memory

Automaton
input memory
CPU
output memory

Program memory

11
 Different Kinds of Automata
Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

12
 Finite Automaton

temporary memory

input memory
Finite
Automaton
output memory

Vending Machines (small computing power)

13
 Pushdown Automaton
Stack Push, Pop

input memory
Pushdown
Automaton
output memory

Programming Languages (medium computing power)

14
 Turing Machine

Random Access Memory

input memory
Turing
Machine
output memory

Algorithms (highest computing power)

15
 Power of Automata

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
16
Languages

17
 Languages
A language is a set of strings

String: A sequence of letters

Examples: “cat”, “dog”, “house”, …

Defined over an alphabet:

Σ = {a, b, c,, z}
18
 Alphabets and Strings
We will use small alphabets: Σ = {a, b}

Strings
a
ab u = ab
abba v = bbbaaa
baba w = abba
aaabbbaabab
19
 String Operations

w = a1a2  an abba
v = b1b2 bm bbbaaa

Concatenation

wv = a1a2  anb1b2 bm abbabbbaaa

20
w = a1a2  an ababaaabbb

Reverse

R
w = an  a2 a1 bbbaaababa

21
 String Length
w = a1a2  an
Length: w =n

Examples: abba = 4
aa = 2
a =1
22
 Recursive Definition of Length
For any letter: a =1

For any string wa : wa = w + 1

Example: abba = abb + 1

= ab + 1 + 1
= a +1+1+1
= 1+1+1+1
=4 23
 Length of Concatenation

uv = u + v

Example: u = aab, u = 3
v = abaab, v = 5

uv = aababaab = 8
uv = u + v = 3 + 5 = 8
24
 Proof of Concatenation Length
Claim: uv = u + v

Proof: By induction on the length v

Induction basis: v =1

From definition of length:

uv = u + 1 = u + v
25
Inductive hypothesis: uv = u + v

for v = 1,2,, n

Inductive step: we will prove uv = u + v

for v = n +1

26
 Inductive Step
Write v = wa, where w = n, a = 1

From definition of length: uv = uwa = uw + 1

wa = w + 1
From inductive hypothesis: uw = u + w

Thus: uv = u + w + 1 = u + wa = u + v
27
 Empty String
A string with no letters: λ

Observations: λ =0

λw = wλ = w

λabba = abbaλ = abba

28
 Substring
Substring of string:
a subsequence of consecutive characters

String Substring
abbab ab
abbab abba
abbab b
abbab bbab
29
 Prefix and Suffix
abbab
Prefixes Suffixes
λ abbab w = uv
a bbab
prefix
ab bab
suffix
abb ab
abba b
abbab λ
30
 Another Operation
n
w =
ww

w
n
2
Example: (abba ) = abbaabba

0
Definition: w =λ
0
(abba ) = λ
31
 The * Operation
Σ * : the set of all possible strings from
alphabet Σ

Σ = {a, b}
Σ* = {λ , a, b, aa, ab, ba, bb, aaa, aab,}

32
 The + Operation
+ : the set of all possible strings from
Σ
alphabet Σ except λ

Σ = {a, b}
Σ* = {λ , a, b, aa, ab, ba, bb, aaa, aab,}

+
Σ = Σ * −λ
+
Σ = {a, b, aa, ab, ba, bb, aaa, aab,}
33
 Language
A language is any subset of Σ*

Example: Σ = {a, b}
Σ* = {λ , a, b, aa, ab, ba, bb, aaa,}

Languages: {λ}
{a, aa, aab}
{λ , abba, baba, aa, ab, aaaaaa} 34
 Note that:

Sets ∅ = { } ≠ {λ}

Set size {} = ∅ = 0

Set size {λ} = 1

String length λ =0
35
 Another Example

n n
An infinite language L = {a b : n ≥ 0}

λ
ab
∈L abb ∉ L
aabb
aaaaabbbbb
36
 Operations on Languages
The usual set operations

{a, ab, aaaa}  {bb, ab} = {a, ab, bb, aaaa}

{a, ab, aaaa}  {bb, ab} = {ab}
{a, ab, aaaa} − {bb, ab} = {a, aaaa}
Complement: L = Σ * −L

{a, ba} = {λ , b, aa, ab, bb, aaa,}

37
 Reverse

R R
Definition: L = {w : w ∈ L}
R
Examples: {ab, aab, baba} = {ba, baa, abab}

n n
L = {a b : n ≥ 0}

R n n
L = {b a : n ≥ 0}
38
 Concatenation

Definition: L1L2 = {xy : x ∈ L1, y ∈ L2 }

Example: {a, ab, ba}{b, aa}

= {ab, aaa, abb, abaa, bab, baaa}

39
 Another Operation
Definition: n
LL
L = L
n
3
{a, b} = {a, b}{a, b}{a, b} =
{aaa, aab, aba, abb, baa, bab, bba, bbb}
Special case: L = {λ}
0

{a, bba, aaa} = {λ}

0

40
 More Examples

n n
L = {a b : n ≥ 0}

2 n n m m
L = {a b a b : n, m ≥ 0}

2
aabbaaabbb ∈ L

41
 Star-Closure

0 1 2
Definition: L* = L  L  L 

Example:
λ , 
a, bb, 
 
{a, bb}* =  
 aa , abb , bba , bbbb , 
aaa, aabb, abba, abbbb,
42
 Positive Closure

Definition: + 1 2
L = L  L 
= L * −{λ }

a, bb, 
+  
{a, bb} = aa, abb, bba, bbbb, 
aaa, aabb, abba, abbbb,
 
43
Grammars

44
 Grammars
Grammars express languages

Example: the English language

sentence → noun _ phrase predicate

noun _ phrase → article noun

predicate → verb 45
article → a
article → the

noun → boy
noun → dog

verb → runs
verb → walks 46
A derivation of “the boy walks”:

sentence ⇒ noun _ phrase predicate

⇒ noun _ phrase verb
⇒ article noun verb
⇒ the noun verb
⇒ the boy verb
⇒ the boy walks
47
A derivation of “a dog runs”:

sentence ⇒ noun _ phrase predicate

⇒ noun _ phrase verb
⇒ article noun verb
⇒ a noun verb
⇒ a dog verb
⇒ a dog runs
48
Language of the grammar:

L = { “a boy runs”,
“a boy walks”,
“the boy runs”,
“the boy walks”,
“a dog runs”,
“a dog walks”,
“the dog runs”,
“the dog walks” }
49
 Notation

noun → boy
noun → dog

Variable Terminal
Production
or
rule
Non-terminal

50
 Another Example
Grammar: S → aSb
S →λ

Derivation of sentence ab :

S ⇒ aSb ⇒ ab

S → aSb S →λ
51
Grammar: S → aSb
S →λ

Derivation of sentence aabb :

S ⇒ aSb ⇒ aaSbb ⇒ aabb

S → aSb S →λ
52
Other derivations:

S ⇒ aSb ⇒ aaSbb ⇒ aaaSbbb ⇒ aaabbb

S ⇒ aSb ⇒ aaSbb ⇒ aaaSbbb

⇒ aaaaSbbbb ⇒ aaaabbbb

53
Language of the grammar

S → aSb
S →λ

n n
L = {a b : n ≥ 0}

54
 More Notation

Grammar G = (V , T , S , P )
V : Set of variables

T : Set of terminal symbols

S : Start variable

P: Set of Production rules

55
 Example

Grammar G: S → aSb
S →λ

G = (V , T , S , P )

V = {S } T = {a, b}
P = {S → aSb, S → λ}
56
 More Notation
Sentential Form:
A sentence that contains
variables and terminals

Example:

S ⇒ aSb ⇒ aaSbb ⇒ aaaSbbb ⇒ aaabbb

Sentential Forms sentence

57
 *
We write: S ⇒ aaabbb

Instead of:

S ⇒ aSb ⇒ aaSbb ⇒ aaaSbbb ⇒ aaabbb

58
 *
In general we write: w1 ⇒ wn

If: w1 ⇒ w2 ⇒ w3 ⇒  ⇒ wn

59
 *
By default: w ⇒ w

60
 Example
Grammar Derivations
S → aSb *
S ⇒λ
S →λ
*
S ⇒ ab
*
S ⇒ aabb
*
S ⇒ aaabbb
61
 Example
Grammar Derivations
S → aSb ∗
S ⇒ aaSbb
S →λ
∗
aaSbb ⇒ aaaaaSbbbbb

62
 Another Grammar Example
Grammar G : S → Ab
A → aAb
A→λ

Derivations:
S ⇒ Ab ⇒ b
S ⇒ Ab ⇒ aAbb ⇒ abb
∗
S ⇒ aAbb ⇒ aaAbbb ⇒ aabbb
63
 More Derivations
S ⇒ Ab ⇒ aAbb ⇒ aaAbbb ⇒ aaaAbbbb
⇒ aaaaAbbbbb ⇒ aaaabbbbb

∗
S ⇒ aaaabbbbb
∗
S ⇒ aaaaaabbbbbbb
∗
n n
S ⇒a b b
64
 Language of a Grammar

For a grammar G
with start variable S :

∗
L(G ) = {w : S ⇒ w}

String of terminals
65
 Example
For grammar G: S → Ab
A → aAb
A→λ

n n
L(G ) = {a b b : n ≥ 0}

∗
n n
Since: S ⇒a b b
66
 A Convenient Notation

A → aAb
A → aAb | λ
A→λ

article → a
article → a | the
article → the
67