Lecture# 13

Compiler Construction
Lexical Analyzer Generators (NFA to DFA Conversion),
Thompson Construction Algorithm,
(Part-III)
by Safdar Hussain
Khwaja Fareed University, RYK
Pakistan
Topics
• Lexical Analyzer Generator Design, NFA & DFA
• Subset Construction Algorithm, Simulating a DFA
• NFA to DFA Conversion (Thompson Construction Algorithm)
 Overview
This chapter contains comprehensive material
• Building a simple Lexical Analyzer
– Lexical Analysis (Every perspective)
– Interaction of Lexical analyzer & Parsing
– Implementing Transition Diagrams
– Design of a Lexical Analyzer (RENFADFA)
– The Subset Construction Algorithm
– Minimization of number of states of a DFA
– Conversion RE to DFA directly (Algorithm & Tree
annotation)

Safdar Hussain, Khwaja Fareed University, RYK 2

Design of a Lexical Analyzer Generator
• Translate regular expressions to NFA
• Translate NFA to an efficient DFA
Optional

regular
NFA DFA
expressions

Simulate NFA Simulate DFA

to recognize to recognize
tokens tokens

Safdar Hussain, Khwaja Fareed University, RYK 3

 Nondeterministic Finite Automata
• Definition: an NFA is a 5-tuple (S,,,s0,F)
where

S is a finite set of states

 is a finite set of input symbol alphabet
 is a mapping from S to a set of states
s0  S is the start state
F  S is the set of accepting (or final) states

Safdar Hussain, Khwaja Fareed University, RYK 4

 Transition Graph/Table
• An NFA can be diagrammatically represented by a
labeled directed graph called a transition graph
a
S = {0,1,2,3}
start a b b  = {a, b}
0 1 2 3
s0 = 0
b F = {3}
Input Input
State
(0,a) = {0,1} a b
(0,b) = {0} 0 {0, 1} {0}
(1,b) = {2} 1 - {2}
(2,b) = {3}
2 - {3}
Safdar Hussain, Khwaja Fareed University, RYK 5
 Deterministic Finite Automata
• A deterministic finite automaton is a special case of
an NFA
– No state has an -transition
– For each state s and input symbol a there is at most one
edge labeled a leaving s
• Each entry in the transition table is a single state
– At most one path exists to accept a string
– Simulation algorithm is simple

Safdar Hussain, Khwaja Fareed University, RYK 6

 Conversion of an NFA into a DFA
• The subset construction algorithm converts an
NFA into a DFA using:
-closure(s) = {s}  {t | s  …  t}
-closure(T) = sT -closure(s)
move(T,a) = {t | s a t and s  T}
• The algorithm produces:
Dstates is the set of states of the new DFA
consisting of sets of states of the NFA
Dtran is the transition table of the new DFA

Safdar Hussain, Khwaja Fareed University, RYK 7

Conversion of an NFA into a DFA

Operations on NFA States

Safdar Hussain, Khwaja Fareed University, RYK 8

Deterministic Finite Automata
A DFA is an NFA with the following restrictions:
•  moves are not allowed
• For every state s S, there is one and only one
path from s for every input symbol a  .
Since transition tables don’t have any alternative options, DFAs
are easily simulated via an algorithm.

Safdar Hussain, Khwaja Fareed University, RYK 9

 Deterministic Finite Automata
Algorithm: Simulating a DFA

Simulating a DFA s  s0
c  nextchar;
while c  eof do
s  move(s,c);
c  nextchar;
end;
if s is in F then return “yes”
else return “no”
Safdar Hussain, Khwaja Fareed University, RYK 10
 Example – NFA to DFA
NFA: RE = (a | b)*abb
a
start a b b
0 1 2 3

DFA:
b
a
start a b b
0 1 2 3
a
b a
What Language is Accepted?
Safdar Hussain, Khwaja Fareed University, RYK 11
 The Subset Construction Algorithm
Initially, -closure(s0) is the only state in Dstates and it is unmarked
while there is an unmarked state T in Dstates do
mark T
for each input symbol a   do
U := -closure(move(T,a))
if U is not in Dstates then
add U as an unmarked state to Dstates
end if
Dtran[T,a] := U
end do
end do

Safdar Hussain, Khwaja Fareed University, RYK 12

 Subset Construction Example 1

a RE = (a | b)*abb
2 3

start    a b b
0 1 6 7 8 9 10

4
b
5


b
Dstates
C A = {0,1,2,4,7}
b
b a B = {1,2,3,4,6,7,8}
start a b b C = {1,2,4,5,6,7}
A B D E
a D = {1,2,4,5,6,7,9}
a
a E = {1,2,4,5,6,7,10}

Safdar Hussain, Khwaja Fareed University, RYK 13

Illustrating Conversion – Example 1
Start with NFA: RE = (a | b)*abb

First we calculate: -closure(0) (i.e., state 0)

-closure(0) = {0, 1, 2, 4, 7} (all states reachable from 0
on -moves)
Let A={0, 1, 2, 4, 7} be a state of new DFA, D.
Safdar Hussain, Khwaja Fareed University, RYK 14
 Conversion Example – continued
2nd , we calculate: a : -closure(move(A,a))
and b : -closure(move(A,b))
a : -closure(move(A,a)) = -closure(move({0,1,2,4,7},a))}
adds {3,8} ( since move(2,a)=3 and move(7,a)=8)

From this we have : -closure({3,8}) = {1,2,3,4,6,7,8}

(since 36 1 4, 6 7, and 1 2 all by -moves)
Let B={1,2,3,4,6,7,8} be a new state. Define Dtran [A,a] = B.

b : -closure(move(A,b)) = -closure(move({0,1,2,4,7},b))
adds {5} ( since move(4,b)=5)

From this we have : -closure({5}) = {1,2,4,5,6,7}

(since 56 1 4, 6 7, and 1 2 all by -moves)
Let C={1,2,4,5,6,7} be a new state. Define Dtran [A,b] = C.
Safdar Hussain, Khwaja Fareed University, RYK 15
 Conversion Example – continued
3rd , we calculate for state B on {a,b}
a : -closure(move(B,a)) = -closure(move({1,2,3,4,6,7,8},a))}
= {1,2,3,4,6,7,8} = B
Define Dtran [B,a] = B.

b : -closure(move(B,b)) = -closure(move({1,2,3,4,6,7,8},b))}
= {1,2,4,5,6,7,9} = D
Define Dtran [B,b] = D.

4th , we calculate for state C on {a,b}

a : -closure(move(C,a)) = -closure(move({1,2,4,5,6,7},a))}
= {1,2,3,4,6,7,8} = B
Define Dtran [C,a] = B.

b : -closure(move(C,b)) = -closure(move({1,2,4,5,6,7},b))}
= {1,2,4,5,6,7} = C
Define Dtran [C,b] = C.
Safdar Hussain, Khwaja Fareed University, RYK 16
 Conversion Example – continued
5th , we calculate for state D on {a,b}
a : -closure(move(D,a)) = -closure(move({1,2,4,5,6,7,9},a))}
= {1,2,3,4,6,7,8} = B
Define Dtran[D,a] = B.

b : -closure(move(D,b)) = -closure(move({1,2,4,5,6,7,9},b))}
= {1,2,4,5,6,7,10} = E
Define Dtran[D,b] = E.

Finally, we calculate for state E on {a,b}

a : -closure(move(E,a)) = -closure(move({1,2,4,5,6,7,10},a))}
= {1,2,3,4,6,7,8} = B
Define Dtran[E,a] = B.

b : -closure(move(E,b)) = -closure(move({1,2,4,5,6,7,10},b))}
= {1,2,4,5,6,7} = C
Define Dtran[E,b] = C.
Safdar Hussain, Khwaja Fareed University, RYK 17
Conversion Example – continued
This gives the transition table Dtran for the DFA of:
Input Symbol
Dstates a b
A B C
B B D
C B C
D B E
E B C
b

b C b

start A a B b D b E
a
a a

Safdar Hussain, Khwaja Fareed University, RYK 18

 Subset Construction Example 2
a
1 2 a1

start
0  3
a
4
b
5
b
6 a2
a b

7 b 8 a3
b
Dstates
a3

?
C A = {0,1,3,7}
b a
b b B = {2,4,7}
start C = {8}
A D
D = {7}
a a
b b E = {5,8}
B E F
Solve step-by-step by yourself F = {6,8}
a1 a3 a2 a3
Safdar Hussain, Khwaja Fareed University, RYK 19
Converting NFA to DFA – Example 3


a 3 b 4
2
 
0  1 5  8
 

6 c 7

From State 0, Where can we move without consuming any input ?

This forms a new state: 0,1,2,6,8 What transitions are defined for
this new state ?

Safdar Hussain, Khwaja Fareed University, RYK 20

 The Resulting DFA
a

0, 1, 2, 6, 8 a 3
a
c b

1, 2, 5, 6, 7, 8
c 1, 2, 4, 5, 6, 8

c
Which States are FINAL States ?
a
A How do we handle
B
a a alphabet symbols not
c b defined for A, B, C, D ?
D
c C
c

Safdar Hussain, Khwaja Fareed University, RYK 21

 NFA Vs DFA


a
1 C
 
start   a b
A B E 3 4 5

  b
b #
2 D FF 6

b
b

123 a 1234 b 1235 b 123

1236

a a a 22
The End

23