1 - Computing Models and The Power of Writing

Computing Models and The Power of Writing
Carolina Mejia, J. Andres Montoya

February 22, 2021
1
Part I
Deterministic Algorithms and
Machines
2
1 Language Recognition
Let be a …nite set. We can use the elements of as symbols of an alphabet,
and construct …nite strings using these symbols. Let be the set of all …nite
strings that can be constructed with the symbols of : We can use those in…nitely
many strings to encode inputs of algorithms. One of the most basic algorithmic
tasks related to those codes is to distinguish between the codes that hold some
speci…c property and the codes that do not hold this property.
De…nition 1 A language over is a set L :
Given a language L; de…ned by some abstract property, it determines an

algorithmic problem, the problem:
Problem 2 Recognition(L)
Input: w 2 :
Problem: decide if w 2 L:
We focus on the above type of problems, which are called decision problems.
Then, when we think in constructing an abstract model of a computing machine,
we think in devices that deal with decision problems.
Remark 3 Perhaps, when the reader thinks in a computing machine, or an

algorithm, she thinks in a device that computes functions from a set like N
into a set like N. Let us suppose that this is the case, and let us use the term
arithmetic functions to denote those functions.
The set N can be identi…ed with the set f1g (the unary basis), with the set
f0; 1g (the binary basis), and with the set f0; 1; :::; k 1g (the k-ary basis). If
we …x a basis, that is, if we identify the set N with the set f0; 1; :::; k 1g ; (for
some k 1), then the computation of arithmetical functions reduces to the com-
putation of functions from the set f0; 1; :::; k 1g into the set f0; 1; :::; k 1g :
Let f : f0; 1; :::; k 1g ! f0; 1; :::; k 1g be one of those functions. The com-
putation of f can in turn be reduced to the recognition of its graph, and this
latter problem can be de…ned as a decision problem, the problem
Input: w 2 (f0; 1; :::; k 1g f0; 1; :::; k 1g) :

Problem: decide if (w1 ; w2 ) 2 graph (f ) ; where given
w = (k1 ; n1 ) (k2 ; n2 ) (km ; nm ) ;
we have that
w 1 = k1 k2 km and w2 = n1 n2 nm :
3
We get that the computation of any computable function can be reduced to
an appropriate decision problem.
Let us consider an elementary language, the language pal; constituted by all

the binary palindromes, that is:
pal = w 2 f0; 1g : w = wR ;
where wR denotes the reverse of w (if w = w [1] w [m] ; then wR = w [m] w [1]).
Our …rst goal is to design a model of automata that can solve the problem
Recognition(pal) : The goal does not seem to be very demanding. Then, we
could begin trying with the most elementary devices.
References
[1] M. Sipser. Introduction to the Theory of Computation. PWS Publishing,
Boston, 1997.
4
2 Finite State Automata
At the beginning there was only read-once automata...
Finite-state machines are a simple type of automata that is studied in au-
tomata theory and the theory of computation. This model was introduced by
W. McCulloch, W. Pitts who though of it as a combinatorial model of human
neurons and neural activity [1].
2.1 De…nition of the Model

Notation 4 Let us …x some notation:
n
1. Let be a …nite alphabet. We use the symbol to denote the subset of
constituted by all the strings of length n:
2. Let w 2 n , and let i n, we use the symbol w [i] to denote the i-th
character of w:
3. Let w 2 n , and let i < j n, we use the symbol w [i; :::; j] to denote the
substring w [i] w [i + 1] w [j] :
4. We use the symbol jwj to denote the length of w:
Let L be a language. We want to process the strings in ; and

tell apart those strings that belong to L: Then, we need to have the ability of
scanning those strings. Therefore, our model of computation must have an input
unit, where the input string w can be written, and which can be accessed by
the machine in order to read the characters of w; either character by character
or in a global fashion. It is not easy to …gure out a machine that can read a
long string in one time unit. It is very much easier to …gure out a machine that
can read the characters of w in a sequential fashion. Thus, let us begin with
an elementary architecture, an automaton provided with an input tape and a
scanning head that can move along the input tape, reading the characters of the
input string. We have to observe that it is useless to read a string if one cannot
save a minimum of information about the characters that are being read. Thus,
our machine must be also equipped with a memory unit that allows it to save
information about the characters that have been read (see the …gure below).
The formal de…nition of …nite state automata goes as follows:
De…nition 5 A …nite state automaton is a tuple M = ( ; Q; q0 ; A; ) ; such

that:
1. is a …nite set of characters called the input alphabet of M.
5
2. Q is a …nite set of inner states used to save information about the char-
acters that have been read. If M is in state q after reading the string w;
then q corresponds to the summary taken by M about the text w:
3. q0 2 Q is the initial state. When M is ready to start the scanning of w;
this automaton should be in a state that represents the fact that M has
not read a single character, this state is the initial state q0 :
4. A Q is the set of accepting states.
5. is the transition function of M, which is a function from the set Q
into the set Q: Notice that an element of Q is a pair (q; a) that can
represent the data that can be accessed by the automaton in a given instant
of the computation: the character a that is written on the cell that is
being read, and the state q that is written on its inner memory at this
time instant. On the other hand, an element of Q is a state p that can
represent the e¤ ect exerted by those data on the computation: a change
in the content of the inner memory that switches from the state q to the
state p:
The computation of M, on input w; proceeds as follows:
1. At instant 1 the string w is written on the input tape, the i-th character
is written on the i-th cell. The scanning head is placed on the …rst cell,
and the inner state is equal to q0 :
2. At instant 2 the scanning head is placed on cell 2; and the inner state has
changed to (q0 ; w [1]) :
3. At instant i the scanning head is placed on cell i: The inner state, at this
time instant, is equal to b (q0 ; w [1; :::; i]) ; where b is the extension of to
the set Q that is de…ned by the following recursion:
For all q 2 Q and for all a 2 the equality b (q; a) = (q; a) holds.
n
For all q 2 Q and for all u 2 the equality
b (q; u) = b (q; u [1; :::; n 1]) ; u [n]
holds.
4. At instant jwj + 1 the scanning head is placed on the …rst empty cell. This
is the halting condition for M, which is always reached after jwj + 1 steps.
The state reached by the automaton is the state b (q0 ; w) : We say that M
accepts the input, if and only if, state b (q0 ; w) belongs to A:
It is important to observe that:
6
1. Finite state automata cannot write. Those automata can only read and
save a bounded amount of information from its readings (at most log (jQj)
bits).
2. Finite state automata are read-once devices, that is: the i-th cell is scanned
only once, at the i-th instant of the computation.
3. Finite state automata are real-time devices.
We ask: which are the languages that can be recognized by …nite state
automata?
2.2 Regular Languages

The languages accepted by …nite state automata are called regular (also rational)
languages.
De…nition 6 Let M = ( ; Q; q0 ; A; ) be a …nite state automaton, we use the

symbol L (M) to denote the language accepted by M; which is the language
n o
w2 : b (q0 ; w) 2 A :
We say that a language L is a regular language, if and only if, there exists M
such that L = L (M) :
The set of regular languages is a set that contains nontrivial languages.

3
Exercise 7 Homework 1. Let = f0; 1; :::; 9g : Let (a; b; c) 20 , 1
we can
a
write this character (and any other character) as a column vector @ b A : Let
c
S be the language
80 1 0 1 9
< a1 am =
@ b1 A @ bm A : m 1 and a1 am + b1 bm = c1 cm :
: ;
c1 cm
We use the symbol REG to denote the set of all regular languages. The set
REG is closed under several operations.
Exercise 8 Prove that REG is closed under unions, intersections and taking
complements.
Exercise 9 Let L; T be two languages, we de…ne the product of L and T

by
L T = fuw 2 : u 2 L and w 2 T g :
Prove that REG is closed under products.
7
Exercise 10 De…ne the star of L as the language
L = fu1 un : n 1 & u1 ; :::; un 2 Lg :
Prove that REG is closed under stars.
We have to conclude, given the above facts, that REG is a large set that
could contain many more languages that we though at a …rst glance. Does REG
contain the language pal?
2.3 The language pal is Nonregular

Let us prove that pal is nonregular.
Theorem 11 The language pal is nonregular.
Proof. Suppose that pal is regular, and let M = (f0; 1g ; Q; q0 ; A; ) be a

…nite state automaton that accepts pal: Let n = jQj and let m be a positive
integer such that 2m > n: Let be the restriction of b to the set f0; 1g : We
m
m
have that cannot be injective. Then, there exists u; w 2 f0; 1g such that
b (q0 ; u) = b (q0 ; w) : We have that:
1. b q0 ; wwR 2 A:
2. uwR 2
= pal:
3. b q0 ; wwR = b q0 ; uwR (see the exercise below).
Note that we get a contradiction, we get that b q0 ; uwR 2 A and hence M

accepts a string that is not a palindrome. The theorem is proved.
Exercise 12 Let M = ( ; Q; q0 ; A; ) be a …nite state automaton, and let u; w

be two input strings such that b (q0 ; u) = b (q0 ; w) : Let X 2 ; prove that the
b b
equality (q0 ; uX) = (q0 ; wX) holds.
References
[1] W. McCulloch, W. Pitts. A Logical Calculus of the Ideas Immanent in Ner-
vous Activity. Bulletin of Mathematical Biophysics. 5 (4): 115 - 133, 1943.
8
3 Myhill-Nerode
The theorem of Myhill-Nerode provides us with a characterization of regular

languages.
Let L be a regular language and let M = ( ; Q; q0 ; A; ) be a …nite state
automaton recognizing L: The function b de…nes an equivalence relation over
the set that has at most jQj equivalence classes. This relation is de…ned by
vRM w; if and only if, b (q0 ; v) = b (q0 ; w)
Let v; w be two strings and suppose that the equality b (q0 ; v) = b (q0 ; w)
holds; we get that for all string X the equality b (q0 ; vX) = b (q0 ; wX) also
holds, and we get that for all string X the condition
vX 2 L, if and only if, wX 2 L
holds.
Exercise 13 Let L be a regular language and let M be a DFA. De…ne SL as:

vSL w; if and only if, for all string X the condition
vX 2 L; if and only if, wX 2 L
holds. Prove that SL is an equivalence relation, which is coarser than RM ,

concludes that SL is a …nite-index relation.
De…nition 14 Let L be a language, we say that v; w 2 are L-

equivalent (vSL w), if and only if, for all X 2 it happens that vX 2 L,
if and only if, wX 2 L:
Exercise 15 Prove that SL is an equivalence relation.
Theorem 16 Let L , we have that L is regular, if and only if, SL is a

…nite index relation.
Proof. We have already proved one of the two implications. Let us focus on
the second one. Thus, suppose that SL is a …nite-index relation. Let QSL be
the …nite set SL : De…ne MSL as QSL ; ; q0SL ; ASL ; SL ; where:
1. State q0SL is equal to ["]SL ; where " is the empty string and given w 2
the symbol [w]SL denotes the equivalence class of w:
2. The set of accepting states is the set
ASL = [w]SL : w 2 L :
9
3. The transition function of MSL is de…ned by the equation
SL [w]SL ; a = [wa]SL :
It is easy to check that MSL accepts the language L:
Exercise 17 Homework 3. We can use Myhill-Nerode Theorem to prove of

certain languages that they are regular, and we can use the same theorem to
prove of some other languages that they are nonregular.
1. Prove that the language consisting of binary representations of numbers

that can be divided by 3 is a regular language.
2. Use the theorem to give an alternative proof of the nonregularity of pal:
3.1 Minimal Automata

Let L be a regular language, and let M be an automaton accepting L: If we add
to M a …nite set of states, which are all inaccessible from q0 ; we get a larger
automaton recognizing the same language L: Thus, we get that the regular
language L can be de…ned by in…nitely many …nite state automata. Set
m (L) = min fjQj : ( ; Q; q0 ; A; ) accepts Lg :
Consider the problem
Problem 18 Automata Minimization
Input: H, where H is a …nite state automaton.

Problem: compute an automaton N , such that L (H) = L (N ) and such
that the number of states of N is equal to m (L (H)) :
The theorem of Myhill-Nerode provides us with a construction of N . We

have that: automaton N is equal to MSL(H) : We also have that m (L (H)) is
equal to SL(H) :
References
[1] A. Nerode. Linear Automaton Transformations. Proceedings of the AMS, 9,
JSTOR 2033204, 1958.
10
4 2DFA, Shepherdson
And then Michael Rabin and Dana Scott discovered the reverse...
Can we recognize palindromes using a read-only machine that can move
backwards and revisit cells that were previously visited?
A two-way deterministic …nite automaton (2DFA, for short) is an automaton
that can revisit characters already processed. As with DFA’s, there are a …nite
number of states with transitions between them based on the current character,
but each transition is also labelled with a value indicating whether the machine
will move its head to the left, right, or stay at the same position.
De…nition 19 A 2DFA is a tuple M = (Q; ; q0 ; T; A; ) that behaves analo-

gously to a …nite state automaton, except that:
A T Q: The set T is the set of halting states, which contains the

set of accepting states and a possibly empty set of rejecting states. If M
reaches a state in T the computation halts.
The transition function is a function from the set Q ( [ f g) into
the set Q f 1; 0; 1g : The symbol is used to indicate that the scanning
head is placed on an empty cell. The symbols in the set f 1; 0; 1g are used
to indicate wether the scanning head will move to the left, stay at the same
position or move to the right.
De…nition 20 Let M = (Q; ; q0 ; T; A; ) be a 2DFA, and let w 2 ; we say

that M accepts w; if and only if, the computation of M, on input w; reaches
an accepting state.
The language accepted by M, denoted with the symbol L (M) ; is equal to
the set of strings that are accepted by M.
Does there exist a 2DFA M that recognizes the language pal? 2DFAs were
introduced in a seminal 1959 paper by Rabin and Scott, who proved them to
have equivalent power to one-way DFAs [1]. That is, any formal language which
can be recognized by a 2DFA can be recognized by a deterministic …nite state
automaton. Since DFAs are obviously a special case of 2DFAs, this implies that
both kinds of machines recognize precisely the class of regular languages. The
proof of Rabin and Scott is long and involved. Shepherdson found, short time
after the communication of Rabin-Scott result, a simpler proof of this fact [2],
(actually, both papers were published in the same journal issue, see below). We
will discuss a proof of this theorem that is based on Shepherdson’s proof.
4.1 The Proof

Note that we can use Myhill-Nerode Theorem to prove a result like the one by
Rabin and Scott. To this end, we have to show, given a 2DFA M, that the
language accepted by M is a …nite-index language. Let us ask: what does it
mean that L (M) is of …nite index?
11
Suppose that Alice and Bob want to make some collaborative computation:
they want to run M, on the input xy; where x is given to Alice, and y is given
to Bob. Alice begins running M, on the input x: When the automaton leaves
x to the right, Alice communicates to Bob the state reached by M, and he uses
this information to continue with the computation: he runs M, on the input
y, but he starts this computation with the scanning head placed on the …rst
character of y; and he set the initial inner state to be equal to the state that
was communicated by Alice. When the scanning head reaches the left end of y;
Bob communicates to Alice the state reached by M, and then she takes charge
of the computation. The computation continues in this way until a halting state
is reached. We have to observe that, the number of messages communicated by
Alice and Bob depends on the length of x and y.
Now suppose that Alice has to go and quit the collaboration. She has to
communicate some bits to Bob, which allow him to take charge of the whole
computation. How many bits must Alice communicate to Bob? It su¢ ces
if Alice communicates O (jxj) bits: it su¢ ces if Alice communicates string x:
Can Alice do better? Suppose that M is a DFA. Then, it su¢ ces if Alice
communicates O (log (jQj)) bits, where Q is the set of states of M: Alice does
her job, if she communicates to Bob the state that is reached by M when the
scanning head reaches the left end of x.
Let C be a class of automata. Consider the following interactive computation.
The spoiler chooses M 2 C as well as two large strings x; y:

The spoiler gives Alice the pair (M; x) ; and he gives Bob the pair (M; y) :
The spoiler asks Alice and Bob to decide wether M accepts xy: They have
to communicate as few bits as possible.
We say that C is an easy class, if and only if, given M 2 C there exists a
constant CM such that it su¢ ces if Alice communicates CM bits to Bob, and
this number of bits does not depend on the pair x; y: We showed that the class
DFA is easy. We have
Theorem 21 C is easy, if and only if, the containment
fL : 9M (M 2 C ^ L = L (M))g REG
holds.
Proof. Suppose C is easy. Let M 2 C be an automaton with input alphabet

N
: There exists N and there exists a function fM : ! f0; 1g such that:
if Alice is given w 2 ; she communicates the string fM (w) to Bob:
We have that:
If fM (w) = fM (w) , then w L(M) u:
12
We get that L(M) 2N :
Now suppose that the containment
fL : 9M (M 2 C ^ L = L (M))g REG
holds. Let M 2 C. We have that L (M) is regular. Let
= f[w1 ] ; :::; [wN ]g ;

L(M)
and suppose that Alice and Bob has to solve the triple (M; x; y), that is: Alice
gets x; Bob gets y and they are asked to decide wether M will accept the input
xy:
They can solve the triple (M; x; y) by communicating a number of bits that
does not depend on (x; y) : They proceed as follows:
1. Alice computes the number i N for which the equivalence x L(M) wi
holds, and then she communicates to Bob the string wi .
2. Bob simulates the computation of M on the input wi y: Note that this
computation produces the same output than the computation of M on
input xy:
Let K = max fjwi j : i N g : We get that Alice must communicate some-
thing like K log (j j) bits, and this quantity does not depend on the pair (x; y) :
We conclude that C is easy.
We get that, according to the above theorem, proving Rabin-Scott result is
equivalent to prove that the class 2DFA is easy.
Theorem 22 2DFA is an easy class.
Proof. Let M = (Q; ; q0 ; T; A; ) be a 2DFA, and suppose that Alice and Bob
are asked to run the computation of M, on the input xy: We suppose that Alice
is given string x; while Bob is given string y:
Alice has to tell Bob what is the state reached by M the …rst time the
scanning head leaves x and goes to the right. Let qR be this state.
Let q 2 Q: Suppose that M is in the state q scanning the …rst character
of y: Suppose also that (q; y [1]) is equal to (a; b; 1) : Then, in the next
time instant the scanning head will enter the pre…x of the input string
that is hidden for Bob, and the automaton will stay some time processing
characters of this pre…x. If Bob wants to continue with the computation,
without knowing x; he will have to ask Alice about the result of this part
of the computation that is hidden for him. This result could be a halting
state, if the automaton reaches such a state without leaving the pre…x x:
This result could also be a nonterminating state, if the automaton reaches
the right end of x and then moves one further position to the right. Note
that in both cases the result of the computation is just a state M. Let us
use the symbol q b to denote this state.
13
Note that it su¢ ces if Alice communicates to Bob the state qR as well as
the function fM;x : Q ! Q that is de…ned by the equation fM;x (q) = q b :
Then, the total number of bits that must be communicated is upperbounded by
log (jQj) + jQj log (jQj) and this quantity does not depend on the pair (x; y) ; it
only depends on the size of M. The theorem is proved.
Exercise 23 Homework 4. Let k 1; and let

n o
k 1
Lk = w 2 f0; 1g : 9u 2 f0; 1g 9v 2 f0; 1g (w = u1v) :
How many states are necessary to recognize Lk using a DFA? How many states
are necessary to recognize the same language using a 2DFA?
References
[1] M. Rabin, D. Scott. Finite automata and their decision problems. IBM Jour-
nal of Research and Development 3 (2): 114–125, 1959.
[2] J. Shepherdson. The reduction of two-way automata to one-way automata.
IBM Journal of Research and Development 3(2): 198 - 200, 1959.
14
5 Minimal Writing Power: Inkdot Automata
There was a time when automata used spots to solve problems...
We think that it was a big surprise for the reader to know that the ability of
moving to the left, and check previously visited cells, does not provide computa-
tional power beyond the power of deterministic …nite state automata (1-DFA’s
for short). However, we have to observe that the scanning head cannot recognize
a previously visited cell, unless this cell is located at a short distance from one
of the ends of the input string, or, unless this cells got marked someway in the
previous visits. Note that checking previously visited cells can be useless if one
cannot recognize those cells. Let us try with marking mechanisms that allow
the automaton to mark cells that were previously visited.
Remark 24 We have that two-wayness without marking is useless. We also

have that marking without two-wayness is useless: a one-way automaton is a
read-once automaton.
Marking is the weakest form of writing. In the next lectures we will inves-
tigate the computational power of di¤erent forms of writing. We consider that
the weakest form of marking power corresponds to provide the automaton with
a bounded number of ink drops, which can be placed on a cell by the scanning
head, which can be sensed by this head, and which cannot be removed.
We study in this lecture the computational power of automata that are
provided with a constant number of ink drops, say k ink drops, and which can
do at most k marking actions along their computations.
De…nition 25 A Ink-k-2DFA is a tuple M = (Q; ; q0 ; T; A; ) that behaves

analogously to a 2DFA, except that it is provided with k ink drops.
Let M be a Ink-k-2DFA. The transition function is a function from the

set
Q f0; :::; kg ( [ f g) f0; 1g
into the set
Q f 1; 0; 1g f0; 1g f0; :::; kg
Let w an input of M, and let us suppose that M is processing input w: Let
t be a time instant along this computation. The automaton sees, at instant t,
its own inner state, how many ink drops has employed so far, a character from
[ f g, and it possibly sees an ink drop on the cell its scanning head is located
in (1 stands for the presence of an ink dot). Given these data the automaton
updates its inner state, decides the direction of its next move, decides wether
it leaves one ink drop on the cell it is located on (provided that there are ink
drops that are still available) and updates its drops-counter.
Let us consider the case k = 1: We have:
Theorem 26 The class Ink-1-2DFA is an easy class.
15
Proof. The proof is similar to the proof of Theorem 22.
Let M = (Q; ; q0 ; T; A; ) be a Ink-1-2DFA, and suppose that Alice is given
string x; while Bob is given string y:
Let i n = jxj ; and let x(i) be the string that is obtained from x after
dropping the inkdot at position i: Suppose that M is processing input xy; and
suppose that M drops the inkdot at position i: We can think of the remainder
of the computation of M, on the input xy; as the computation of a 2DFA
on the string x(i) y: We know that Bob can take charge of this computation
after receiving the appropriate information from Alice. What information? The
string x(i) y determines the function fi : Q ! Q that is de…ned by:
Suppose that the scanning head enters x from the right,

suppose that its inner state is equal to q;
and suppose that the inkdot is located at position i:
If the automaton reaches the halting state p before leaving x,
then fi sends q in p:
On the other hand, if the automaton leaves x to the right after reaching the
non-halting state r; then the function fi sends q in r:
Function fi is the navigation chart that must be known by Bob, and this naviga-
tion chart can be described using O (jQj log (jQj)) bits. The symbol f0 represents
the function that is determined by x:
We also have to consider the segment of the computation that occurs before
the dropping of the inkdot, and which determines the place at which the inkdot
is dropped. Each time the scanning head enters the pre…x x while carrying the
ink drop, it could drops the ink before leaving this pre…x. The occurrence or
not occurrence of this dropping is determined by the state at which the machine
enters the pre…x x; and this state also determines the position at which the ink
is dropped. Bob must be informed wether the dropping occurs, and he must
also be informed about the position at which the ink is dropped. The important
information about this position i is the function fi that it determines.
If we put all the above pieces together we get that: it su¢ ces if Alice com-
putes the function
M;x :Q f0; 1g ! Q f0; 1g QQ
that is de…ned by:
Let (q; 0) 2 Q f0; 1g : The pair (q; 0) stands that the scanning head
enters the pre…x x from the right, without the ink drop. The equality
M;x (q; 0) = (f0 (q) ; 0; f0 )
tells us that: if the ink dot is not in x; then f0 (q) is the outcome of this
segment of the computation that is hidden to Bob.
16
Let (q; 1) 2 Q f0; 1g : The pair (q; 1) stands that the scanning head enters
the pre…x x from the right with the ink drop. Suppose that the equality
M;x (q; 1) = (f0 (q) ; 1; f0 )
holds. The triple (f0 (q) ; 1; f0 ) tells us that the scanning head will leave
x carrying the ink drop, and that f0 (q) will be the inner state of M at
this precise instant.
Let (q; 1) 2 Q f0; 1g : Suppose that the equality
M;x (q; 1) = (q ; 0; f )
holds. The triple (q ; 0; f ) tells us that the scanning head will leave x
without the ink drop, that q will be the inner state of M at this precise
instant, and that the ink drop will placed at a position of x; say position
i; such that f = fi . The function f is the function that must be used by
Bob in the future: if some time in the future, the scanning head would
enter x from the right while being in the state q; then the outcome of this
segment of the computation, that is hidden to Bob, will be equal to f (q) :
We claim that function M;x can be used by Bob to solve the following task:
Bob is given a string y; and he is asked to simulate the computation of M
on input xy:
2
We get that it su¢ ces if Alice communicates O jQj log (jQj) bits to Bob,
since this number of bits is enough to describe the function M;x : The theorem
is proved.
Exercise 27 Prove that Ink-k-2DFA is easy.
Exercise 28 Homework 5. Prove that one can use an unbounded number of

inkdots to recognize pal:
We get, from the latter exercise, that the class Ink-1-2DFA is strictly more
powerful than the class DFA. Note that this additional computational power
comes from the ability of those automata to make an unbounded number of
marking actions along its computations. It is questionable wether an unbounded
tank …lled with in…nitely many ink drops is a realistic resource. On the other
hand, the ability of making an unbounded number of marking actions can be
achieved using a bounded number of removable and reusable marks that we
call pebbles. We consider, in the next few lectures, an elementary architecture
for automata that gives place to devices that can make an unbounded number
of marking actions using a bounded number of pebbles. We claim that those
automata are able of nontrivial writing, and we investigate their computational
power.
17
References
[1] A. Szepietowski. Turing Machines with Sublogarithmic Space. Springer-

Verlag, 1994.
18
6 One-Pebble Automata
And then, Manuel Blum invented pebbles...
We study in this lecture the computational power of automata that are
provided with a single pebble, which can be placed on the tape, which can be
sensed, and which can be removed and placed again as many times as necessary.
De…nition 29 A Pebble-1-2DFA is a tuple M = (Q; ; q0 ; T; A; ) that behaves

analogously to a 2DFA, except that it is provided with 1 pebble.
Let M be a Pebble-1-2DFA. The transition function is a function from the

set
Q f0; 1g ( [ f g) f0; 1g
into the set
Q f 1; 0; 1g f0; 1; 2g f0; 1g
Let w an input of M, and let us suppose that M is processing input w: Let
t be a time instant along this computation. The automaton sees, at instant t,
its own inner state, wether it has used the pebble, a character from [ f g,
and it possibly sees the pebble on the cell its scanning head is located in. Given
these data the automaton updates its inner state, decides the direction of its
next move, decides wether it leaves the pebble on the actual cell (provided that
it has the pebble), or wether it picks up the pebble (provided the pebble is on
the actual cell) or wether it neither leaves nor picks up the pebble.
Note that Pebble-1-2DFA’s can make an unbounded number of marking
actions. We ask: can Pebble-1-2DFA’s recognize the language pal? We have:
Theorem 30 The class Pebble-1-2DFA is easy.
Proof. The proof is similar to the proof of Theorem 26.

Let M = (Q; ; q0 ; T; A; ) be a Pebble-1-2DFA, suppose that Alice is given
string x; and suppose that Bob is given string y:
Let i n = jxj ; and let xp(i) be the string that is obtained from x after
placing the pebble at position i: The string xp(i) y determines the function gi :
Q ! Q f0; 1g that is de…ned by:
Suppose that the scanning head enters x from the right,

suppose that its inner state is equal to q;
and suppose that the pebble is located at position i:
If the automaton reaches the halting state p before leaving x,
then gi sends q in (p; 0) :
On the other hand, if the automaton leaves x to the right after reaching the
non-halting state r; then the function gi sends q in (r; ") : The value " = 1
indicates that the scanning head leaves x carrying the pebble.
The symbol g0 represents the function that is determined by x:
19
It su¢ ces if Alice computes the function
Q
M;x :Q f0; 1g ! Q f0; 1g (Q f0; 1g)
Let (q; 0) 2 Q f0; 1g : The pair (q; 0) represents that the scanning head
enters the pre…x x from the right, without the pebble. The equality
M;x (q; 0) = (g0 (q) ; 0; g0 )

tells us that the automaton will leave x without the pebble. On the other
hand, the equality
M;x (q; 0) = (g0 (q) ; 0; g0 )
tells us that the automaton will leave x with the pebble.

Let (q; 1) 2 Q f0; 1g : The pair (q; 1) stands that the scanning head
enters the pre…x x from the right, while carrying the pebble. The equality
M;x (q; 1) = (g0 (q) ; 1; g0 )

tells us the scanning head will leave x with the pebble.
Let (q; 1) 2 Q f0; 1g : Suppose that the equality
M;x (q; 1) = (q ; 0; g)
holds. The triple (q ; 0; g) tells us that the scanning head will leave x
without the pebble, that q will be the inner state of M at this precise
instant, and that the pebble will be placed at a position of x; say position
i; such that g = gi .
We claim that function M;x can be used by Bob to solve his task. We get
2
that it su¢ ces if Alice communicates O jQj log (jQj) bits to Bob, since this
number of bits is enough to describe the function M;x : The theorem is proved.
Remark 31 Blum and Hewitt proved that the languages recognized by Pebble-
1-2DFA’s are regular languages. The proof uses the following result: Let N be a
one-tape Turing machine that runs in linear time, the language recognized by N
is a regular language. We think that it was worth to present a proof of this result
that follows from basic principles and the very de…nition of Pebble-1-2DFAs.
Exercise 32 Homework 6. Prove that one cannot recognize the language pal
using a 2DFA that is equipped with a pebble and k ink drops.
References
[1] M. Blum, Ch. Hewitt. Automata on a 2-Dimensional Tape. Proceedings of
SWAT (FOCS) 1967: 155-160.
20
7 Two, Three,... A Bounded Number of Pebbles
At the beginning there was only one pebble, but then a second pebble came to
light...
Why are Pebble-1-2DFAs unable to recognize the language pal? Suppose
we get a string w; and suppose we are asked to decide wether this string is
a palindrome. Processing w forces us to work on both sides of w; scanning
characters on the left and then on the right, and then once again on the left, on
then on the right and so on. Thus, it seems that we need at least two pebbles
to recognize palindromes. Do two pebbles su¢ ce?
7.1 Two Pebbles

It is easy to construct a Pebble-2-2DFA that recognizes the language pal: Two
pebbles su¢ ce to implement a zig-zag strategy:
One pebble is placed on the left end of the input string, and the other pebble
is placed on the right end. Those two pebbles are used to mark the two cells
that must be compared. The left pebble is moved one position to the right
after each checking, while the left pebble is moved one position to the left. The
automaton halts either if it …nds a missmatch, or if the two pebbles meets each
other in the middle of the input string.
Exercise 33 Homework 7. Let M be a Pebble-2-2DFA. The running time

of M is the function tM : N ! N that is de…ned by
n
tM (n) = max ftM (w) : w 2 g;
where tM (w) denotes the number of transitions that constitute the computation
of M, on the input w: Prove that the running time of the above automaton is
equal to n2 : Is it possible to recognize pal using a Pebble-2-2DFA that runs
in subquadratic time? Hint: investigate about Hennie’s Theorem.
Exercise 34 Let SQU ARES be the language
w 2 f0; 1g : 9v 2 f0; 1g (w = vv) :
Prove that this language can be recognized by a Pebble-2-2DFA. It is important

to remark that the language SQU ARES is an example of a language that is
context-sensitive but which is not context-free. Investigate about context-free
languages.
7.2 Three and More Pebbles

Two pebbles are strictly more powerful than one pebble. Let us ask:
1. Which are the languages that can be recognized by Pebble-2-2DFAs?

2. Does any additional pebble increase the computational power of pebble
automata?
21
3. Which are the languages recognized by Pebble-k-2DFAs?
Pebble automata can be used to recognize complex languages:
Exercise 35 Prove that three pebbles su¢ ce to recognize the language
n n o
a2 : n 1 :
And it seems that the addition of pebbles allows the recognition of more
complex languages:
Exercise 36 Prove that one can use three pebbles to recognize the language
fap : p is a prime numberg :
Exercise 37 Prove that one can use four pebbles to recognize the language
n n
o
an #a2 : n 1 :
Conclude that one can use four pebbles to compute the function 2n :
Exercise 38 Homework 8. Prove that one can use …ve pebbles to recognize
the language n o
n
an #ak #ak : n 1 ;
conclude that …ve pebbles su¢ ce to make arithmetic.
Thus, it seems that the addition of pebbles provides computing power. Let
k 2; does there exist a language which can be recognized using k + 1 pebbles,
but which cannot be recognized using k pebbles?
7.3 A Hierarchy of Pebble Automata

Let REG k be the set of languages that can be recognized by Pebble-k-2DFAs.
We have that
REG = REG 0 = REG 1 ;
we also have that
Theorem 39 For all k 1 the containment REG k REG k+1 is strict.
Proof. Let k 1; we have to construct a language Lk such that Lk 2 REG k+1
and Lk 2= REG k : The construction is recursive. We can choose L1 to be any
nonregular language in REG 2 : Let us set L1 = pal: Suppose the language Lk
was already constructed, let k be the alphabet of this language, and let # 2
= .
We set
Lk+1 = fw1 #w2 # #w2m : m > 0 & jfi 2m : wi 2 Lk gj = mg :
We claim that Lk+1 2 REG k+2 and Lk+1 2 = REG k+1 :
It is easy to …gure out a Pebble-k + 2-2DFA automaton Mk+2 that works,
on input w1 #w2 # #w2m ; as follows:
22
Let i 2m, automaton Mk+2 uses k + 1 pebbles to check wether wi 2 Lk :
Automaton Mk+2 uses the remaining pebble to keep track of the number
of i’s such that wi 2 Lk :
Hsia and Ye proved that Lk+1 requires k + 2 pebbles [1]. Let Mk+2 be a
Pebble-k + 2-2DFA automaton recognizing Lk+1 and let m > 0: It is easy to
prove that there is a string w1 #w2 # #w2m ; there is i 2m; and there is an
instant of the computation of Mk+2 on this string such that Mk+2 places k + 1
pebbles on wi : This implies that Mk+2 has to use k + 1 pebbles to recognize
the i’s such that wi 2 Lk ; and it requires an additional pebble to keep track of
the number of those i’s.
It is important to observe that j k j = k + 1: This means that we are us-
ing increasing alphabets to construct the sequence fLk : k 1g : Moreover, the
recursive nature of this sequence makes it hard to visualize the languages in-
cluded in it. It would be great if we could construct a more uniform sequence
of languages separating the levels of the pebble hierarchy.
7.4 The Powers of pal and Programming Strategies using

Pebbles
Let k 1; we use the symbol pali to denote the language
w1 w2 wi 2 f0; 1g : w1 ; :::; wi 2 pal :
We have that pal1 is equal to pal; and we have that pal1 requires two pebbles
to be accepted by a pebble automaton. It seems that pal2 requires three pebbles.
It is natural to conjecture that the language palk requires k+1 pebbles. However,
we have to observe that
Lemma 40 P al4 can be recognized with four pebbles.
Proof. Let w 2 f0; 1g : Let M4 be the pebble automaton that uses the
following divide and conquer strategy:
M4 uses one pebble to divide the input w into two factors, a pre…x
w [1; :::; i] and a su¢ x w [i + 1; :::; jwj] :.
M4 uses the remaining three pebbles to check wether w [1; :::; i] belongs
to pal2 . If this …rst check is positive M4 moves those three pebbles to the
right and checks wether w [1; :::; i] belongs to pal2 : If this latter check is
also positive M4 halts and accepts the input, otherwise it moves the …rst
pebble one position to the right and repeats the above work with the pair
w [1; :::; i + 1] and w [i + 2; :::; jxj].
If the …rst pebble reaches the right end of the input string the automaton
halts and rejects the input.
23
We note that we can use a similar strategy to prove that palk 2 REG log(k)+2 :
We conjecture that palk 2
= REG log(k)+1 :
Exercise 41 Investigate the above conjecture.
References
[1] P. Hsia, R. Ye. Marker Automata. Information Sciences 8: 11-88, 1975.
24
8 Simulating Automata by Automata and the
Emergence of Programmable Machines
Pebbles are computational resources that can be smartly used to implement
non-naive programming strategies. Pebbles can also be used for programming.
We say that an automaton is a programmable machine, if and only if, it can
be used to execute an in…nite number of programs. We show that one can use
a …nite number of pebbles to search the transition tables of in…nitely many
automata, as well as to execute the programs consigned in those tables.
8.1 Executing a Set of Automata

An automaton is a program (algorithm). We use to think of programs as strings
that are executed by machines, and it seems that it is not case with automata.
However, we have to note that:
1. Automata can be suitably described by strings.

2. Let M be an automaton. It is okay if we say that machine U executes M,
provided that U can be driven to solve the task that is solved by M.
De…nition 42 Let C be a set of automata, and let U be an automaton. Suppose

that the input alphabet of U is equal to [ f#g : We say that U can simulate
the automata in C, if and only if, the following condition is satis…ed:
There exists an injective function c : C ! ; and there exist functions
ci : f1; :::; ig ! : i 1 such that given M 2 C, an automaton with input
alphabet f1; :::; ig ; and given w 2 f1; :::; ig ; it follows that
M accepts w; if and only if, U accepts c (M) #ci (w) :
Suppose that U simulates the automata in C. Let M 2 C and let w 2

f1; :::; ig be an input string of M. The computation of M, on input w; is a
sequence of machine con…gurations that begins with the initial con…guration
determined by the pair (M; w) ; and …nishes with a halting con…guration of
M. On the other hand the computation of U, on input c (M) #ci (w) ; is also
a sequence of con…gurations, this time of the machine U. Let c1 ; :::; cn be the
former sequence, and let d1 ; :::; dm be the latter. It should happens that:
1. The inequality m > n holds.

2. There exists an injection, say f; of the set f1; :::; ng into the set f1; :::; mg ;
such that for all i n the con…guration df (i) of automaton U represents,
in a very speci…c and precise sense, the con…guration ci of automaton M.
Let us illustrate the above informal de…nition with the construction of a

Pebble-3-2DFA that we denote with the symbol U0 and which simulates the set
DFA.
25
To begin with the construction we have to de…ne a encoding of DFAs as
strings over a suitable alphabet. Let 0 be equal to f0; 1; #g : Let M =
(Q; ; q0 ; T; A; ) : We can suppose that
Q = f1; :::; ng ; q0 = 1; T = fm; m + 1; :::; ng
and
A = fm + k; m + k + 1; :::; ng :
We also suppose that = f1; :::; lg : We associate to M a string wM 2 0 that
provides a full description of automaton M. Given i 2 N we use the symbol
bi to denote the binary code of i: String wM is a suitable concatenation of the
following strings:
1. The string
wstate = b
1#b
2# #b
n
2. The string
walphabet = b
1#b
2# #b
l:
3. The string
\
waccepting = m + k#m \
+ k + 1# #b
n:
4. The string w that describes the transition function . Note that can
be described by its table, which is …nite list of triples of the form (p; a; q) ;
where p; q 2 f1; :::; ng and a 2 f1; :::; lg : The triple (p; a; q) can be de-
scribed by the string pb#ba#bq : If is equal to
(p1 ; a1 ; q1 ) ; :::; (ps ; as ; qs ) ;
we set
w = pb1 #ab1 #qb1 ##pb2 #ab2 #qb2 ## ##pbs #abs #qbs :
We set
wM = wstate ###walphabet ###waccepting ###w :
We observe that M can be fully reconstructed from the string wM :

k
Suppose the input alphabet of M is equal to f1; :::; lg : Let u 2 f1; :::; lg ;
we use the symbol [u] to denote the string
ud
[1]#ud
[2]# #ud
[k] 2 0:
Remark 43 Note that we have constructed a programming language: the func-

tion that assigns to each DFA (algorithm) M the string wM is a programming
language.
26
Now, we have to construct a computing device that can execute all the
aforementioned programs, that is: we have to construct a Pebble-3-2DFA, that
we will denote with the symbol U0 ; and which satis…es the following
M accepts u; if and only if, U0 accepts the input string wM #### [u] :
Let us present a high-level description of this automaton. Thus, suppose

that the input string is equal to
wstate ###walphabet ###waccepting ###w #### [u] :
The computation of U0 ; on this input, is divided in the following phases:
1. The automaton places its pebbles.
U0 places a pebble over the substring b 1#b

2# #b n: This pebble is
used to keep track of the inner state of M, and is placed at the end
of this …rst phase over the character b
1. We use the term state-pebble
to designate this pebble.
U0 places a pebble over the substring w : This pebble is used to search
the table of : This pebble is placed on the left end of this substring.
We use the term search-pebble to designate this pebble.
U0 places a pebble over the substring [u] : This pebble is used to
simulate the one-way movement of the scanning head. We use the
term scanning-pebble to designate this pebble.
2. The automaton uses its pebbles.

Let us suppose that
b (1; u [1; :::; i]) = k and (k; u [i + 1]) = r:
Let us also suppose that the state-pebble of U0 is placed over the substring
b
k; while the scanning-pebble is placed over the substring the encodes the
i-th character u: This pebble con…guration suitably represents the con-
…guration reached by M after i transitions. Thus, we are assuming that
the simulation has worked well until this point. We can suppose that the
inner state of U0 is indicating to the machine that it has just …nished the
simulation of the i-th transition of M, and that it is time to begin with the
simulation of the (i + 1)-step. Then, the automaton moves the scanning
pebble to the substring that encodes the (i + 1)-character of u: Suppose
that this character is equal to a: Note that U0 can use the search-pebble
and the scanning-pebble to recognize the substrings of w that encode
triples of the form ( ; a; ) : Note also that U0 can use the search-pebble
and the state-pebble to recognize the substrings of w that encode triples
of the form (q; ; ) : Then, U0 can use those three pebbles to locate the
substring of w that encodes the unique triple of the form (q; a; ) :
27
Suppose that the substring qb#ba#b p has been located. Note that U0 can
use the search-pebble and the state-pebble to place the latter over the
substring pb. Notice that, by doing so, U0 is simulating a further step of
the computation of M on the input u:
Remark 44 The language pal has been intensively studied along the historical
development of automata theory. We claim that there are good reasons for this
prominence of pal; the most important of which is the following one:
The ability of recognizing palindromes and squares is closely related to the
ability of searching tables and executing the programs that are consigned in those
tables.
Note that U0 can search for the right triple, and then for the right state, only
because this automaton can use pairs of pebbles to identify pairs of substrings
that are identical. U0 uses those pebbles to identify the right substrings using
a zig-zag strategy that is analogous to the siz-zag strategies that are used to
recognize palindromes and squares: recognizing palindromes is a prerequisite for
the emergence of programmability.
Exercise 45 Homework 9. Let k 2; show that there exists a Pebble-k + 2-

2DFA, say Uk ; which can simulate (execute) all the automata in the class Pebble-
k-2DFA.
References
[1] T. Koetsier. On the prehistory of programmable machines: musical au-
tomata, looms, calculators. Mechanism and Machine Theory, Elsevier, 36
(5): 589–603, 2001.
28
9 Unbounded pebbles: Counter Automata
The constructors of automata were accumulating pebbles for some time until
they could proceed with the construction of a supermachine, an automaton with
an unbounded number of pebbles...
What can be done with an unbounded number of pebbles? There are many
things that can be done, and we will employ some time discovering those pos-
sibilities. The …rst thing that comes to mind is to stack pebbles in a pile, and
use this pile to count items beyond any possible bound. Note that this was
the …rst computational application of pebbles in the history of computing: the
construction of counters.
De…nition 46 A one-counter automaton is a …nite automaton with an addi-

tional memory cell that can hold one nonnegative integer number (of unlimited
size), which can be incremented, decremented, and tested for being zero.
One-counter automata recognize a superset of the set of regular languages.

We denote the former set of languages with the symbol COU N T .
Exercise 47 Give a formal de…nition of one-counter automata, as well as a

formal de…nition of a language L being accepted by one-counter automata.
Exercise 48 Homework 10. Prove that one-counter automata can recognize

the language
f0n 1n : n 0g ;
prove that this language is not regular.
We can use pebbles to simulate counters. To this end we can stack pebbles
on the …rst empty cell, and use this stack as a counter, or we can horizontally
stack those pebbles, using the empty cells of the tape, placing at most one pebble
on each one of those cells. We have to observe that COU N T is a (relatively)
small set that does not contain the language pal: Then, we ask: do we need an
unbounded stock of pebbles to simulate a counter? We have
Theorem 49 COUN T is contained in REG 2 .
Proof. It is easy to prove that one can simulate a one-counter automaton

using two pebbles. One can use one pebble to keep track of the position of the
scanning head, and the second pebble to simulate the counter: the height of the
counter gets represented by the number of cells to the left of this second pebble.
Exercise 50 Prove that pal 2

= COUN T . Conclude that COU N T is strictly
contained in REG 2 .
What can be done if we combine pebbles and counters?
29
Exercise 51 De…ne COUN T k as the class of problems that can be recognized
using k pebbles and one counter. Prove that
REG k+1 COUN T k REG k+2 :
Which of those inclusions are strict?
9.1 An Universal Pebble Automaton U1

We can use two pebbles to simulate a counter. An unbounded number of pebbles
should allow us to achieve very much higher goals. Let us ask: what can be
done using an unbounded number of pebbles.
Note that we can construct a Pebble-1-2DFA that can be used to simulate
any automaton equipped with a …nite number of pebbles. We use the symbol
U1 to denote this automaton.
Exercise 52 Prove that U1 can execute any one-counter automaton equipped

with a …nite number of pebbles.
Note that we can use U1 to simulate all the automata considered so far: U1
can simulate 1DFAs, 2DFAs, one-counter automata, pebble automata. Then, it
can be argued that U1 is an universal automaton. Is U1 universal? An a¢ r-
mative answer would imply that any problem that can be solved by automata
and computing machines can be solved by pebble automata.
References
[1] L. Valiant, M. Paterson. Deterministic one-counter automata. Automaten
Theorie und Formale Sprachen: 104-115, 1973.
30
10 Two Counters, Colored Pebbles and Deter-
ministic Pushdown Automata
Once upon the time in Neverland someone dreamed with colored pebbles...
10.1 The PDA Architecture

Pushdown automata (PDA, for short) are automata that are provided with
writing abilities. However, there is an important restriction: all the writing
has to occur in a very special and bizarre notebook that is called the pushdown
stack.
A pushdown stack is an external memory unit that can be used to write,
or, it is better to say, that can be used to stack characters on a pile. Observe
that counters are external memory units that are used to stack indistinguish-
able pebbles. It happens that the pebbles that are stacked on a pile can be
read only once: if one checks the height of a counter, he has to pop all the
symbols(pebbles) stacked on that counter. Pushdown stacks are like counters,
except that one can stack colored pebbles, and use those colors to write non-
unary ‡ammable strings: pushdown stacks are read-once devices, and if one
checks the content of a pushdown stack, he burns all the scanned symbols. We
think that this feature make pushdown stacks a bizarre architecture. However,
it is a bizarre architecture that has played a very important role in the devel-
opment of computing.
The above …gure illustrates the architecture of a PDA:
The automaton is provided with a read-only input tape that can be

scanned from left to right.
The automaton is provided with a second tape, the pushdown stack. This
second tape counts with its own scanning head that we call the stack-
head. This head has, for the …rst time, the ability of writing on its tape.
However, the stack-head has to ful…ll a very strong constraint: it has to be
placed on the last non-empty cell. This constraint means that the stack-
head is not provided with ink but with colored pebbles, and this constraint
also means that the stack-head is not provided with a tape, but with a
single cell that it can use to stack its colored pebbles.
Why are pushdown automata so important in the history of computer sci-

ence? We would like to discuss this question in some depth, but before of this
let us introduce the formal de…nition of pushdown automata.
De…nition 53 A pushdown automaton is a tuple M = (Q; q0 ; ; ; A; ) ; where:
31
1. Q is a …nite set of states.
2. q0 2 Q is the initial state
3. is a …nite set, the input alphabet of M.
4. is a …nite set, the stack-alphabet of M.
5. A Q is the set of accepting states.
6. The transition function is a function from the set Q into the set
P ow (Q fp; w; ng ) : The equality
(q; a; x) = f(q1 ; "1 ; x1 ) ; :::; (qn ; "n ; xn )g
means that: if the inner state is equal to q; the input-head is scanning the
character a; and the stack-head is scanning the character x; then automa-
ton M nondeterministically chooses one the triples in the set
f(q1 ; "1 ; x1 ) ; :::; (qn ; "n ; xn )g ;
and executes the order encoded by this triple. Suppose that M chooses
(qi ; "i ; xi ) : Then, M switches from q to qi ; advances its input-head one
step to the right, and makes some work in the pushdown stack. The work
that is done in the pushdown stack is determined by the pair ("i ; xi ) : If
"i is equal to p; the stack-head pops the character that it is reading at this
moment. If "i is equal to w; the stack-head push the character xi on the
top of the stack. If "i is equal to n; the stack-head does nothing.
Let M be a pushdown automaton, and let w be an input of M. Automaton

M reads w from left to right, and halts when it …nds the …rst …rst empty cell
on its input tape. Meanwhile, it updates its inner state taking into account the
data that are being read. The computation of M; on input w; is accepting, if
and only if, the …nal state is an accepting state.
We have to observe, at this point, that the nondeterministic character of
has the following e¤ect: the input w does not determine a unique computation
of M, it (possibly) determines exponentially many computations, which could
have di¤erent outcomes. When do we say that M accepts the input w?
De…nition 54 The nondeterministic acceptance criterion estates that M ac-

cepts the input w; if and only if, there exists an accepting computation of M on
this input.
Exercise 55 prove that pal can be accepted by a pushdown automaton.
Exercise 56 Homework 11. Let palstar be the language
w 2 f0; 1g : 9i < jwj 9w1 9wi (jw1 j ; :::; jwi j > 1 & w1 ; :::; wi 2 pal) :
Prove that palstar can be accepted by a pushdown automaton.
32
10.2 Pushdown Acceptors and Context-Free Languages
Pushdown automata were not created by an engineer, those automata were
…rst designed by a linguistic, Noam Chomsky, who was looking for a reason-
able mathematical model of natural languages such as English, French, etc.
Pushdown acceptors correspond to the machine-characterization of context-free
languages, which are the languages generated by context-free grammars. The
concept of pushdown acceptors was in the air since 1954, and it was …rst formal-
ized by Chomsky [1] and Evey [2]. In the late 1960, it was discovered that the
"ALGOL-like" languages, were identical with the context-free languages. This
fact led to a ‡urry of activity in the theoretical development of context-free
languages. Much of this work was done by those concerned either with natural
languages in connection with computers, or with programming languages. The
remainder was done by mathematicians and logicians intrigued by the inherent
problems, techniques, and results.
Pushdown stacks were not introduced because of the likely advantages of
their unlikely architectures. Pushdown stacks were introduced because this is
the architecture that captures the computation of strings using context-free
grammars. Those bizarre automata determine a set of computable languages,
the set of context-free languages, that we can use to test the universality of U1 :
10.3 Context-Free Languages that Cannot be Accepted

With a Finite Number of Pebbles
Read only automata with a bounded number of pebbles (also called marker
automata, or marking automata) were introduced as an alternative to pushdown
automata [3]. Let us quote the abstract of a related reference (see [4]).
An o¤-line, memory-restricted Turing machine model, the mark-

ing automaton (MA) is presented here as a device strictly interme-
diate between …nite and linear bounded automata. Although much
more restricted than the latter, MA are shown capable of recogniz-
ing, deterministically, various kinds of context-free (CF) languages
and an important related class, as well as such non-CF languages
as {xx}. It is not known whether all CF languages are recognizable
by MA; however, among the familiar subclasses shown to consist of
MA recognizable languages are the Dyck, standard, and bounded
CF languages. More importantly, each member of the class of struc-
tured CF languages, consisting of all structural descriptions (Phrase-
markers) of the sentences in a CF language, is shown to be MA
recognizable. The closure of the MA recognizable languages under
various set (e.g., boolean) operations is revealed in the proof that
all bounded CF languages are MA recognizable.
Let us focus on the following sentence: It is not known whether all CF

languages are recognizable by MA. What do we know nowadays? We know that:
33
Pebble-k-2DFA are a very restricted type of logspace Turing machines, and it
is very unlikely that any context-free language can be recognized using logspace.
However, it is still an open problem whether context-free languages can be
decided in logarithmic space. The best known result shows that they can be
decided in deterministic space O log2 (n) . It is also known that: if all context-
free languages can be decided in deterministic logarithmic space, then L = N L
[5]. The equality between logarithmic space and nondeterministic logarithmic
space is considered to be very unlikely. We have that it is very unlikely that
all context-free languages can be recognized using a …nite number of pebbles.
This suggests that there are computable problems beyond the power of pebble
automata, which could be tackled by automata provided with an unbounded
number of pebbles. We would like to exhibit a context-free language that cannot
be recognized by pebble automata. Recall a previous conjecture: there does not
exist k such that all the powers of pal belong to REG k : We have:
Theorem 57 Suppose that there does not exist k such that for all i 1 the
language pal(i) belongs to REG k : Let pal be equal to the language
n o
w 2 f0; 1; 2g : there exists i 1 such that w = 2i u and u 2 pal(i) :
[
We have that pal is a context-free language that does not belong to REG k :
k 1
Exercise 58 Homework 12. Prove the above theorem.
The above theorem indicates that our previous conjecture deserves some
consideration. We will discuss some additional facts related to this conjecture
in a future lecture.
10.4 Simulating PDAs

Can we simulate the PDA architecture using a …nite number of pebbles? If our
previous conjectures are right, and L is di¤erent of N L; we cannot simulate all
the pushdown automata using a …nite number of pebbles. Can we simulate the
PDA architecture using an in…nite number of pebbles?
Let M be a pushdown automaton whose stack alphabet is equal to f1; :::; kg :
Suppose that we are provided with in…nitely many colored pebbles, in…nitely
many of each color in the set f1; :::; kg : It is straightforward to simulate the stack
of M stacking those pebbles: pushdown stacks are stacks made with colored
pebbles. However, it is debatable if colored pebbles is a feasible resource. Can
be simulate the PDA architecture using monochromatic pebbles? Yes, we can.
We can use that pushdown automata can be simulated by two-counter automata,
and then we can use that it is straightforward to simulate two-counter automata
using pebbles: we can use two empty cells, and two stacks of pebbles to simulate
two counters. We can also simulate a pushdown stack of height n using the …rst
n cells of a pebble automaton:
34
Let M be a pushdown automaton whose stack alphabet is equal to f1; :::; kg.
We can use i pebbles to simulate the character i; k + 1 pebbles to simulate the
stack-head, and 2k + 1 pebbles to simulate the input-head. Thus, suppose that
the stack content is equal to w1 wn ; and suppose that the input head is
located at position i > n: This con…guration of M is represented by a tape that
contains w1 pebbles placed on the …rst cell,...,wn + k + 1 pebbles placed on the
n-th cell, and 2k + 1 pebbles located on the i-th cell.
References
[1] N.Chomsky. Context-free grammars and pushdown storage. MIT Res. Lab.
Electron. Quart. Prog. Rept. 65, 1962.
[2] R. Evey. The theory and applications of pushdown store machines. Mathe-
matical Linguistics and Automatic Translation, Harvard Univ. Computation
Lab. Rept. NSF-IO, May, 1963.
[3] F. Springsteel. Context-free languages and marking automata. Doctoral dis-

sertation, University of Washington, New York, 1967.
[4] W. Ritchie, F. Springsteel. Language recognition by marking automata. In-
formation and Control 20(4): 313 - 330, 1972.
[5] I. Sudborough. On tape-bounded complexity classes and multi-head …nite

automata. Proc. 14th Annual IEEE Symp on Switching and Automata The-
ory, 1973, pp. 138-144.
35
11 Using the Input Tape to Write: Linear Bounded
AutomataXXXVan 12
Someone asked - why do we need a pushdown stack? -. He added - we can use
the cells of the input tape to construct small monochromatic stacks... And use
the concatenation of those stacks to simulate a pushdown stack...
11.1 The LBA Architecture

Our simulation of pushdown stacks yields very much more than expected: serepen-
dity, we invented deleble ink and discovered the power of writing.
Let M be a pushdown automaton whose stack alphabet is equal to f1; :::; kg.
Suppose that M is processing input w; and suppose that the height of the
pushdown stack is equal to n: We use k + 1 pebbles to simulate the stack-head
of M. We have to observe that nothing forces us to place those k + 1 pebbles on
the n-th cell. We can move those pebbles to cell 1 while maintaining the short
stacks placed on the …rst n cells, and which are simulating the content of the
stack, that is: our simulator can read any entry of the stack without deleting
the characters that are placed above this entry. Observe that we are writing
with pebbles: we can use any cell to write a character from f1; :::; kg ; we can
read those traces as many times as necessary, and we can delete those characters
and overwrite the corresponding cells as many times as necessary. We invented
deleble ink! and we discovered the LBA architecture.
De…nition 59 A linear bounded automaton is a tuple M = (Q; q0 ; ; ; H; A; ) ;

where:


4. is a …nite set, the work-alphabet of M, and we have that [f g :
5. A H Q, where H is the set of halting states and A is the set of
accepting states.
Q fw; ng f 1; 0; 1g : The equality
(q; a) = (p; "; y; )
means that: if the inner state is equal to q and the input-head is scanning
the character a; then the machine executes the order encoded by (p; "; y; ) ;
that is: M switches its inner state from q to p; it makes some work on the
cell its work-head is placed in, and then it moves its input-head according
to : The work that is done on the aforementioned cell is determined by
36
the pair ("; y) : if " is equal to n the machine does nothing, otherwise it
deletes the actual character and overwrites y: Moreover, the equality
(q; ) = (p; "; y; )
implies that = 1: Note that this equality implies that the machine
cannot use the empty cells as work space.
Let M be a linear bounded automaton, and let w be an input of M. Suppose

that jwj = n: Automaton M can use the …rst n cells of the tape, to write, to
overwrite and to make any work that can be encoded by its transition function.
The computation of M; on input w; is accepting, if and only if, it reaches an
accepting state after a …nite number of steps.
LBAs were introduced by John Myhill in 1960 [1]. In 1963, Peter S. Landwe-
ber proved that the languages accepted by deterministic LBAs are context-
sensitive languages [2]. In 1964, S. Kuroda introduced the more general model of
nondeterministic linear bounded automata, and noted that Landweber’s proof
also works for those automata. Then, Kuroda proved that the languages ac-
cepted by them are precisely the context-sensitive languages. In this seminal
paper, Kuroda also stated two research challenges, which subsequently became
famously known as the LBA problems (also called Kuroda problems). The …rst
Kuroda problem is whether the class of languages accepted by nondeterministic
LBAs is equal to the class of languages accepted by deterministic LBAs. The
second Kuroda problem is whether the class of languages accepted by nonde-
terministic LBA is closed under complement. We will be studying this type
of questions in the second part of this booklet, which pretends to be a crash
introduction to computational complexity theory.
Exercise 60 Homework 13. Let SAT be the problem:
Input : , where is a boolean formula.

Problem : decide if is satis…able.
Prove that this problem can be solved by a LBA. What is the running time
of the LBA solving problem SAT?
References
[1] J. Myhill. Linear Bounded Automata (WADD Technical Note). Wright Pat-
terson AFB, Wright Air Development Division, Ohio, 1960.
[2] P. Landweber. Three Theorems on Phrase Structure Grammars of Type 1.
Information and Control 6(2): 131–136, 1963.
[3] S. Kuroda. Classes of languages and linear-bounded automata. Information
and Control 7 (2): 207–223, 1964.
37
12 Turing Machines
Alan Turing was an heroic character. He observed that we could venture into
the obscure region that was made of empty cells...
12.1 The Turing Machine Architecture

A one-tape Turing machine is like a linear bounded automaton that can venture
into the twilight zone. These machines can enter the unbounded region of the
tape that is made of empty cells and write on those cells. Then, those machines
can use an unbounded amount of cells as work space. O¤ course, it is very much
better if those machines use as few cells as possible: the number of cells used as
work space is a computational resource that must be spared. We will discuss,
in future lectures, some topics related with this issue (the space complexity of
algorithms and problems).
A multihead Turing machine is a machine provided with k scanning heads,
which can explore and use all the cells of the input tape. Moreover, the …rst of
those heads can write on the input tape.
De…nition 61 A k-head one-tape Turing machine is a tuple
M = (Q; q0 ; ; ; H; A; ) ;
where:

4. is a …nite set, the work-alphabet of M, and we have that [f g :
5. A H Q, where H is the set of halting states and A is the set of
accepting states.
k
Q fw; ng f 1; 0; 1g : The equality
(q; a1 ; :::; ak ) = (p; "; y; 1 ; :::; k )
means that: if the inner state is equal to q and the scanning heads are
reading the tuple (a1 ; :::; ak ) ; then the machine executes the order encoded
by (p; "; y; 1 ; :::; k ) ; that is: M switches its inner state from q to p; makes
some work on the cell that is occupied by the head with label 1; and then
moves all its heads according to the tuple 1 ; :::; k : The work that is done
on the aforementioned cell is determined by the pair ("; y) : if " is equal
to n, then the machine does nothing on this cell, otherwise it deletes the
actual character and overwrites y:
38
Warning. The equality
(q; a1 ; :::; ai 1; ; ai+1 ; :::; ak ) = (p; "; y; 1 ; :::; k )
does not imply that i = 1:
We can simulate multitape Turing machines using the same ideas used in the
simulation of linear bounded automata automata. Thus, given a Turing machine
M there exists a pebble automaton PM , provided with in…nitely many pebbles,
and such that PM can simulate the computations of M.
Exercise 62 Formulate in a precise way the following assertion: there exists

a universal Turing machine that can simulate the computations of any other
Turing machine. Prove the corresponding theorem.
Let U be a universal Turing machine, we have that the pebble automaton

PU can be programed to simulate any Turing given machine given as target.
We have:
Question 63 The Church-Turing-Pebble Thesis

The pebble automaton PU can be programed to solve any problem that can be
solved by computers.
12.2 A Fictional Chronology of the Development of Com-

puters
Let us summarize the topics studied so far by presenting a (…ctional chronology)
of the historical development of automata, and the study of their capabilities.
1. In 1911 W. McCulloch, W. Pitts introduced the model of 1DFAs, as a

model of neural activity.
2. In 1913 M. Rabin and D. Scott invented the two-way head and introduced
the model of 2DFAs.
3. 1915, a …nite state automaton constructed by IBM left an oil-spot over its
tape. Inkdot automata were accidentally discovered. The elusive search
for an automaton recognizing pal remained un…nished.
4. 1916, M. Blum and Ch. Hewitt invented pebble automata. They con-
structed a two pebble automaton recognizing the language pal:
5. 1920, L Valiant and M. Paterson began to stack pebbles. Counter au-

tomata came to light.
6. 1924, N. Chomsky began the construction of automata for the analysis of
natural languages. The automata constructed by Chomsky used colored
pebbles, the pushdown stack was invented.
39
7. 1932, J. Myhill began to use pebbles to write arbitrarily long strings on
the input tape. The writing machine was invented.
8. 1937, A. Turing discovered the power of the twilight zone. He constructed
an universal Turing machine, proved that the halting problem is undecid-
able and formulated the Church-Turing-Pebble Thesis [1].
9. 1939-1945, the second world war, pebble supercomputers are constructed

for military purposes.
10. 1971, S. Cook proved that SAT is NP-complete [2]. Complexity theory
was born.
11. 1985, D. Deutsch replaced pebbles by spins [3]. The Quantum Turing
Machine was invented.
12.2.1 Fe de Erratas: A Correct Chronology

The history of ideas does not develop as one could expect. It occurs quite often
that some ideas that should precede some other ideas, came to light long time
after the emergence of their conceptual heirs. We feel that the above …ctional
chronology could correspond to the natural development of automata theory.
However, the actual chronology is very di¤erent.
1. 1937, A. Turing discovered the power of the twilight zone. He constructed

an universal Turing machine and proved that the halting problem for
Turing machines is undecidable.
2. 1939-1945, the second world war, analogical computers are constructed for
military purposes.
3. In 1943, W. McCulloch, W. Pitts introduced the model of 1DFAs as a

model of neural activity.
4. In 1959, M. Rabin and Dana Scott introduced the model of 2DFAs.
5. 1960, J. Myhill introduced the model of linear bounded automata.
6. 1962, N. Chomsky introduced the model of pushdown automata.

7. 1967, M. Blum and Ch. Hewitt introduced pebble automata.
8. 1971, S. Cook proved that SAT is NP-complete. Complexity theory was
born.
9. 1973, L Valiant and M. Paterson introduced one-counter automata.

10. 1985, D. Deutsch introduced The Quantum Turing Machine.
40
References
[1] A. Turing. On Computable Numbers, with an Application to the Entschei-
dungsproblem. Proceedings of the London Mathematical Society. 42(2): 230
- 265, 1937.
[2] S. Cook. The complexity of theorem proving procedures. Proceedings of the
Third Annual ACM Symposium on Theory of Computing. pp. 151–158, 1971.
[3] D. Deutsch. Quantum theory, the Church-Turing principle and the universal
quantum computer. Proceedings of the Royal Society A. 400 (1818): 97–117,
1985.
41
13 Why Do We Need Turing Machines and Other
Computer Architectures
We have pebbles, why do we need Turing machines?
Pebble automata constitute one of the most rudimentary and ine¢ cient ar-
chitectures, and this makes those automata become inappropriate for theoretical
analysis. On the other hand, The Turing Machine architecture is an ine¢ cient
architecture that is versatile enough to be appropriate for the theoretical analy-
sis of algorithmic problems. Let us focus on the problem of recognizing the
language pal: What are the perfomances that we can expect from those two
models of universal computation?
We can recognize the language pal in time 32 n + O (1) using a two-head
Turing machine M that works, on input w; as follows:
1. Head number 2 goes to the right end of w:

2. Head number 1 and head number 2 move simultaneously, the …rst one
from left to right, the second one from right to left. If a missmatch is
found the machine halts and rejects, otherwise it accepts. In this latter
case the computation …nishes when both heads meet each other.
On the other hand, we know that we can recognize pal using a pebble au-
tomaton that runs in time n2 and which implements the naive zig-zag strat-
egy discussed before. It is important to observe that this zig-zag automaton cor-
responds to the simulation of M using pebbles. Thus, we have that the solution
using pebbles is a parsimonious version of the solution using Turing machines.
Does there exists a better solution using pebbles? We prove that any solution
using pebbles runs in time n2 : The proof of this fact mimics the proof of
Hennie’s Theorem [1].
Remark 64 Hennie’s Theorem estates that any one-head Turing machine recog-
nizing pal runs in time n2 : This result of Hennie is one of the few known
superlinear lower bounds that hold for an universal model of computation, and
it is also one of the oldest lower bounds that can be found in the computing
literature. Thus, we are …nishing this part of the book with our pebble version
of one of the earliest results of complexity theory.
Theorem 65 Suppose that M is a pebble automaton that recognizes pal: We

have that the running time of M belongs to n2 :
Proof. The proof shows that a pebble automaton recognizing pal is an au-
tomaton that implements a strategy that is very close to the zig-zag strategy
discussed before. Thus, the proof shows that any pebble automaton recognizing
pal is an automaton that has to go so many times from left to right and from
right to left that it spends quadratic time processing its inputs.
n
Let M be a pebble automaton recognizing the language pal: Let w 2 f0; 1g ,
and let sw = w0n wR : Suppose that M is processing input sw : We de…ne Sw
42
as the ordered sequence of states that are reached by w each time the scanning
head of M goes from cell n + 1 to cell n after visiting the su¢ x wR :
Claim 66 Suppose that Sw = Su ; we have that M accepts the input w0n uR :
Let g : N ! N be the function de…ned by

n
g (n) = max fw 2 f0; 1g : jSw jg ;
n
where jSw j denotes the length of the sequence Sw : We have that f0; 1g can be
g(n) n
injected into jQj ; and we get that the inequality g (n) log(jQj) holds. Let
n
w 2 f0; 1g such that jSw j = g (n) : The running time of M, on input sw ; is
n2
bounded below by log(jQj) . We get that tM (n) 2 n2 :
Exercise 67 Prove The Claim 66.
References
[1] F. Hennie. Crossing sequences and o- ine Turing machine computations.
Proceedings of 6th Annual Symposium on Switching Circuit Theory and Log-
ical Design (SWCT 1965).
43
Part II
Nondeterminism: a Crash
Introduction to Complexity
Theory
44
14 The Power of Nondeterminism
Pushdown automata exhibit a characteristic that tells them apart from all the
other models of automata considered in this work. We are referring to the
following nontrivial fact: let M = (Q; q0 ; ; ; A; ) be a PDA, the transition
function of M is a function from Q into P ow (Q fp; w; ng ) : This
implies that an instant con…guration (q; a; x) does not determine a single action
but a …nite set of possible actions, say the set f(qi ; ri ; xi ) : i ng ; which are
all equally likely, and which do not lead to the same outcomes. Thus, the
nondeterministic character of those machines make them either unreliable or
unfeasible:
Suppose that w 2 L (M). The string w could determine exponentially many
computations of M. Suppose that this is the case, and suppose that the ac-
cepting computations constitute a small subset of the former set. If we run the
automaton only once we will see a rejecting computation with a high probabil-
ity, but the outcome of this computation is clearly wrong. Then, if we want to
obtain a de…nitive answer from M, we will have to simulate all the exponen-
tially many computations of M on w: The latter can be done in exponential
time using a Turing machine, and it can also be done in linear time using expo-
nentially many processors working in parallel. We need, in those both cases, a
exponential amount of computational resources.
It can be said that nondeterministic machines work on paper and cannot be
used in practice. Nondeterministic machines are machines that do not …t our
most naive conceptions about computation. We have to ask: how found this
bizarre concept its way into computing theory?
14.1 A Very Brief History of Nondeterministic Computa-

tion
The conception of nondeterministic computation is usually attributed to Michael
Rabin and Dana Scott. They de…ned nondeterministic …nite automata in their
famous 1959 paper Finite Automata and Their Decision Problems [1]. It is
important to observe that …nite automata were not introduced as models of
computer machines. Those automata were originally introduced as models of
reactive systems and neural activity. We think that nondeterministic transitions
are natural in those two scenarios, and this could explain why nondeterministic
…nite automata were introduced long time before nondeterministic Turing ma-
chines. However, we have to note that Rabin and Scott were thinking in …nite
automata as models of computer devices.
Exercise 68 Give a precise de…nition of nondeterministic …nite automata (NFAs,

for short).
Rabin and Scott proved that NFAs are as powerful as DFAs.
Exercise 69 Prove that NFAs constitute an easy set of automata.
45
Thus, we have to ask: how could this bizarre notion of nondeterministic
computation survive to the fact that it does not yield computational power
to the devices it was applied? Rabin and Scott proved that nondeterministic
transitions allow the construction of automata with fewer states.
Exercise 70 Let Lk be the language
w 2 f0; 1g : w [jwj k + 1] = 0 :
Prove that Lk can be recognized by a NFA with k states. Prove that a DFA
recognizing Lk has at least 2k states (use Myhill-Nerode Theorem).
The above fact is a very important one: we have to take into account that
memory-size (number of states) is the single computational resource of …nite
automata, and it is the only computational resource that we should spare. Thus,
Rabin and Scott showed that nondeterminism allows the construction of more
e¢ cient automata. They also showed that nondeterminism allows more natural
constructions, and they exempli…ed this fact by providing natural constructions
of NFAs for the concatenation of two regular languages and for the Kleene star
of a regular language.
Exercise 71 Construct the aforementioned NFAs.
However, it can(could) be argued that the aforementioned advantages of
nondeterminism were not enough to accept this bizarre concept. Let us ask:
does nondeterminism provide computational power?
14.2 Nondeterminism and Formal Grammars

N. Chomsky introduced the model of (nondeterministic) pushdown automata
short time after the publication of Rabin-Scott paper [2]. Those automata were
introduced as the machine characterization of context-free grammars, and non-
determinism naturally occur in those grammars, where several production rules
can be applied to the same nonterminal symbol (see below). Thus, it seems that
the nondeterministic character of those automata naturally arise from their con-
nection with the mathematical modeling of natural languages. However, if we
think of those automata as computing devices its nondeterministic natural be-
comes problematic. Recall that NFAs solve the same problems solved by DFAs.
Thus, it is natural to ask: is nondeterminism essential for capturing context-free
grammars and languages? Shortly after the introduction of PDAs, M. Schutzen-
berger began to study the set of languages that are accepted by deterministic
PDAs, the so called deterministic context-free languages [3]. Schutzenberger
proved that this set and the set of CFLs cannot be the same: he showed that
the former set is closed under operations under which the latter set is not closed.
Exercise 72 Let L be a language, and let
min (L) = fw 2 L : no proper pre…x of w belongs to Lg ;
max (L) = fw 2 L : no proper su¢ x of w belongs to Lg :
46
It can be proved that, if L is a deterministic context-free language then min (L)
and max (L) are also deterministic context-free languages. Use this to prove
that pal cannot be recognized by a deterministic pushdown automata (Hint. you
will have to prove that either min (pal) is not context-free or max (pal) is not
context-free, to prove the latter you can use the pumping lemma for context-free
languages).
The above facts show that nondeterminism provides computational power

to pushdown automata.
Exercise 73 Let palM f0; 1; 2g be the language
w2wR : w 2 f0; 1g :
Prove that palM is a deterministic context-free language that cannot be recog-

nized with one pebble. Use this language as the basis of a recursive construction
that shows that for all k > 0 there exists a deterministic context-free language
such that Lk 2
= REG k :
References
[1] M. Rabin, D. Scott. Finite automata and their decision problems. IBM Jour-
nal of Research and Development 3 (2): 114–125, 1959.
[2] N.Chomsky. Context-free grammars and pushdown storage. MIT Res. Lab.
Electron. Quart. Prog. Rept. 65, 1962.
[3] M. Schutzenberger. On context-free languages and pushdown automata. Inf.
and Control, 6(3): 246 - 264, 1963.
47
15 Nondeterministic Pushdown Automata and
Context-Free Grammars
Linguists describe the grammars of natural languages in terms of their block
structure, and describe how sentences are recursively built up from smaller
phrases, and eventually individual words or word elements. Context-free gram-
mars provide a simple and mathematically precise mechanism for describing
the methods by which phrases are built from smaller blocks. The simplicity of
context-free grammars makes the formalism amenable to rigorous mathematical
study. On the other hand, the expressive power of context-free grammars allows
them to describe the basic recursive structure of sentences, the way in which
clauses nest inside other clauses, and the way in which lists of adjectives and
adverbs are swallowed by nouns and verbs
De…nition 74 A context-free grammar is a tuple G = (V; T; S; R) ; where:
1. V is a …nite set of nonterminal symbols.

2. T is a …nite set of terminal symbols.
3. S 2 V is the initial variable.
4. R is a …nite set of production rules. A production rule is a expression like
X ! ; where X 2 V and 2 (V [ T ) :
The formalism of context-free grammars was developed in the mid-1950s by

Noam Chomsky [2]. In Chomsky’s generative grammar framework, the syntax
of natural language was described by context-free rules combined with transfor-
mation rules.
Let us consider an arti…cial
n language
o as it is the language of propositional
b
logic. Let us …x the set i : i 2 N as the set of propositional letters, where
given i 2 N the symbol bi denotes the binary code of i: Let GP L = (V; T; S; R)
be the context-free grammar given by:
1. The set V is equal to fS; X; var; cong :

2. The set T is equal to f^; _; :; (; ); 0; 1g.
3. The set R is constituted by the following production rules:
S ! X;
X ! : (X) j ((X) con (X)) j var;
con ! ^ j _;
var ! 0var j 1var j ";
where the symbol " denotes the empty string, and the rule var ! "
corresponds to a deletion rule.
48
Let G = (V; T; S; R) be a context-free grammar and let w 2 T : A derivation
of w is a …nite sequence 1 ; :::; n such that:
1. For all i n it happens that i 2 (V [ T ) :

2. 1 = S and n = w:
3. For all i n 1 it happens that i can be obtained from i by applying
a production rule in R:
Remark 75 Let w1 wn 2 (V [ T ) ; suppose that wi = X and suppose that

X ! is a production rule in R: The string
w1 wi 1 wi+1 wn
can be obtained from w1 wn after applying the production rule X ! :
We claim that GP L correctly describes the syntax of propositional logic,

that is: a string 2 T can be derived in GP L ; if and only if, the string
corresponds to a well formed formula of propositional logic. Consider the
formula ((1) _ (10)) : The sequence:
1. S;
2. X;
3. ((X) con (X)) ;

4. ((var) con (X)) ;
5. ((var) con (var)) ;
6. ((var) _ (var)) ;
7. ((1var) _ (var)) ;
8. ((1var) _ (1var)) ;
9. ((1) _ (1var)) ;
10. ((1) _ (10var)) ;

11. ((1) _ (10)) :
Is a derivation of this formula.
De…nition 76 Let G be a grammar, we use the symbol L (G) to denote the set
of strings that can be derived in G:
Exercise 77 Construct a context-free grammar Gpal such that L (Gpal ) is equal

to pal:
49
Remark 78 Block structure was introduced into computer programming lan-
guages by the Algol project (1957–1960), which, as a consequence, also featured
a context-free grammar to describe the resulting Algol syntax. This became
a standard feature of computer languages. The "block structure" aspect that
context-free grammars capture is so fundamental to grammar that the terms
syntax and grammar are often identi…ed with context-free grammar rules, espe-
cially in computer science. Formal constraints not captured by the grammar are
then considered to be part of the "semantics" of the language.
Chomsky proved that given a context-free grammar G; there exists a push-

down automaton MG such that L (G) = L (MG ) : Chomsky also proved that
given a PDA M; there exists a context-free grammar GM such that L (M) =
L (GM ) : This equivalence is the reason behind the universal acceptance of the
PDA architecture and its nondeterministic transitions.
Exercise 79 The elementary functions are the functions of calculus, which are
constructed from the monomials, the trigonometric functions, the exponential
and the logarithmic functions using sums, products, divisions and compositions.
Construct a context-free grammar GCalculus for the language constituted by all
those functions.
15.1 Nondeterministic LBAs and Context-Sensitive Gram-

mars
Let us begin with an exercise
Exercise 80 Investigate about the pumping lemma for context-free languages.

Let SQU ARES be the language
ww : w 2 f0; 1g ;
use the pumping lemma to prove that this language is not context-free.
The language SQU ARES require a di¤erent type of grammars, the so called
context-sensitive grammars.
De…nition 81 A context-sensitive grammar is a tuple G = (V; T; S; R) ; where:
1. V is a …nite set of nonterminal symbols.

2. T is a …nite set of terminal symbols.
3. S 2 V is the initial variable.
4. R is a …nite set of production rules. A production rule is a expression like
X ! ; where X 2 V; ; 2 (V [ T ) and 2 (V [ T ) :
50
Context-sensitive grammars were also studied by Chomsky, who placed them
in the top level of his hierarchy of generative grammars, the so called Chom-
sky Hierarchy, and which is constituted by the regular grammars (regular lan-
guages), context-free grammars (context-free languages), and the context-sensitive
grammars (context-sensitive languages) [3].
Exercise 82 Construct a context-sensitive grammar for SQU ARES:
Exercise 83 Can you characterize the context-free grammars that give place to
regular languages?
If the construction of a grammar for SQU ARES was a hard task, then the
reader will become scared if we ask him to prove that the following language is
context sensitive
SAT = f 2 f^; _; :; (; ); 0; 1g : is a well formed formula that is satis…ableg :
The reader can easily solve this latter task, if she(he) takes into account the
following important result by Kuroda [1].
Theorem 84 A language L is context-sensitive, if and only if, it can be recog-

nized by a nondeterministic linear bounded automaton (NLBA, for short).
Remark 85 Kuroda was a mathematician, (his grandfather was Teiji Takagi)

who took part of the …rst graduating class from the new Department of Linguis-
tics at MIT, where he wrote his dissertation under Chomsky’s supervision.
References
[1] S. Kuroda. Classes of languages and linear-bounded automata. Information
and Control 7(2): 207–223, 1964.
[2] N. Chomsky. Three models for the description of language. IEEE Transac-
tions on Information Theory, 2(3): 113–124, 1956.
51
16 Parsing
Let G be a context-free grammar. Consider the problem:
Problem 86 Recognition of the Context-Free Language L (G) :
Input: w 2 T :
Problem: decide if w 2 L (G) :
The proof of the theorem of Chomsky is constructive, and this implies that
we can assume that we are given a PDA MG that recognizes the language L (G) :
Suppose that L (G) is a nondeterministic context-free language that cannot be
recognized by a deterministic PDA. We get that MG is a nondeterministic PDA.
Perhaps the reader is tempted to use MG to solve the recognition problem for
L (G) : However, we have to take into account the nondeterministic nature of
MG : nondeterministic machines are not intended to be used in practice. If
we insist in use MG we will have to …gure out a method for simulating all
the computations of MG on the input string. This can be done, but it could
have an exponential cost. Thus, we have to conclude that MG provides us
with a (nondeterministic) solution to the recognition problem, but it could be
a very ine¢ cient solution: nondeterministic algorithms can be simulated by
deterministic ones, but this simulation can have an (super)exponential slow
down. Can we e¢ ciently solve the recognition problem for L (G)? If L (G) is
deterministic, then we can use a deterministic PDA that solves the recognition
problem, and this PDA provides us with a real-time solution. Thus, let us ask:
can be e¢ ciently solve the recognition problem for nondeterministic context-free
languages?
We can try to solve a harder problem. Consider the following problem:
Problem 87 Computing Derivations for L (G) :
Input: w 2 T :
Problem: decide if w 2 L (G) ; in that case compute a derivation of w in
G:
Or, even better, we can try to solve the problem of computing parse trees
for the strings in L (G) that are given as input.
16.1 Parse Trees

Generative grammars are important for linguistic analysis because those gram-
mars allow the construction of parse trees for the sentences within the languages
being analyzed. A parse tree decomposes a sentence into its constituent com-
ponents and shows the way those components are integrated into the sentence.
Let us consider an example
52
This is a parse tree for the formula (:p ^ q) ) (p ^ (q _ :r)) : One could argue
that the above parse tree is the meaning of this formula:
The meaning of a formula is its extension, which is equal to the set of its
models. Then, it can be argued that the meaning of a formula is a speci…cation of
the way this formula behaves with respect to all the possible truth assignments.
The above parse tree can be understood as such speci…cation. Given a truth
assignment for the formula, the value of this truth assignment corresponds to
the propagation (from the bottom to the root) of the truth values assigned to
the leaves of this tree.
Let w 2 L (GP L ) : A parse tree for w is a labeled tree (T; r; L) ; that satis…es:
1. r is the root of T:
2. L is a labeling function, that assigns to each node of T an element of
(V [ T ) :
3. L (r) = S:
4. The labels assigned to the leaves of T belong to T:
5. If one reads those leaves from left to right, he reads the string w:
6. The labels assigned to the inner nodes belong to V:

7. Suppose that v is a inner node, and suppose that v1 ; :::; vn are its sons,
ordered from left to right. Suppose also that the equalities
L (v) = X; L (v1 ) = a1 ; :::; L (vn ) = an
hold. Then the production rule X ! a1 an is a production rule in R:
Exercise 88 Let be the formula (((1) ^ (2)) _ (: ((3) ^ (: (2))))) that belongs
to L (GP L ) : Construct a parse tree for this formula.
Exercise 89 Use the grammar GCalculus to construct an algorithm for the

(symbolic) di¤ erentiation of elementary functions. Can you use the same gram-
mar for symbolic integration?
16.2 CYK Algorithm: Unreliable Machines Can be E¢ -

ciently Simulated by Reliable Machines
The Cocke–Younger–Kasami Algorithm (CYK algorithm, for short) is a pars-
ing algorithm for context-free grammars that employs bottom-up parsing and
dynamic programming [1].
53
16.2.1 The Algorithm
Let G be a context-free grammar. We say that G is in Chomsky normal form,
if and only if, any production rule in G has either the form A ! or the
form A ! BC; where 2 T and B; C 2 V: It is not hard to prove that any
context-free grammar G that does not produce the empty string is equivalent
to a grammar in Chomsky normal form. Moreover, given G as above, one can
compute G such that G is in Chomsky normal form and G is equivalent to
G :
The CYK algorithm receives as input a Chomsky grammar G and a string w;
and it outputs either a parse tree for w or a message indicating that w 2
= L (G) :
This algorithm is a dynamic programming algorithm, which means that it com-
putes an array, of which some entries can be easily computed at the beginning of
the computation, while the remaining entries are computed in a dynamic way
propagating the data that are encoded into the previously computed entries.
The computed array is equal to [Pl;s;v ]l s jwj;v2V ; where
Pl;s;v = 1; if and only if, the string w [l; s] can be derived from the variable v:
Note that it is easy to compute all the entries of the form Pl;l;v : To do
this, one simply has to check if G contains the production rule v ! w [l] : To
propagate the previously computed values one uses the following rule
Pl;s;v = 1 and Ps+1;r;w = 1 and (X ! vw) 2 R imply that Pl;r;X = 1:
It is easy to decide, after computing the whole array, wether w 2 L (G) : one
only has to check if the equality P1;jwj;S = 1 holds.
16.2.2 Generating a parse tree

The above algorithm is a recognizer that will only determine if w belongs to
L (G). It is simple to extend this algorithm into a parser that also constructs a
parse tree, by storing parse tree nodes as elements of the array, instead of the
boolean value 1. The node is linked to the array elements that were used to
produce it, so as to build the tree structure. Only one such node in each array
element is needed since we want to compute only one parse tree.
Exercise 90 Write down the above sketched parser algorithm.
16.2.3 Valiant’s Algorithm and Matrix Multiplication

3
The worst case running time of CYK is O jwj jGj : This makes CYK one of
the most e¢ cient algorithms for recognizing general context-free languages in
practice. Valiant gave an extension of the CYK algorithm [2]. His algorithm
computes the same parsing table as the CYK algorithm; yet he showed that
algorithms for e¢ cient multiplication of matrices with 0-1-entries can be uti-
lized for performing this computation. As a result, Valiant invented the …rst
54
subcubic algorithm for context-free recognition. The dependence on e¢ cient
matrix multiplication cannot be avoided altogether: Lee has proved that any
parser for context-free grammars working in time O n3 " jGj can be e¤ectively
converted into an algorithm computing the product of boolean matrices in time
"
O n3 3 (see [3]).
References
[1] T. Kasami. An e¢ cient recognition and syntax-analysis algorithm for
context-free languages- Technical report AFCRL. 65-758, 1965.
[2] L. Valiant. General context-free recognition in less than cubic time. J. Com-
put. Syst. Sci. 10 (2): 308–314, 1975.
[3] L. Lee. Fast context-free grammar parsing requires fast Boolean matrix mul-
tiplication. J. ACM. 49(1): 1–15, 2002.
55
17 The LBA Problems: Unreliable Machines Can-
not be E¢ ciently Simulated by Reliable Ma-
chines
Let G be a context-sensitive grammar. Consider the problem:
Problem 91 Recognition of the Context-Sensitive Language L (G) :
Input: w 2 T :
Problem: decide if w 2 L (G) :
Suppose we are given a NLBA M that recognizes L (G) : We can use M

to solve the above problem. O¤ course, we cannot use the machine M, which
is unreliable, but we can use the program encoded by M and simulate, using
a deterministic machine, all the computations of M on input w: We have to
ask: how many computations we have to simulate? How long could be those
computations?
Suppose that w has length n: We know that M cannot use the cells that
are to the right of the n-th cell. Then, any halting computation of M, on this
input, is a sequence c1 ; :::; cN such that for all i N the con…guration ci is a
triple (qi ; posi ; Wi ) ; where qi 2 Q; posi 2 f1; :::; ng and Wi 2 n : Note that any
n
halting computation cannot repeat a con…guration. There are at most jQj n j j
of those con…gurations, and this implies that the halting computations of M,
n
on input w; cannot take more than jQj n j j steps. Now, we assume the worst
possible case, we assume that M has to take a nondeterministic choice at each
step of its computations. Recall that the choices of M are elements of the
set Q fw; ng f 1; 0; 1g : Thus, at each step the machine has no more
than 6 jQj j j ; choices, and this implies that the machine can execute at most
jQjnj jn
(6 jQj j j) di¤erent halting computations of this length.
We get that M can be simulated using a deterministic machine SM , and we
n jQjnj jn
get that the running time of this machine could be close to jQj n j j (6 jQj j j) ;
which is astronomical. We also get that SM uses a lot of work space. The work
n
space used by this machine is close to jQj n j j cells, and this number is very
much greater than the n cells used by M. We ask: can we do better? There
does exist something like a CYK Algorithm for context-sensitive languages?
17.1 Space Complexity

What should we expect of a CYK-like algorithm for context sensitive languages?
We can forget nondeterminism if we can solve the same problems using the
same computational resources. Let G be a context-free grammar. The PDA MG
solves the recognition problem for L(G) in polynomial time, and CYK algorithm
solves the same problem in polynomial time. Thus, it could be said that we can
forget our worries about the nondeterministic architecture of PDAs since we
56
can use the CYK deterministic algorithm instead of those PDAs. We can also
dispense us of using nondeterministic linear bounded automata, because we can
use the naive simulation-algorithm sketched in the previous section. However,
we have to note that we will pay a very high prize if we use this simulation
algorithm: the running time, and the space requirements of this algorithm are
astronomical. Then, we have to look for better algorithms retaining some of the
advantages of the original NLBAs. We have to ask: what are those advantages?
Suppose that we are given a NPDA M that recognizes L (G) : The running time
of this automaton could be exponential. Then, we have to conclude that the
strength of M is not related to its running time. On the other hand, we have
to observe that a linear bounded automaton is a Turing machine that satis…es
a further constraint that is related to the amount of memory used by those
automata.
De…nition 92 Let M be a nondeterministic Turing machine which is provided
with a read-only input tape and a …nite number of work tapes, one head per tape.
We say that M is a multitape Turing machine. Let w be an input of M, we
de…ne sM (w) as
max f#cells (c) : c is a computation of M on wg ;
where #cells (c) is equal to the maximum number of cells of the work tapes that
are simultaneously occupied along the execution of c:
We de…ne SM : N ! N by the equation
n
SM (n) = max fsM (w) : w 2 g:
Function SM measures the memory requirements of the machine M.
Note that linear bounded automata are essentially the same as multitape
Turing machines that satisfy the inequality SM (n) n:
Exercise 93 Let C > 0; and let M be a multitapeTuring machine that satis-
…es the inequality SM (n) Cn. Prove that there exists a machine M that
recognizes the language L (M) and which satis…es the inequality SM (n) n:
Let f : N ! N be a non-decreasing function. We de…ne two complexity
classes:
L : L can be recognized by a multitape

N SP ACE (f ) = ;
Turing machine M such that SM (n) 2 O (f )
L : L can be recognized by a multitape deterministic
SP ACE (f ) =
Turing machine M such that SM (n) 2 O (f )
We use the symbol N LIN to denote the class N SP ACE (n) ; and the symbol
LIN to denote the class SP ACE (n) :
Exercise 94 Prove that N LIN is equal to the set of context-sensitive languages
(the class LIN is then equal to the set of deterministic context-sensitive lan-
guages).
57
17.2 The LBA Questions
The two LBA questions go back to Kuroda, who asked:
1. Can we forget nondeterministic LBAs, and work only with deterministic

LBAs: Let M be a NLBA, does there exist a deterministic LBA that
recognizes the same language as M? Or, equivalently: is N LIN equal to
LIN ?
2. Is N LIN closed under complement?
The …rst of the above two questions remains open. Let us discuss in brief
the second question.
Let M be a deterministic machine, and let co-L (M) be the language
fw : w is rejected by Mg :
Can we recognize co-L (M) using a machine that satis…es the same time bounds
and the same memory requirements? Yes, we can, and it is enough if we inter-
change the role of the accepting and rejecting states of M. Now suppose that
M is a nondeterministic machine. It is not clear if we can recognize co-L (M)
using the same type of machines. Note that:
1. If L can be recognized by a NFA, then co-L can be recognized by a NFA.
2. There exists a context-free language L such that co-L is not context-free,

that is: there exists a PDA M such that co-L (M) cannot be recognized
by PDAs.
Exercise 95 Prove that the languages co-SQU ARES is context-free.
What can be said about context-sensitive languages?
References
[1] R. Floyd. Nondeterministic Algorithms. Journal of the ACM. 14 (4): 636 -
644, 1967.
58
18 The Immerman-Szelepcsényi Theorem
The Immerman–Szelepcsényi theorem states that nondeterministic space com-
plexity classes are closed under complementation. It was proven independently
by Neil Immerman and Róbert Szelepcsényi in 1987, for which they shared the
1995 Gödel Prize (see [1] and [2]). In its general form the theorem states that
NSPACE(s (n)) = co-NSPACE(s (n)) for any function s (n) log (n) : We are
interested in the special case s (n) = n:
18.1 Sketch of the Proof

We prove that the set of context sensitive languages is closed under complement.
Exercise 96 Let M a deterministic LBA, and let L be the language accepted

by M. Prove that co-L is also context-sensitive, and prove that it can be recog-
nized by a deterministic LBA. Observe that the proof of this assertion is quite
straightforward. Suppose that M is nondeterministic, explain why it is hard to
prove that co-L is context-sensitive.
Let M = (Q; q0 ; ; ; H; A; ) be a NLBA. We know what this is; merely a

machine which has more than one possible move at each step. And, if it can
achieve a halting con…guration, it accepts.
Exercise 97 Prove that the number of possible con…gurations of M on input

jwj
w is equal to jwj jQj j j :
Let sw be the initial con…guration of M, on input w, and let m be the

number of con…gurations that can be reached from sw : Note that the binary
code of m …ts the tape of a NLBA that is processing input w: This means that
we can dream of building a NLBA N that counts the number of con…gurations
that are reached by M from the initial con…guration sw (the same automaton
N for all input w):
Lemma 98 Let M be a NLBA and let w be an input of M. There exists a

NLBA N , which only depends on M, and which can locate and examine m
con…gurations reachable by M provided that there are at least m con…gurations
that are reachable from sw :
Proof. We have an NLBA M which is …xed and determines the construction

of N . Let w be an input of M of length n, let m be a positive integer, and
suppose that we know that there are at least m con…gurations reachable from
sw . Our task is to …nd them. Consider the following procedure:
Set x = 0:
n
For i = 1 to n jQj j j generate the i-th possible con…guration of the ma-
chine and call it ci .
59
– Guess a path from s to ci : Verify that it is a proper path.
– If the guessed path is proper set x = x + 1:
Verify that x m (otherwise reject).
This procedure is a good example of the guess and verify technique used in
nondeterministic algorithms. All we did was exploit our de…nition of nondeter-
minism. We looked at all possible con…gurations and counted those which were
reachable from the starting con…guration.
Lemma 99 There is a NLBA N which can count the number of con…gurations

reachable from sw :
Proof. Consider the following procedure:
Set n0 = 1.
Set i = 0:
Repeat
– i = i + 1.
– ni = 0.
– Set m = 0:
n
– For j = 1 to n jQj j j generate sj : Guess whether sj can be reached
in i steps or less. If a path from sw to sj is veri…able then ni = ni +1:
If reached in less than i steps then m = m + 1:
– Verify that m = ni 1 (otherwise reject).
until ni = ni 1:
The guessing step is just the algorithm of our previous lemma. We do it by

…nding all of the con…gurations reachable in less than i steps, and seeing if any
of them is sj or if one more step will produce sj . Since we know ni 1 , we can
verify that we have looked at all of them. The remainder is just counting. We
do not have to save all of the ni ’s, just the current one and the previous one.
All this can be done on n squares of tape (using several tracks).
Theorem 100 The set of context-sensitive languages is closed under comple-

ment.
Proof. Most of the work has already been done. To built a machine which
accepts the complement of L (M) involves
putting the previous two procedures together. First …nd out exactly how
many con…gurations are reachable. Then examine all of them, and if any ac-
cepting con…gurations are encountered, reject. Otherwise accept. The theorem
is proved.
60
References
[1] N. Immerman. Nondeterministic space is closed under complementation.
SIAM Journal on Computing, 17(5): 935 - 938, 1987.
[2] R. Szelepcsényi. The method of forcing for nondeterministic automata. Bul-

letin of the EATCS, 33: 96 - 100, 1987.
61
19 L vs NL
There are problems that can be solved using a small amount of workspace, and
these are precisely the problems that can be e¤ectively parallelized to be solved
in polylogarithmic time.
19.1 The Class L

The class L (logspace, DSPACE(log (n))) is the class of problems that can be
solved using a logarithmic amount of workspace. Observe that we are talking
about machines that use a sublinear amount of workspace, and this forces us to
separate the input tape, which always has linear size, and the tapes where the
computing work is done, which can be of sublinear size.
De…nition 101 A logspace Turing machine is a machine with k + 1 tapes,
one head per tape. We can suppose that those tapes are numbered, from 1 to
k + 1; and we suppose that tape 1 is a read only input tape. We suppose that
the remaining k tapes are work tapes that are empty at the beginning of the
computation. Moreover, there exists a constant c such that the machine uses at
most c log (n) cells of each work tape along its computations on inputs of size n:
There are nontrivial problems that can be solved in logspace. Let be a
…rst order sentence over the vocabulary : Consider the problem:
Problem 102 MC( ): Model Checking of .

Input: U, where U is a …nite -structure.
Problem: decide if U satis…es :
Exercise 103 Prove that MC( ) can be solved using a logspace Turing ma-
chine.
19.2 The class NL

The class NL is equal to NSPACE(log (n)). We have that L NL and it is not
known if this containment is strict. What do we know about the class NL?
1. A famous theorem of W. Savitch states that NL is contained in DSPACE(log2 (n)).
Given s : N ! N such that s (n) log (n) the containment NSPACE(s (n))
2
DSPACE((s (n)) ) holds [1] (see below).
2. The class NL is equal to the class co-NL. This equality is a consequence
of the Immerman-Szelepcsényi Theorem, which, in its general form, states
that for all s : N ! N, if the inequality s (n) log (n) holds, then the
equality NSPACE(s (n)) = co-NSPACE(s (n)) holds.
Exercise 104 Your work is to adapt the proof of The Immerman–Szelepcsény
Theorem discussed in the previous lecture to get a proof of the general case.
62
19.3 Complete Problems for NL
The L vs NL question is one of the most important problems of theoretical
computer science. This question, as well as the P vs NP question, has been
attacked using suitable notions of reducibility and completeness.
De…nition 105 Let L; T be two languages. We say that L is logspace reducible

to T; if and only if, there exists a logspace Turing machine M which, on any
input x instance of L; computes an instance of T that we denote with the symbol
rx and such that
x 2 L, if and only if, rx 2 T:
We use the symbol L 4logspace T to denote that L is logspace reducible to T: We
have:
Exercise 106 Suppose that L; T belong to NL and suppose that L 4logspace T .
1. If T belongs to L, then L also belongs to L.

2. If L does not belong to L, then T does not belong to L.
De…nition 107 Let L be a language in NL, we say that L is NL-complete, if

and only if, given T that belongs to NL, the language T is logspace reducible to
L:
Exercise 108 Let L be a language that is complete for NL. We have that L is
equal to NL, if and only if, L belongs to L.
The above exercise indicates that the notion of NL-completeness can help
us to settle the L vs NL question. However, we have to ensure that this notion
is nonempty. Do there exist problems that are complete for NL? Consider the
following problem:
Problem 109 GA: Digraph Accessibility.
Input: (G; s; T er), where G is a …nite digraph, s 2 V (G) and T er V (G)

Problem: decide if T er is accessible from s:
Problem GA belongs to NL. To see the latter it is enough to consider the

following nondeterministic procedure:
Start at s; and guess a next-neighbor until a node in T is reached. If you
can guess a path from s to T accept the input, otherwise reject.
We have:
Theorem 110 Problem GA is NL-complete.
63
Proof. Let L be a language in NL, and let M be a logspace Turing machine
that recognizes the language L: Let w be an instance of L: Let sw be the initial
con…guration of M determined by the input w: The set of con…gurations of M
that can be accessed from w has polynomial size with respect to jwj ; and this
implies that all its elements can be described using a logarithmic number of bits.
Moreover, the whole graph can be constructed using logspace. Let GM;w be this
graph. Note that sw 2 V (GM;w ) : Let Tw be the set of accepting con…gurations
that are contained in the set V (GM;w ) : We have that w 2 L; if and only if,
(GM;w ; sw ; Tw ) is a YES-instance of GA. The function that sends w in the triple
(GM;w ; sw ; Tw ) is a logspace reduction of L into GA. The theorem is proved.
19.4 Savitch’s Theorem

W. Savitch was a graduate student working for Steve Cook, when he proved
his theorem; actually the theorem was his thesis. In the 1960’s one of the main
focuses of people working on theoretical computer science was language theory.
In 1965 at IFIP Congress in New York City an important paper was presented:
Memory bounds for recognition of context-free and context-sensitive languages
by Philip Lewis, Juris Hartmanis and Richard Stearns . This paper contains
the following result:
Theorem 111 [2]Let L be a context-free language, the language L belongs to

the class N SP ACE log2 (n) .
Savitch realized that the LHS paper had a fundamental idea, an idea that was
very clever, yet an idea that even the authors did not completely understand.
In proving that every context-free language was accepted using O log2 (n)
nondeterministic work space, Lewis et al had used a clever method for keeping
track of the pushdown states. Essentially, Savitch saw that when separated
from context-free languages, what they were doing was exactly what he need to
prove his theorem.
19.4.1 Sketch of the proof

Observe that the class DSPACE(log2 (n)) is closed under logspace reductions.
Then, if we prove that GA belongs to DSPACE(log2 (n)) we get the containment.
The basic idea of the algorithm is to solve recursively a somewhat more general
problem, testing the existence of a path from a vertex s to another vertex t
that uses at most k edges, where k is a parameter that is given as input to the
recursive algorithm; GA may be solved from this problem by setting k to n. To
test for a k-edge path from s to t, one may test whether each vertex u may be
the midpoint of the path, by recursively searching for paths of half the length
from s to u and u to t. This search calls itself to a recursion depth of O (log (n))
levels, each of which requires O (log (n)) bits to store the function arguments
and local variables at that level, so the total space used by the algorithm is
O log 2 (n) . Although described above in the form of a program in a high-level
64
language, the same algorithm may be implemented with the same asymptotic
space bound on a Turing machine.
References
[1] W. Savitch. Relationships between nondeterministic and deterministic tape
complexities. Journal of Computer and System Sciences, 4(2): 177–192, 1970.
[2] P. Lewis, R. Stearns, J. Hartmanis. Memory bounds for recognition of
context-free and context-sensitive languages. SWCT (FOCS) 1965: 191-202.
65
20 The Pebble Hierarchy Revisited
A nondeterministic k-pebble automaton is a k-pebble automaton whose transi-
tion function is nondeterministic. We have to ask: what is the relation between
the set of languages accepted by deterministic k-pebble automata and the set
of languages recognized by nondeterministic k-pebble automata. We use the
symbol N REG k to denote the latter class.
Let k 0: It is easy to check that any k-pebble automaton can be sim-
ulated by a logspace Turing machine: one can use logarithmic space to track
the position of the k-heads. An analogous fact holds true for nondeterministic
pebble automata: those automata can be simulated by nondeterministic Turing
machines. It is a little bit more surprising that the reciprocal of those two facts
also hold true. It follows form the work of Petersen [1] and Sudborough [2] that:
[
1. L = REG k :
i 0
[
2. NL = N REG k :
i 0
We get that the deterministic and the nondeterministic pebble hierarchies

correspond to hierarchical decompositions of L and NL (respectively). We also
get that a strong separation between deterministic and nondeterministic peb-
ble classes would yield a solution to one of the most important problems of
theoretical computer science, the question L vs NL.
20.1 Context-free Languages and the Pebble Hierarchy

Recall a question that was asked in a previous lecture: are there CF languages
that cannot be computed with a bounded number of pebbles? Suppose that
there is such a context-free language, and call it L: We get that L does not
belong to L. It is known the existence of such a context-free language would
imply that L is di¤erent of NL. Thus, it wont be easy to construct such a
language (supposing that such a language actually exists). However, we have a
good candidate, the language
pal : ci w : w 2 pali :
We have:
Exercise 112 If there

[ does not exist k such that all the powers of pal belong to
REG k ; then pal 2
= REG k :
k 0
Exercise 113 pal is a context-free language.
We also have that:
66
Lemma 114 The language pal belongs to N REG 4 :
Proof. We can construct a nondeterministic 4-pebble automaton recognizing

the language pal : This automaton uses a pebble to count the number of palin-
dromic factors encountered, and it uses the remaining three pebbles to locate
palindromic factors.
Suppose that the scanning head is located over the cell that contains the i-
th palindromic factor, and suppose that it is carrying the three search-pebbles.
The automaton leaves a pebble on this cell, advances to the right carrying the
remaining two search-pebbles and nondeterministically guesses the center of the
next palindromic factor. Then, it uses one pebble to move to the left, and the
other one to move to the left. If the left pebble encounters the third pebble,
without having found a missmatch, then the (i + 1)-th palindromic factor has
been found. The automaton moves one position to the right the counter-pebble,
and collect the three search pebbles at the right end of the (i + 1)-palindromic
factor. The lemma is proved.
20.1.1 The Language pal2 and Two Pebbles

It is easy to prove that:
Exercise 115 For all k 1 the language palk belongs to N REG 3 :
Let us consider the special case k = 2: We have:
Lemma 116 The language pal2 belongs to N REG 2 :
Proof. Let M the 2-pebble automaton that works as follows:

At the beginning of the computation the scanning head advances to the
right carrying the two pebbles until M nondeterministically guesses the middle
of the …rst palindromic factor. Then, M begins to move one pebble rightward,
the other pebble leftward, until either the latter pebble reaches the …rst cell
of the tape or a missmatch is found. Suppose the former. We have that the
rightmost pebble is placed at the right end of a palindromic pre…x. Then, M
places the other pebble at the right end of the output string, and runs a zig-
zag strategy to check if the su¢ x that is placed between the two pebbles is a
palindrome.
Can we prove that pal2 does not belong to REG 2 ? Suppose it is true that
2
pal does not belong to REG 2 : A proof (from the book) of this fact could yield
k
a proof of a very much stronger fact, namely, that pal2 does not belong to
REG k+1 : Observe that this latter fact implies that L is di¤erent of NL. Thus, it
is natural to feel that such a proof is out of scope. Can we prove a weaker fact,
and pave the way towards a future proof of this seemingly important fact?
A 2-head automaton is a two-way …nite state automaton that has two scan-
ning heads, which cannot sense each other. It is easy to recognize pal using a
suitable 2-head automaton, and it is easy to prove that:
67
Exercise 117 It is possible to simulate two non-sensing heads using two pebbles
(it is possible to simulate k heads using k pebbles).
It is very much harder to prove that one can simulate k pebbles using k + 1
nonsensing heads. Let REG h2 be the set of languages that can be recognized
using two nonsensing heads. We have that
REG h2 REG 2 REG h3 REG 3 :
It seems that REG h2 is strictly contained in REG 2 : Thus, to begin with, we

could try to prove that pal2 does not belong to REG h2 : To do the latter we can
use a characterization of REG h2 in terms of two-way automata provided with an
auxiliary pushdown stack.
De…nition 118 A 2DCSA (two-way deterministic checking stack automaton)

is a two-way …nite state automaton provided with an auxiliary checking stack.
Let M be a 2DCSA and let w be an input of M. The computation of M, on
input w, is constituted by two phases. In the …rst writing phase the scanning
head moves to the right while the automaton push characters on the checking
stack. The second phase begins when the scanning head turns to the left for the
…rst time. Then, the automaton cannot write on the stack anymore, but it can
enter this stack in a read-only mode.
We have that L belongs to REG h2 ; if and only if, L can be accepted by a

2DCSA. It seems that 2DCSA’s cannot recognize the language pal2 :
References
[1] H. Petersen. The Equivalence of Pebbles and Sensing Heads for Finite Au-
tomata. FCT 1997: 400-410.
[2] I. Sudborough. On tape bounded complexity classes and multihead …nite
automata. J. Compt. System Sci. 10:338 - 345, 1975.
68
21 P vs NP
With this lecture we arrive to one of the most important problems of theoretical
computer science P 6= NP.
21.1 The Complexity Class P

The complexity class P is the set constituted by all the problems that can be
solved by polynomial time deterministic Turing machines. It is said that P is
constituted by the problems that can be solved e¢ ciently. Thus, most people
think that the notion of polynomial time deterministic Turing machine and the
notion of e¢ cient algorithm are almost the same notions. Why do we say that
P is constituted by all the tractable problems? This is o¤ course a convention,
but this a convention is based on solid grounds:
1. Polynomial functions are functions of slow growth (though it cannot be

said a polynomial of degree 100 grows slowly).
2. The set of polynomial functions is closed under sum, multiplication and
composition, and this implies that the set of deterministic Turing machines
that run in polynomial time is closed under di¤erent forms of composition.
The latter implies that P is closed under polynomial time reductions (see
below).
3. Cobham noted in 1964 that the class P remained invariant when one chose
a di¤erent feasible model of computation, that is: let C be one of the
universal models of computation that was known at this time, and let
PC be the set of problems that can be solved by C-machines that run
in polynomial time, Cobham observed that for all those C’s the equality
P=PC holds (see [1]).
We will assume the above point of view in the remainder of this work, that
is: we will assume that the complexity class P is constituted by all the tractable
problems.
21.2 The Complexity Class NP

The reader should observe that most problems in mathematics, and most theo-
rems in mathematics have an existential character: those problems ask for the
construction of a structure with some special properties. We can cope with
those problems using the following strategy:
Guess a candidate.
Check that this candidates holds the intended property.
If the set of candidates is in…nite the above strategy gives place to nonhalting
algorithms. If the set of candidates is a large …nite set, this strategy gives place
69
to ine¢ cient brute force algorithms, and if the set of candidates is small (can
be made small) we get an e¢ cient algorithm.
Suppose that the input is a …nite structure, and suppose that we are asked
to compute a substructure exhibiting some special property. Observe that this
scenario corresponds to the above second category: the set of substructures has
exponential size. Most problems of combinatorial optimization have this kind
of structure: most of those problems ask for the construction of substructures
exhibiting some special properties. Most of those problems also have a second
special feature: the structural property the de…nes the problem is easy to check.
Consider the following examples:
The Hamiltonian cycle problem.

The Eulerian trail problem.
The traveling salesman problem.

The k-coloring problem.
The clique problem.
We can solve any one of those problems by guessing a substructure and then
checking that this substructure holds the property that de…nes the problem. If
the candidate does not satisfy this property, then we try a second candidate,
and then a third, and so on. We halt when, either we …nd a substructure that
satis…es the property, or we try all the possible candidates without …nding a
solution to the problem. By doing so, we are solving the existential problem
by performing exponential many checks, that can be done, each one of them,
in polynomial time. We can work in a sequential fashion and use exponential
time, or we can use an exponential number of processors and solve the problem
in polynomial time, or we can work nondeterministically with a machine that
always do the correct guess. We have to observe that none of those three options
is feasible. Then, it is worth to consider a fourth option, namely: can the search
spaces of all those existential problems be reduced to sets of polynomial size?
The reader should think in the Eulerian problem. Let G be an instance of
the problem. The set of possible Eulerian trails for G is equal to the set of
permutations of E (G). However, the Theorem of Euler tells us that this large
3
set can be replaced by the set V (G) ; and that the special individual that must
be searched, in this latter very much smaller space, is a triple such that all
its three nodes are odd degree nodes. This theorem yields a polynomial time
algorithm for the problem. We ask: do any existential problem like the …ve
listed above count with an Euler-like Theorem? We would like to deal with this
question. To this end we have to de…ne, in a precise way, what do we mean with
the expression: a problem like the …ve listed above.
De…nition 119 The complexity class NP is the set of languages that can be
recognized by nondeterministic polynomial time Turing machines.
70
Exercise 120 Explain why the complexity class NP is also equal to the set of
languages whose instances can be correctly classi…ed by doing an exponential
number of easy checks.
Exercise 121 The complexity calls NP can be presented in terms of the notion
of certi…cate. Let R be a binary relation. We say that R is p-balanced,
if and only if, the following two conditions are satis…ed:
Let # be a character that does not belong to : The language
LR = fx#y : (x; y) 2 Rg
can be recognized in polynomial time.

There exists a polynomial function p (n) ; such that given x 2 there
exists y such that (x; y) 2 R; if and only if there exists y such that (x; y) 2
R and jyj p (n) :
Let L9R be the language fx : 9y ((x; y) 2 R)g : Prove that L belongs to NP,
if and only if, there exists a p-balanced relation R such that L = L9R :
Let x 2 L9R ; and let y be a string such that (x; y) 2 R; we say that y is a
certi…cate for x:
21.3 Cook’s Theorem

Suppose we are given a language in NP, say L, and suppose that we are asked
to decide if x belongs to L:
We can choose to work deterministically, look hard for a certi…cate; say y;

and then check that (x; y) 2 RL .
Or, we can choose to work nondeterministically, skip the search for y; guess
such a y (y is given by a friendly advisor), and then do the easy check: we
guess a certi…cate, and we check that this is a correct certi…cate.
Remark 122 We cannot implement the second option, what we do in practice

is to simulate this strategy listing all the possible certi…cates and checking, one
by one, if there is one that works.
It is very much harder to construct a mathematical proof than to check a

mathematical proof. This suggests that the deterministic procedure takes very
much longer than the nondeterministic one, and this suggests that NP contains
problem that do not belong to P. Which problems? An easy answer to this
question would be: the hardest problems in NP.
De…nition 123 Let L ; and let T , we say that L is polynomial time

reducible to T; if and only if, there exists a polynomial time algorithm M, which,
on input x 2 ; computes rx 2 such that:
x 2 L, if and only if, rx 2 T:
71
Exercise 124 Suppose that L is polynomial time reducible T (L 4pt T ). It can
be argued that T is at least as hard as L : if T belongs to P, the language L
also belongs to P.
We would say that L is the hardest problem in NP, if and only if, any problem
in NP can be polynomial time reduced to L:
Exercise 125 Choose one of your favorite problems in NP, and reduce it to
SAT.
It seems that many people did the same exercise around 1970, and it seems
that most of those people arrived to the same conclusion we arrived: it is easy
to reduce a problem in NP to the problem SAT. This suggests that SAT is one
of the hardest problems in NP.
De…nition 126 Let L be a language, we say that L is NP-hard, if and only

if, any problem in NP can be polynomial time reduced to L: We say that L is
NP-complete, if and only if, the problem is NP-hard and belongs to NP.
Theorem 127 Cook’s Theorem

SAT is NP-complete.
Remark 128 Leonid Levin independently proved an equivalent result [2].
Remark 129 Before Cook’s paper there was not much interest in problems
computable in polynomial time by nondeterministic Turing machines. The class
NP only became an interesting class after Cook’s Theorem.
Exercise 130 Suppose that L is NP-hard, and also suppose that L 4pt T: Prove
that T is NP-hard.
21.4 NP and co-NP

A language L belongs to co-NP, if and only, the language co-L belongs to NP.
The class can be de…ned as follows: a decision problem is in co-NP precisely if
for any no-instance x there is a "certi…cate" which a polynomial-time algorithm
can use to verify that x is a no-instance, and for any yes-instance there is no
such certi…cate. Equivalently, L belongs to co-NP, if and only if, there exists
a nondeterministic Turing machine M such that for every instance x, this is a
yes instance, if and only if, any computation of M, on input x; is an accepting
computation.
The complement of any language in NP is a problem in co-NP. A language
L is co-NP complete, if and only if, L is in co-NP and for any problem in co-
NP there exists a polynomial-time reduction from that problem into L. An
example of such a problem is that of determining if a formula in propositional
logic is a tautology: that is, if the formula evaluates to true under every possible
assignment to its variables.
Consider the following problem:
72
Problem 131 Integer Factorization
Input: (m; n) ; such that m and n are positive integers.
Problem: decide if m has a factor k such that 1 < k n:
This problem is one of the few problems that is known to belong to both
NP and co-NP (but not known to be in P). It is easy to check that the problem
belongs to NP: if m does have such a factor then the factor itself is a certi…cate.
Membership in co-NP is also straightforward: one can just list the prime factors
of m, all greater or equal to n, which the veri…er can con…rm to be valid by
multiplication and the AKS primality test.
References
[1] A. Cobham. The Recognition Problem for the Set of Perfect Squares. SWAT
(FOCS) 1966: 78 - 87, 1973.
[2] L. Levin. Universal search problems. Problems of Information Transmission
(in Russian) 9 (3): 115116.
[3] N. Chomsky. On Certain Formal Properties of Grammars. Inf. Control. 2(2):
137 - 167, 1959.
[4] N. Chomsky. A Note on Phrase Structure Grammars. Inf. Control. 2(4): 393
- 395, 1959.
73
22 NP-Completeness: Automata Synchroniza-
tion
Let L be a problem. Suppose that we suspect that L is NP-hard. This conjecture
leads us to search for a NP-hardness proof for L. It follows from the de…nition
that L is NP-hard, if and only if, SAT is polynomial time reducible to L. It also
follows from the de…nition of NP-hardness that problem SAT is as reducible
to L as any other NP-complete problem, that is: theory tells us that it is the
same to look for a reduction of SAT into L than to look for a reduction into L
of a NP-hard problem that seems to be closely related to this latter problem.
However, theory tells us one thing, and practice tells us a completely di¤erent
thing: if we want to prove that L is NP-hard, we should look into the enormous
list of problems that are known to be NP-complete, and then choose those
problems that exhibit structural analogies with problem L: Sometimes one can
easily …gure out a reduction of SAT (CNF-SAT) into L: Sometimes it is very
much easier to …gure out a reduction of a suitable NP-complete problem into L
than a reduction from SAT.
We would like to study two algorithmic problems related to automata. Those
two problems are NP-hard, and they represent two opposite sides of the wide
spectrum of possibilities. The …rst one is the minimal-synchronizing word prob-
lem (MSS, for short), and the second one is the separating words problem (SW,
for short).
It can be easily proved that MSS is NP-hard. The NP-hardness proof for
this problem is a straightforward reduction of CNF-SAT into it (this reduction
is one the most crystal-clear reduction this author knows). It can also be proved
that SW is NP-hard. However, this second result is a recent result, and the NP-
hardness proof is a reduction of a technical problem about …nite semigroups into
SW. It is true that one can use this reduction, and other auxiliary reductions,
to construct a reduction of SAT into SW. We can search the literature and
reconstruct the path that goes from SAT to L: However, if we do the latter we
will have to compose no less than four nontrivial polynomial time reductions.
22.1 Automata Synchronization

Let M = (Q; ; ) be a triple such that is a function from Q into Q:
Observe that M is a DFA without initial and accepting states. Observe also
that we can think of M as a …nite labelled digraph, the labelled digraph GM
V (GM ) = Q:
Given p; q 2 Q there exists an edge (p; q) with label a; if and only if, the
equality (p; a) = q holds.
We can think of GM as a territory that is navigated by agents, whose soft-

ware is equal to ; and which interpret programs that belong to : suppose
74
agent x is placed at node q; and suppose that it has to execute the program
w 2 ; then after i time units this agent will be placed at node b (p; w [1; :::; i]) :
Now suppose that we have several agents scattered over GM ; suppose that we
do not know the speci…c position of each agent, and suppose that we want to
send a message w 2 , the same message for all the agents, that leads all
those agents to same node q: What we need is a synchronizing string for the
automaton M.
De…nition 132 Let M = (Q; ; ) be a DFA (without initial and accepting
states), and let w 2 : We say that w is a synchronizing string for M; if and
only if, for all p; q 2 Q the equality b (p; w) = b (q; w) holds. We say that M
is a synchronizing automaton, if and only if, there exists a synchronizing string
for M.
There are DFAs that are not synchronizing.
Exercise 133 Show that any unary automaton without a sink state is not syn-
chronizing. Prove also that any automaton with more than one sink is not
synchronizing.
Consider the following problem:
Problem 134 Recognition of Synchronizing Automata

Input: M, where M is a DFA.
Problem: decide if M is synchronizing automata.
It is easy to recognize the automata that are synchronizing, we have:

Theorem 135 The problem RSA (recognition of synchronizing automata) be-
longs to P.
Proof. Let M be a DFA. We say that M is pair-synchronizing, if and only if,
for all p; q 2 Q there exists a string wp;q such that b (p; wpq ) = b (q; wpq ) : It is
easy to prove that M is synchronizing, if and only if, M is pair-synchronizing.
Then, the problem RSA reduces to the problem of recognizing the automata
that are pair synchronizing. Let M(2) be the pair-automaton that is de…ned by:
Q(2) = fS Q : jSj 2g.
(2)
is de…ned by the equation
(2)
(S; a) = f (p; a) : p 2 Sg :
We have that M is pair-synchronizing, if and only if, the set
n o
S 2 Q(2) : jSj = 1
can be accessed from any other state of M(2) : Thus, the problem of recog-
nizing the automata that can be synchronized is polynomial time reducible
to an accessibility problem about digraphs. The theorem is proved.
75
Let M be a synchronizing automaton. We de…ne the reset treshold of M as
rt (M) = min fjwj : w is synchronizing for Mg :
We have:
Exercise 136 There is a naive strategy for the construction of synchronizing

strings. This strategy goes as follows: we synchronize a pair p; q using a string
wpq : After that we synchronize a pair of the states that belong to the image of
Q under the action of w: We continue in this way, working on pairs, until we
synchronize all the states of M. Prove that this naive strategy gives place to
strings of cubic length.
Let us set
rt (n) = max frt (M) : M is a n-state synchronizing DFAg ;
we have that r (n) 2 O n3 : The famous Cerný Conjecture estates that rt (n)
2
is equal to (n 1) (see [1]). Then, if this conjecture is true, there is a large gap
between the length of the shortest synchronizing strings, and the length of the
strings that can be constructed using the above sketched naive strategy. One of
the ubiquitous questions of computer science is the question about the existence
of algorithms that beat the naive algorithmic solutions. Thus, it is natural to
ask wether there exists a polynomial time algorithm for the construction of
shortest synchronizing strings. Consider the problem:
Problem 137 Shortest Synchronizing Strings
Input: (M; k), where M is a synchronizing automaton and k is a positive

integer.
Problem: decide if rt (M) k:
We will prove that this problem is NP-hard.
Remark 138 If P6=NP there exist synchronizing automata whose reset length
is lesser than the length of the synchronizing strings that are constructed using
the naive pair-synchronization strategy.
22.2 Eppstein Reduction

D. Eppstein proved that SSS (shortest synchronizing strings) is NP-hard [?, ?].
The proof is a reduction from CNF-SAT, and this is one of the most crystalline
reductions this author knows. The reduction consists in representing the CNF
; given as input, as a labelled sink-digraph.
Theorem 139 The problem SSS is NP-hard.
76
Proof. Consider the propositional formula
(X1 ^ :X2 ^ :X3 ) _ (X1 ^ X3 ) ;
and consider the automaton

Let be a CNF (like the above formula) with k clauses and n variables.
The reduction consists in the drawing of a labeled digraph G (like the above
automaton) which resembles the structure of : The digraph G is made of k
directed paths of length n + 1 that are appended to a sink node s: The i-th
path represents the i-th clause, and the node qij represents the way in which
the variable Xi occurs in cj :
If Xi occurs positively in the clause cj ; there is a shortcut-edge from qji

to s with label 1:
If Xi occurs negatively in the clause cj ; there is a shortcut-edge from qji
to s with label 0:
If Xi does not occur in cj ; then there is not shorcut at all, and the two
edges that go out from qji are directed toward the next node.
Observe that the digraph G is synchronizing. However, if formula is not

satis…able the reset threshold of G is equal to n + 1; while if is satis…able,
the reset threshold is at most n: We get that the function
7! (G ; n)
is a polynomial time reduction of CNF-SAT into SSS. The theorem is proved.
Exercise 140 Write down a detailed description of the reduction used in the
proof of the above theorem.
References
[1] J. Cerný, A. Pirická, B. Rosenauerová. On directable automata. Kybernetika
7(4): 289 - 298, 1971.
[2] D. Eppstein. Reset Sequences for Monotonic Automata. SIAM J. Comput.
19(3): 500 - 510, 1990.
77
23 NP-completeness: Discerning I
We study the problem of separating words with …nite automata. This problem
can be regarded as “the simplest computational problem you could ask to solve”.
To begin with we suppose that there are two parties, say Alice and Bob,
which can communicate with each other using a noiseless communication chan-
nel. We also suppose that those two parties have unlimited computational
power.
n
First, we suppose that Alice has a string w 2 f0; 1g , and that she wants
to communicate this string to Bob. What is the minimum number of bits that
must be transmitted by Alice in order to achieve her goal? This is the question
that is addressed by Kolmogorov Complexity [1]. We know that the answer is
O (K (w)) ; where K (w) is the Kolmogorov complexity of string w: Recall that
K (w) is equal to the length of the smartest (shortest) program that separates
the string w:
De…nition 141 We say that a program (machine) separates the string w; if

and only if, program M accepts w and rejects any other input string.
It is known that there are strings whose Kolmogorov complexity is (almost)

equal to its length, and it is also known that there are strings (like the string
1n ) whose Kolmogorov complexity is just logarithmic in the length of those
strings. We have that K is a string measure that takes many di¤erent values
n
when restricted to f0; 1g : The latter is a feature that must be exhibited by
any non-trivial measure.
Exercise 142 Suppose for an instant that Alice is not allowed to compute smart
programs separating the string w (she cannot compute unrestricted Turing ma-
chines), and that she is restricted to compute deterministic …nite state automata.
We have that in this latter case she is obligated to communicate (n) bits: prove
that a minimal DFA separating the string w has n + 2 states.
Let us consider a second scenario. In this second scenario, Alice has a string
n n
w 2 f0; 1g , Bob has a string u 2 f0; 1g , and they must decide if those two
strings are either equal or di¤erent. What is the minimum number of bits that
must be communicated by Alice and Bob in order to achieve their common goal?
This is the question that is addressed by Communication Complexity [2]. To
begin with we notice that there is a naive solution to the problem that is being
faced by Alice and Bob: Alice communicates to Bob K (w) bits describing the
string w: However, we know that there exists nondeterministic communication
protocols that allow Alice and Bob to solve their problem by communication
very much fewer bits. Consider the following protocol:
1. Alice guesses a position i 2 f1; :::; ng :
2. Alice sends to Bob the string biw [i], where bi is the binary code of i:
78
3. Bob checks if the equality w [i] = u [i] holds.
Notice that, if the equality w = u holds, Alice and Bob will agree that those
two strings are equal. On the other hand, if w 6= u Alice can guess a position
i n for which w [i] 6= u [i] : In the latter case Alice and Bob will agree that those
two strings are di¤erent. Thus, we can conclude that the above nondeterministic
protocol computes the equality function using O (log (n)) communication bits.
It is known that the latter problem requires (log (n)) communication bits to
be correctly solved in the nondeterministic sense [2].
Let us consider a third scenario. Suppose that Alice and Bob have an un-
n
ordered pair of strings, say the strings w; u 2 f0; 1g , and suppose that Alice
chooses one of those two strings. What is the minimum number of bits that
must be communicated to Bob in order to let he knows which is the string cho-
sen by Alice? It happens that the latter question is closely related to the former
two questions:
If Alice wants to solve the latter problem e¢ ciently, then she has to compute
the smartest program separating u from w; and then she has to communicate
a succinct description of this program to Bob.
Remark 143 We say that program M separates u from w, if and only if, the
…nal states of the computations of M on those two inputs are di¤ erent states.
Observe that the above solution to the discerning problem, as we call our
problem, can be turned into a nondeterministic communication protocol that
computes the equality function using (roughly speaking) the same number of
communication bits. Consider the following protocol:
1. Alice guesses a smart program M separating u from w:

2. Alice runs M, on input w:
c (M; w), where M
3. Alice sends to Bob the string Mo c is a short description
of program M and o (M; w) is the …nal state of the computation of M
on input w:
4. Bob runs M, on input u; and he checks if the equality o (M; w) = o (M; u)
holds.
It is also easy to transform a nondeterministic protocol for the equality

function into a deterministic protocol that solves the discerning problem using
(roughly speaking) the same number of communication bits.
n
Let u; w 2 f0; 1g ; and let S (w; u) be the minimum number of bits that must
be communicated by Alice in order to solve the discerning problem for the pair
u; w: We get from the latter observation that S (u; v) 2 O (log (n)). Moreover,
n
we have that there are strings w; u 2 f0; 1g such that S (u; v) 2 (log (n)).
79
23.1 Separating Strings with Finite Automata
Suppose for an instant that Alice is not allowed to compute smart programs,
and that she is restricted to compute DFA’s. We use the symbol SDF A (u; v) to
denote the number of bits that must be communicated by Alice in this latter
scenario. We are interested in the asymptotic behavior of function SDF A .
Notice that any n-state DFA can be fully described using O (n log (n)) bits.
We can forget the logarithmic factor and focus our attention on the functions
Sep : ! N and sep : N ! N that are de…ned by
Sep (u; w) = the number of states of a minimal DFA separating u and w;

n
sep (n) = max fSep (u; w) : u; w 2 f0; 1g g :
We are interested in upper and lower bounds for function sep: Let us begin
with two naive bounds.
Lemma 144 Let f : N ! N be a function such that f (n) log (f (n)) belongs to
(log (n)), we have that:
1. sep (n) 2 (f (n)) :

2. sep (n) n + 2:
Proof. The lower bound comes from the connections between the discerning
problem and nondeterministic communication complexity that were discussed
at the end of the previous section. The upper bound is obvious.
Observe that the gap between those two bounds is exponential. Thus, there
is a lot of room for improvements. It seems that the …rst published reference on
this subject is an old paper of Goralcik and Koubek [3]. This paper includes the
…rst non-trivial upper bound for function sep : Goralcik and Koubek proved that
2 3
sep (n) 2 o (n). Robson proved, shortly after, that sep (n) 2 O n 5 log 5 (n) ,
and it is the best known upper bound for function sep [4].
Notice that the gap between those two bounds is still exponential, and notice
that those bounds have not been improved after more than thirty years. We
heard that Je¤rey Shallit o¤ered 100 pounds to the …rst person that improves
one of those bounds.
23.2 The Goralcik-Koubek Bound

It is easy to prove that sep (n) 2 (log (n)) : To do the latter one can consider
the set of GK pairs de…ned by
nn o o
GK = 1n 0n+lcm(1;:::;n) ; 1n+lcm(1;:::;n) 0n : n 1 :
Notice that the di¤erences between those strings are somewhat obvious, and
then one is tempted to conjecture that there must be pairs of strings that are
very much harder to separate. Therefore, it is a little bit frustrating to know
80
that (log (n)) is the best known lower bound for function sep; and this bound
is achieved by the GK pairs.
Can we shorten the gap between upper and lower bounds for function sep? It
could happens that one of the two bounds is sharp, either the lower bound or the
upper bound. Thus, if we want to shorten the gap between those two bounds,
we will have to carefully decide which of those two bounds can be improved.
Can complexity theory help us to take a suitable decision?
References
[1] M. Li, P. Vitanyi. An introduction to Kolmogorov Complexity and Its Ap-
plications. Springer Verlag, Berlin, 2008.
[2] E. Kushilevitz, N. Nisan. Communication Complexity. Cambridge University
Press, 1996.
[3] P. Goralcik, V. Koubek. On discerning words by automata. In L. Kott,
editor, Proc. 13th Int’l Conf. on Automata, Languages, and Programming
(ICALP), volume 226 of Lecture Notes in Computer Science, pages 116-122.
Springer-Verlag, 1986.
[4] J. Robson. Separating strings with small automata. Inform. Process. Lett,
30: 209-214, 1989.
81
24 NP-Completeness: Discerning II
Consider the algorithmic problem
Problem 145 SEP
Input: (u; v; k), where u; v are binary strings of the same length and k is
a positive integer.
Problem: decide if sep (u; v) k:
What is the algorithmic complexity of problem SEP? The latter problem

is closely related to the dfa consistency problem, which asks for the size of a
minimal automaton separating two …nite sets of binary strings. Notice that
SEP is the restriction of the latter problem to sets of size 1: It is known that
the dfa consistency problem is NP-hard [1]. Thus, it is reasonable to conjecture
that SEP is a NP-hard problem. Let us suppose that SEP is NP-hard. We have
Proposition 146 Suppose that there exists m such that sep (n) 2 O (logm (n)) ;
and suppose that SEP is NP-hard. Then, given a problem L in NP there exists
c
cL such that L can be solved in time O 2log L (n) .
Exercise 147 Prove the above proposition.
It is widely believed that the problems that are NP-hard cannot be solved
in pseudopolynomial time, that is: it is widely believed that given a NP-hard
problem L; there does not exist cL > 0; such that L can be solved in time
c
O 2log L (n) .
Exercise 148 Suppose that the above hypothesis holds, and suppose that SEP
is NP-hard, conclude that the containment sep (n) 2 (logm (n)) holds for all
m > 0.
Thus, if we prove that SEP is NP-hard, we will get enough evidence to

believe that the GK bound is weak and can be improved.
24.1 NP-hardness of SEP

We want to prove that SEP is NP-hard. Then, we will have to choose a NP-
hard problem L and reduces it to SEP. The choice of SEP is not a minor fact.
A good choice could immediately reveal a polynomial time reduction of L into
SEP, while a bad choice could lead us to believe that SEP is not so hard.
Recall that a closely related problem, the dfa consistency problem, is NP-
hard. One is tempted to believe that this problem is the right choice, and that
we should look for a reduction of this problem into the problem SEP. However,
if one checks the NP-hardness proof for this problem, she(he) will conclude that
82
it not possible to adapt this proof, which uses in a very strong way that the
sets to be separated could be very large. We claim that it is hard to …gure out
a reduction of the dfa consistency problem into the problem SEP. Is SEP NP-
hard? Yes, the problem SEP is NP-hard, and we will able to prove this result
thanks to an important connection between automata and …nite transformation
semigroups.
24.1.1 DFAs and Finite Transformation Semigroups

Let k > 0: A transformation semigroup over the set f1; :::; kg is a subset of
f1;:::;kg f1;:::;kg
f1; :::; kg that is closed under composition. The set f1; :::; kg is
itself a transformation semigroup, and it is called the full transformation semi-
group of order k: We use the symbol Tk to denote this semigroup.
De…nition 149 Let be a …nite alphabet, an interpretation of in the semi-

group Tk is a function : ! Tk that assigns to each letter in an element
of Tk : Given w 2 ; and given , we interpret w as the function
fw; = (w [n]) (w [1]) :
De…nition 150 We say that a pair of strings, say the pair (u; v) ; is an identity
for Tk , if and only if, for all interpretation
: f0; 1g ! Tk
the equality fu; = fv; holds.
Suppose that (u; v) is and identity for Tk : We have:
Given i k; the pair (u; v) it is also an identity for Ti :

If u 6= v; there exists k such that (u; v) is not an identity for Tk :
Thus, it makes sense to de…ne the function
I (u; v) = max fk : (u; v) is not an identity for Tk g ;
and it also makes sense to consider the following algorithmic problem
Problem 151 IDENT
Input: (u; v; k) ; where u; v are binary strings and k is a positive integer.

Problem: decides if I (u; v) k:
Volkov et al proved that IDENT is co-NP hard [2]. We use this result to
prove that SEP is NP-hard.
83
Remark 152 The NP-hardness of SEP is mentioned for the …rst time in [?],
where a proof of this fact is barely sketched. It seems that the authors of the
aforementioned preprint were not aware of the (conditional) implications of this
result on the asymptotic behavior of function I: they closed their paper claiming
that they have few hopes about obtaining, in the near future, superlogarithmic
lower bounds for function I:
Theorem 153 SEP is NP-hard.
Proof. Let (u; v; k) be an instance of IDENT. We only have to prove that the
latter pair is an identity of Tk , if and only if, this pair cannot be separated using
k states. The latter equivalence follows from the well studied relation between
…nite state automata an transformation semigroups [2]. Given
f0 ; f1 : f1; :::; kg ! f1; :::; kg ;
those two functions determine a binary automaton
Mf0 ;f1 = f1; :::; kg ; f0; 1g ; Mf0 ;f1 ;
whose transition function is de…ned by
Mf0 ;f1 (i; a) = fi (a) :
Notice that Mf0 ;f1 separates the pair (u; v) ; if and only if, the equality
fu; = fv; does not hold, with the interpretation given by the equalities
(0) = f0 and (1) = f1 :
We get the …rst implication: if (u; v) is an identity for Tk , then the pair
(u; v) cannot be separated with k states. On the other hand, given a n-state
automaton M; we can always suppose that QM is equal to f1; :::; kg : If we do the
latter, we get that automaton M determines an interpretation M : f0; 1g ! Tk
which is given by:
M (i) (s) = M (s; i) :
Notice that M separates the pair (u; v) ; if and only if, the equality fu; M =
fv; M
does not hold. We get the second implication, and the theorem is proved.
The NP hardness of SEP can also be obtained from a reduction of the prob-
lem SAT into the former problem. However, it seems that the most natural
of those reductions is the composition of no less than three reductions that
involve some technical problems about …nite groups and …nite transformation
semigroups.
References
[1] M. Gold. Complexity of automata identi…cation from given data. Informa-
tion and control, 378(3): 302-320, 1978.
84
[2] J. Almeida, M. Volkov, S. Goldberg. Complexity of the identity checking
problem for nite semigroups. J. Math. Sciences, 158(5): 605 - 614, 2009.
[3] A. Bulatov, O. Karpova, A. Shur, K. Startsev. Lower Bounds on Words
Separation: Are There Short Identities in Transformation Semigroups? The
Electronic Journal of Combinatorics 24(3), 2017.
85

1 - Computing Models and The Power of Writing

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1 - Computing Models and The Power of Writing

Uploaded by

Copyright:

Available Formats

Computing Models and The Power of Writing

Carolina Mejia, J. Andres Montoya

De…nition 1 A language over is a set L :

Given a language L; de…ned by some abstract property, it determines an

Remark 3 Perhaps, when the reader thinks in a computing machine, or an

Input: w 2 (f0; 1; :::; k 1g f0; 1; :::; k 1g) :

w = (k1 ; n1 ) (k2 ; n2 ) (km ; nm ) ;

Let us consider an elementary language, the language pal; constituted by all

2.1 De…nition of the Model

Let L be a language. We want to process the strings in ; and

The formal de…nition of …nite state automata goes as follows:

De…nition 5 A …nite state automaton is a tuple M = ( ; Q; q0 ; A; ) ; such

1. is a …nite set of characters called the input alphabet of M.

The computation of M, on input w; proceeds as follows:

b (q; u) = b (q; u [1; :::; n 1]) ; u [n]

It is important to observe that:

3. Finite state automata are real-time devices.

2.2 Regular Languages

De…nition 6 Let M = ( ; Q; q0 ; A; ) be a …nite state automaton, we use the

The set of regular languages is a set that contains nontrivial languages.

Exercise 9 Let L; T be two languages, we de…ne the product of L and T

L = fu1 un : n 1 & u1 ; :::; un 2 Lg :

Prove that REG is closed under stars.

2.3 The language pal is Nonregular

Theorem 11 The language pal is nonregular.

Proof. Suppose that pal is regular, and let M = (f0; 1g ; Q; q0 ; A; ) be a

Note that we get a contradiction, we get that b q0 ; uwR 2 A and hence M

Exercise 12 Let M = ( ; Q; q0 ; A; ) be a …nite state automaton, and let u; w

The theorem of Myhill-Nerode provides us with a characterization of regular

vRM w; if and only if, b (q0 ; v) = b (q0 ; w)

vX 2 L, if and only if, wX 2 L

Exercise 13 Let L be a regular language and let M be a DFA. De…ne SL as:

vX 2 L; if and only if, wX 2 L

holds. Prove that SL is an equivalence relation, which is coarser than RM ,

De…nition 14 Let L be a language, we say that v; w 2 are L-

Exercise 15 Prove that SL is an equivalence relation.

Theorem 16 Let L , we have that L is regular, if and only if, SL is a

It is easy to check that MSL accepts the language L:

Exercise 17 Homework 3. We can use Myhill-Nerode Theorem to prove of

1. Prove that the language consisting of binary representations of numbers

2. Use the theorem to give an alternative proof of the nonregularity of pal:

3.1 Minimal Automata

m (L) = min fjQj : ( ; Q; q0 ; A; ) accepts Lg :

Consider the problem

Problem 18 Automata Minimization

Input: H, where H is a …nite state automaton.

The theorem of Myhill-Nerode provides us with a construction of N . We

De…nition 19 A 2DFA is a tuple M = (Q; ; q0 ; T; A; ) that behaves analo-

A T Q: The set T is the set of halting states, which contains the

De…nition 20 Let M = (Q; ; q0 ; T; A; ) be a 2DFA, and let w 2 ; we say

4.1 The Proof

The spoiler chooses M 2 C as well as two large strings x; y:

Theorem 21 C is easy, if and only if, the containment

Proof. Suppose C is easy. Let M 2 C be an automaton with input alphabet

if Alice is given w 2 ; she communicates the string fM (w) to Bob:

If fM (w) = fM (w) , then w L(M) u:

= f[w1 ] ; :::; [wN ]g ;

Exercise 23 Homework 4. Let k 1; and let

Remark 24 We have that two-wayness without marking is useless. We also

De…nition 25 A Ink-k-2DFA is a tuple M = (Q; ; q0 ; T; A; ) that behaves

Let M be a Ink-k-2DFA. The transition function is a function from the

Theorem 26 The class Ink-1-2DFA is an easy class.