Back

Course : Computation Theory

Introduction 1
Pramod Sethy

4th Semester 2021

 Back

outlines

• Syllabus
• Why study the theory of computing?
• Automata Theory.
• Historical Perspective.
• Languages & Grammars.
• The Chomsky Hierarchy.
• Examples.
• According to Chomsky Hierarchy.
Computation Theory Books Back
 Books
Back

Introduction to the Theory of

Computation 3rd Edition by Sipser

Gain a clear understanding

of even the most complex,
highly theoretical
computational theory topics
in the approachable
presentation

Introduction.ppt
 Books
Back

An Introduction to Formal Languages

and Automata 6th Edition
by Peter Linz
The author, Peter Linz,
continues to offer a
straightforward, uncomplicated
treatment of formal languages
and automata and avoids
excessive mathematical detail
allowing students to focus on
and understand the key
underlying principles.
 Books
Back

Introduction to Automata Theory,

Languages, and Computation (3rd
Edition)

This classic book on formal

languages, automata theory,
and computational complexity
has been updated to present
theoretical concepts in a
concise and straightforward
manner with the increase of
hands-on, practical
applications.
Gradiance 
 Back

Why study the theory of computing?

Core mathematics of CS (has not changed in

over 40 years)
Many applications, especially in design of
compilers and programming languages
Important to be able to recognize
uncomputable and intractable problems
Need to know this in order to be a computer
scientist, not simply a computer programmer
 Back

What is Automata Theory?

• Study of abstract computing devices, or
“machines”
• Automaton = an abstract computing device
– Note: A “device” need not even be a physical
hardware!
• A fundamental question in computer science:
– Find out what different models of machines can do
and cannot do
– The theory of computation
• Computability vs. Complexity

8
 Back

Historical Perspective.
• The history of computer science began long before our
modern discipline of computer science, usually appearing
in forms like mathematics or physics. Developments in
previous centuries alluded to the discipline that we now
know as computer science.

• This progression, from mechanical inventions and

mathematical theories towards modern computer
concepts and machines, led to the development of a
major academic field, massive technological advancement
across the Western world, and the basis of a massive
worldwide trade and culture.

9
History: Alchemy, Fiction and Hoaxes

• ~800: Jabir ibn Hayyan’s Takwin

• 1206: Al-Jazari’s programmable
automata
• ~1500: Paracelsus’ homunculus
• ~1580: Rabbi Judah Loew ben
Bezalel of Prague’s Golem
• 1769: Wolfgang von Kempelen’s
The Turk
• 1818: Mary Shelley’s
Frankenstein
• 1920: Karel Čapek’s Rossum’s
Universal Robots (R.U.R.)
10
 History: Formal Reasoning
• 1275: Ramon Llull’s Ars Magna
• Early 17th century: René
Descartes: bodies of animals are
nothing more than complex
machines (but not mind or soul)
• 1641: Thomas Hobbes in
Leviathan: “reason is nothing but
reckoning”
• 1672: Gottfried Leibniz:
– Stepped Reckoner
– Characteristica universalis
11
 History: Formal Reasoning
• 1854: George Boole’s Boolean algebra
– “Investigate the fundamental laws of those
operations of the mind by which reasoning is
performed, to give expression to them in the
symbolic language of a calculus”
• 1913: Bertrand Russell and Alfred North
Whitehead publish Principia Mathematica
• 1920s and 1930s:
– David Hilbert’s program
– Alonzo Church’s lambda calculus
– The Turing machine
– Kurt Gödel’s incompleteness theorems
• Within certain limits, any mathematical
reasoning could be mechanized

12
 Back

Alan Turing (1912 –1954)

• Father of Modern Computer
Science

• English mathematician

• Studied abstract machines called

Turing machines even before
computers existed

• Heard of the Turing test?

13
 Back

Church–Turing thesis
• In 1933, Kurt Gödel: general
recursive functions.
• In 1936, Alonzo Church created a
method for defining functions called
the λ-calculus.
• in 1936, Alan Turing created a
theoretical model for machines, now
called Turing machines,
• Church and Turing proved that these
three formally defined classes of
computable functions coincide: a
function is λ-computable if and only if
it is Turing computable, and if and
only if it is general recursive.
14
 Back

Stephen Cole Kleene

• One of the students of Alonzo
Church.
• is best known as a founder of the
branch of mathematical logic known
as recursion theory, which
subsequently helped to provide the
foundations of theoretical computer
science.
• He also invented regular
expressions in 1951 to describe
McCulloch-Pitts neural networks, and
made significant contributions to the
foundations of mathematical
intuitionism.
15
 Theory of Computation: A
Back

Historical Perspective
1930s • Alan Turing studies Turing machines
• Decidability
• Halting problem
1940-1950s • “Finite automata” machines studied
• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages
1969 Cook introduces “intractable” problems
or “NP-Hard” problems
1970- Modern computer science: compilers,
16
computational & complexity theory evolve
 Back

Languages & Grammars

• Languages: “A language is a
Or “words” collection of sentences of
finite length all constructed
from a finite alphabet of
symbols”
• Grammars: “A grammar can
be regarded as a device that
enumerates the sentences of
a language” - nothing more,
nothing less

• N. Chomsky, Information and

Control, Vol 2, 1959
Image source: Nowak et al. 17
Nature, vol 417, 2002
 Back

The Chomsky Hierarchy

• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) free Context-
Recursively-
(PDA) sensitive
enumerable
(LBA)
(TM)

18
 Back

Automata theory
Automata theory studies the laws of computation.

In reality, the laws of computation are not quite understood,

but automata theory is a good start.
 Back

A simple computer
I TCH
SW

BATTERY

input: switch
output: light bulb
actions: flip switch
states: on, off
 Back
Back

A simple “computer”
I TCH
SW

BATTERY start off on

input: switch
bulb is on if and only if
output: light bulb
there was an odd number of
actions: f for “flip switch” flips
states: on, off
 Back

Another “computer”
1
1 start off off
1

2 2 2 2
BATTERY
1
2
off on
1

inputs: switches 1 and 2

bulb is on if and only if
actions: 1 for “flip switch 1”
both switches were flipped
actions: 2 for “flip switch 2”
an odd number of times
states: on, off
 Back

A design problem

1 4

=
5
BATTERY

2
3

Design a circuit where the light is on only when all

switches are flipped the same number of times
 Back

A design problem

• Such devices are difficult to reason about,

because they can be designed in an infinite
number of ways
f

off on

• By representing them as abstract

computational devices, or automata, we will
learn how to answer such questions
 Back

These devices can model many

things
• They can describe the operation of any
“small computer”, like the control component
of an alarm clock or a microwave
• They are also used in lexical analyzers to
recognize well formed expressions in
programming languages:

ab1 is a legal name of a variable in C

5u= is not
 Back

A gumball machine
machine takes $5 and $10 coins
a gumball costs $15
actions: +5, +10, Release
一五 +10 +10

+5 +5 +5, +10
$0 $5 $10
R R R +5, +10

R
 Different kinds of machines
Back

+10 +10

+5 +5 +5, +10
$0 $5 $10
R R R +5, +10

R
• This was only one example of a machine

• We will look at different kinds of machines and ask:

– What kinds of problems can this kind of machine solve?
– What things are impossible for this kind of machine?
– Is machine A more powerful than machine B?
 Some kinds of machines Back

finite automata Devices with a small amount of memory.

Used to model very simple things.

push-down Devices with infinite memory that can be

automata accessed in a restricted way.
Used to parse grammars

Turing Machines Devices with infinite memory.

These are the real computers

time-bounded Infinite memory, but bounded running

Turing Machines time.
These are computers that run reasonably
fast.

In our Course: related to Chomsky hierarchy
Type-3 Regular Languages and
Regular Grammars:
Type-2  Context-Free Languages (CFL)
Context-Free Grammars (CFG)
Type-1 Context-Sensitive Language ,
Context-Sensitive Grammar
Type-0  Devices with infinite
memory ( These are the real
computers)
Back
1-Deterministic Finite automata
(DFA).
2- Non-Deterministic Finite
automata (NFA),
3- Regular Expressions,
4- Regular Grammars.
Push-down automata
(Which is non-deterministic )
Context-Free Grammars
linear bounded automaton (a
nondeterministic Turing machine
Turing Machines
(Which is Deterministic)
 Back

Some highlights of the course

• Finite automata
– Automata are closely related to the task of
searching for patterns in text
find (ab)*(ab) in abracadabra
–
• Grammars
– Grammars describe the meaning of sentences in
English or Arabic, and the meaning of programs in Java
– We will see how to extract the meaning out of a program
 Back

Some highlights of the course

• Turing Machines
– This is a general model of a computer, capturing anything
we could ever hope to compute
– But there are many things that computers cannot do:

#include <stdio.h>
main(t,_,a)char *a;{return!
0<t?t<3?main(-79,-13,a+main(- banana
Given the code of a 87,1-_,

?
main(-86,0,a+1)+a)):1,t<_?
main(t+1,_,a):3,main(-94,-

computer program, can 27+t,a)&&t==2?_<13?

main(2,_+1,"%s %d
%d\n"):9:16:t<0?t<-72?

you main(_,t,
"@n'+,#'/*{}w+/w#cdnr/+,
{}r/*de}+,/*{*+,/w{%

tell if the program prints +,/w#q#n+,/#{l,+,/n{n+,/

+#n+,/#\
;#q#n+,/+k#;*+,/'r :'d*'3,}

the string “ banana” ? {w+K w'K:'+}e#';dq#'l \

q#'+d'K#!/
+k#;q#'r}eKK#}w'r}eKK{nl]'/#;#
q#n'){)#}w'){){nl]'/+#n';d}rw'
i;# \
 Back

Some highlights of the course

• Time-bounded Turing Machines
– Many problems are possible to solve on a computer in
principle, but take too much time in practice
– Traveling salesman: Given a list of cities, find the
shortest way to visit them and come back home
Madinah

Badr
Yanbu
Makah
Rabigh
Jeddah

– Hard in practice: For 100 cities, this would take 100+

years even on the fastest computer!
 Back

Preliminaries of automata
theory
• How do we ask the question
Can machine A solve problem B?

• First, we need a way of describing the

problems that we are interested in solving
 Back

Problems
• Examples of problems we will consider
– Given a word s, does it contain “to” as a subword?
– Given a number n, is it divisible by 7?
– Given two words s and t, are they the same?

• All of these have “yes/no” answers.

• There are other types of problems, like “Find this”
or “How many of that” but we wont look at them.
 Back

From Discrete Math

You have to know some

Mathematical Preliminaries
• Sets
• Functions
• Relations
• Graphs
• Proof Techniques

35