Theory of

Automata
Why Study Automata?
What the Course is About?

1
Automata – plural of “automaton”

A moving mechanical device made in

imitation of a human being.
"a collection of 19th century French automata:
acrobats, clowns, and musicians“

A machine which performs a range of

functions according to a predetermined
set of coded instructions.
“Sophisticated automatons continue to run factory
assembly lines"
2
 What is the study of Automata Theory?

The study of abstract computing devices, or “machines.”

Days before digital computers

What is possible to compute with an abstract machine
Seminal work by Alan Turing (1940s)

Why is this useful?

Direct application to creating compilers, programming languages,
designing applications.
Formal framework to analyze new types of computing devices, e.g.
biocomputers or quantum computers.
Develop mathematically mature computer scientists capable of
precise and formal reasoning!
 Why Study Automata?
A survey of Stanford grads 5 years out
asked which of their courses did they
use in their job.
Basics like Computer Programming took
the top spots, of course.
But among optional courses, Theory of
Automata stood remarkably high.
3X the score for AI, for example.
4
 How Could That Be?
Regular expressions are used in many
systems.
E.g., UNIX a.*b.
E.g., DTD’s describe XML tags with a RE
format like person (name, addr, child*).
Finite automata model protocols,
electronic circuits.
Theory is used in model-checking.
5
 How? – (2)
Context-free grammars are used to
describe the syntax of essentially every
programming language.
Not to forget their important role in
describing natural languages.
And DTD’s taken as a whole, are really
CFG’s.

6
 How? – (3)
When developing solutions to real
problems, we often confront the
limitations of what software can do.
Undecidable things – no program
whatever can do it.
Intractable things – there are programs,
but no fast programs.
ToA gives you the tools.
7
 Other Good Stuff in ToA
We’ll learn how to deal formally with
discrete systems.
Proofs: You never really prove a program
correct, but you need to be thinking of why
a tricky technique really works.
We’ll gain experience with abstract
models and constructions.
Models layered software architectures.
8
Outline of the course contents

Computation

CPU memory

Department of Computer Science & 9

Engineering
temporary memory

input
CPU
output

Program memory

10
 Example: f ( x) = x 3

temporary memory

input
CPU
output
Program memory
compute xx
compute x x
2
11
 f ( x) = x 3

temporary memory
input
x=2
CPU
output
Program memory
compute xx
compute x x
2
12
 temporary memory f ( x) = x 3
z = 2*2 = 4
f ( x) = z * 2 = 8
input
x=2
CPU
output
Program memory
compute xx
compute x x
2
13
 temporary memory f ( x) = x 3
z = 2*2 = 4
f ( x) = z * 2 = 8
input
x=2
CPU
f ( x) = 8
Program memory output
compute xx
compute x x
2
14
 Automaton
temporary memory

Automaton
input
CPU
output

Program memory

15
 Automaton
temporary memory

Automaton
input

output
transition

state

16
 Different Kinds of Automata
Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

17
 Finite Automaton

temporary memory

input
Finite
Automaton
output

Example: Elevators, Vending Machines

(small computing power)
18
 Pushdown Automaton
Temp.
memory Stack Push, Pop

Pushdown input

Automaton
output

Example: Compilers for Programming Languages

(medium computing power) 19
 Turing Machine

Temp.
memory Random Access Memory

input
Turing
Machine
output

Examples: Any Algorithm

(highest computing power) 20
 Power of Automata
Simple More complex Hardest
problems problems problems

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
21
Turing Machine is the most powerful
computational model known

Question: Are there computational

problems that a Turing Machine
cannot solve?

Answer: Yes (unsolvable problems)

22
Time Complexity of Computational Problems:

NP-complete problems
Believed to take exponential
time to be solved

P problems
Solved in polynomial time

23
 Course Outline
Regular Languages and their
descriptors:
Finite automata, nondeterministic finite
automata, regular expressions.
Algorithms to decide questions about
regular languages, e.g., is it empty?
Closure properties of regular languages.

24
 Course Outline – (2)
Context-free languages and their
descriptors:
Context-free grammars, pushdown
automata.
Decision and closure properties.

25
 Course Outline – (3)
Recursive and recursively enumerable
languages.
Turing machines, decidability of problems.
The limit of what can be computed.
Intractable problems.
Problems that (appear to) require
exponential time.
NP-completeness and beyond.
26
 Course Outline
Finite State Models Push Down Automata
◼ Language definitions preliminaries ◼ Making a Parser
◼ Regular expressions/Regular ◼ Pumping lemma and non-context
languages free languages
◼ Finite automata (FAs) ◼ Decidability
◼ Transition graphs (TGs) ◼ Chomsky’s hierarchy of grammars
◼ NFAs, kleene’s theorem
◼ Making a Scanner Turning Machines Theory
◼ Transducers (automata with output) ◼ Turing machines
◼ Pumping lemma and non regular ◼ Post machine
language
◼ Turing Machine project
Grammars and PDA ◼ Variations on TM
◼ Context free grammars ◼ TM encoding, Universal Turing
◼ Derivations, derivation trees and Machine
ambiguity ◼ Context sensitive Grammars
◼ Operations preserving CFLs ◼ Defining Computers by TMs
◼ Normal form grammars and parsing
Aug 15, 2016 Theory of Automata 27
 Text
Hopcroft, Motwani, Ullman, Automata
Theory, Languages, and Computation
3rd Edition.
Course covers essentially the entire
book.

28
 Text and Reference Material
1. Introduction to Computer Theory, by
Daniel I. Cohen, John Wiley and Sons,
Inc., 1991, Second Edition
2. Introduction to Languages and Theory of
Computation, by J. C. Martin, McGraw
Hill Book Co., 1997, Second Edition
3. Class Lecture Slides & Handouts
4. Class Room Sessions & Discussions
Theory of Automata 29
Google Classroom

rd5yf6d
30
Comments About Gradiance
Homework
The intent is that everyone will get
100% on all homeworks.
You are allowed to try as many times as
you like.
Only the last try counts.
Don’t be afraid to guess and try again.
You’ll get some advice if you make a
mistake.
31
 Tentative Grading
Mid-Term Exams (2) 25
Assignments 10
Class Activities
Quizzes 10
Project 10
Final Exams 40
Seminar (optional) 05
Marks division might change during the
semester
32
 Objectives
The course aims to develop an
appreciation of theoretical foundations of
computer science through study of
mathematical & abstract models of
computers and theory of formal languages

33
 Background
We shall study several mathematical models that
will describe with varying degrees of accuracy parts
of computers, types of computers, and similar
machines.
In particular, the way we shall be studying about
computers is to build mathematical models, called
machines, and then to study their limitations by
analyzing the types of inputs on which they can
operate successfully.
The collection of these successful inputs is called the
language of the machine.
34
 Every time we introduce a new machine, we will learn
its language; and every time we develop a new
language, we will try to find a machine that
corresponds to it. (Language vis-à-vis Machine)
 This interplay between languages and machines will
be our way of investigating problems and their
potential solutions by automatic procedures, which we
call algorithms.
 We will study different types of theoretical machines
that are mathematical models for actual physical
processes. By considering the possible inputs on
which these machines can work, we can analyze their
various strengths and weaknesses.
35
 What is mathematical about the
models?
We are obliged to prove the truth about whatever we
discover.
So consider only question of whether certain tasks
can be done at all. Conclusion are of the form “this
can be done” or “this can never be done”.
When we reach conclusions of the second type, we
mean not just that techniques for performing these
tasks are unknown at the present time, but that
such techniques will never exist in the future.

36
We will arrive at what we may believe to be the
most powerful machine possible. When we do, we
will be surprised to find tasks that even such
machine cannot perform.
Our ultimate result is that no matter what machine
we build, there will always be questions that are
simple to state and that the machine can not
answer.
Limitation of a machine / algorithm / mathematical
model / computer
Vision/hear/smell/touch - emotions/intuition

37