CT111-3-2

Computing Theory

Introduction
 Contact

• Lecturer: Prof. Dr. R. Logeswaran

• Email: Logeswaran@apu.edu.my
Important: APU Student Email Policy -
students must only use their official APU
email account for correspondence.

• Consultation: email to check availability.

CT111-3-2- Computing Theory Introduction
 Learning Outcomes

LO1: Design automata and generate regular

expressions for specified languages.
LO2: Design and analyse context-free
grammars for specific syntactic rules.
LO3: Describe and Design Turing machines
to accomplish specified computations.
LO4: Explain and discuss complexity and
decidability of problems.

CT111-3-2- Computing Theory Introduction

 Assessment

• COURSEWORK (50%) (LO1, 2 and 4):

– Individual assignment to design different types of
FSM and grammars, convert between equivalently
powerful notations for a language, determine string
membership to a language, and research about
complexity and decidability problems and their
significance.
• EXAM (50%) (LO1,2 and 3):
– Written examination to demonstrate understanding
automata, regular expressions, context free
grammars, and the Turing machines.

CT111-3-2- Computing Theory Introduction

 Main References

• Hopcroft, J.E, Motwani, R., Ullman, J.D. (2007),

Introduction to Automata Theory, Languages, and
Computation (3rd Edition). Pearson (ISBN: 978-
0321455369)
• Sipser, M. (2012). Introduction to the Theory of
Computation (3rd Edition). Course Technology (ISBN:
978-1133187790)
• Maheshwari, A., Smid, M. (2014), Introduction to Theory
of Computation. Carleton University, Ottowa CA. e-book
available at:
http://cglab.ca/~michiel/TheoryOfComputation/TheoryOf
Computation.pdf
•
CT111-3-2- Computing Theory Introduction
 Computation

• Computation is a general term for any type of

information processing that can be represented
as an algorithm precisely. (mathematically)
Examples:
• Adding two numbers in our brain, on a piece of
paper or using a calculator.
• Converting a decimal number to its binary
presentation or vise versa.

CT111-3-2- Computing Theory Introduction

 What is computable?
• Examples:
– check if a number n is prime
– compute the product of two numbers
– sort a list of numbers
– find the maximum number from a list

• Hard but computable:

– Given a set of linear inequalities, maximize a linear
function
Eg. maximize 5x+2y
3x+2y < 53
x < 32
5x – 9y > 22
 Computing Theory
Primary aim of the course: What is “computation”?
• Can we define computation without referring to a modern computer?

• Can we define, mathematically, a computer?

(yes, Turing machines)

• Is computation definable independent of present-day engineering

limitations, understanding of physics, etc.?

• Can a computer solveany problem, given enough time and disk-space?

Or are they fundamental limits to computation?

In short, understand the mathematics of computation

 Computing Theory

-What can be computed?

-Can a computer solve any problem,
Computability
given enough time and disk-space?

- How fast can we solve a problem?

Complexity - How little disk-space can we use
to solve a problem

-What problems can we solve given

Automata really very little space?
(constant space)
 Computing Theory
What problems can a computer solve?
Not all problems!!!

Computability Eg. Given a C-program, we cannot

check if it will not crash!

Verification of correctness of programs

is hence impossible!
Complexity
(The woe of Microsoft!)

Automata
 Computing Theory
What problems can a computer solve?

Even checking whether a C-program will

Computability halt/terminate is not possible!

input n;
assume n>1; No one knows
Complexity while (n !=1) { whether this
if (n is even) terminates on
n := n/2; on all inputs!
else
n := 3*n+1;
Automata }

17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1.

 Computing Theory

How fast can we compute a function?

Computability How much space do we require?

• Polynomial time computable

• Non-det Poly Time (NP)
• Approximation, Randomization
Complexity
Functions that cannot be computed fast:
• Applications to security
• Encrypt fast,
Automata • Decryption cannot be done fast

• RSA cryptography,
web applications
 Computing Theory
Machines with finite memory:--
traffic signals, vending machines
Computability hardware circuits

Tractable.
Applications to pattern matching,
Complexity modeling,
verification of hardware, etc.

Automata
 Computing Theory

What can we compute?

-- Most general notions of computability
Computability -- Uncomputable functions

What can we compute fast?

-- Faster algorithms, polynomial time
Complexity
-- Problems that cannot be solved fast:
* Cryptography

What can we compute with very little space?

Automata -- Constant space (+stack)
* String searching, language parsing,
hardware verification, etc.
N
I Computing Theory
C
R Turing machines (1940s):
E -- The most general notion of computing
A -- The Church-Turing thesis
S
Turing machines
-- Limits to computing:
I Uncomputable functions
N
G Context-free languages
Context-free --- Grammars, parsing
C --- Finite state machines with recursion (or stack)
O . languages --- Still a simple setting; but infinite state
M
P
Automata:
L
Automata --- Foundations of computing
E
--- Mathematical methods of argument
X
--- Simple setting
I
T
Y
 Alonzo Church

First notions of computable functions

• First language for programs

• -- lambda calculus
• -- formal algebraic language
• for computable functions
Alonzo Church:
1903 - 1995
 Alan Turing
• “Father of computer science”
• Defined the first formal notion of a
computer (Turing machine) in 1936:
Proved uncomputable functions exist
(“halting problem”)
• Church-Turing thesis: all real
world computable functions are
Alan Turing: 1912 - 1954
Turing m/c computable
• Cryptanalysis work breaking
Enigma in WW-II
 Noam Chomsky
• Introduced the notion of formal
languages arguing generative grammars
are at the base of natural languages
• Hierarchy of formal languages that
coincides with computation
• Eg. Context-free grammars capture
most skeletons of prog. languages
Noam Chomsky: 1928-

“Logical Structure of Linguistic Theory” (1957)

 Rabin and Scott
• Automata: machines with
finite memory

• “Finite Automata and Their

Decision Problem”
- Rabin and Scott (1959)

• Introduced nondeterministic automata

and the formalism we still use today

• Initial motivation: modeling circuits

• Turing Award (1976)
N
I Theory of Computation
C
R
E Turing: 1931
A
S
Turing machines
I
N
G
Context-free Chomsky: 1957
C
O . languages
M
P
L
E Automata Rabin-Scott: 1959
X
I
T
Y
 A result you would know at the end...

• Proving that it is impossible to check if

a C program will halt.

• Formal proof!
You can convince a friend using a paper-
argument

• No computer *ever* will solve this

problem (not even a quantum computer)
 Textbook
Michael Sipser
Introduction to Theory
of Computation
(2nded; 1st ed may be ok)

Hopcroft-Ullman
 How to do well…

• This is essentially a math course:

– you must learn the concepts well; if you don’t there’s
almost no chance of success
– if you do learn the concepts, there is very little else (facts,
etc.) to learn; you can do really well!
– You must do problems. There’s no replacement for this.
– Attending lectures is highly adviced!
• It will be very hard to learn the concepts by yourself / from textbook.
– Don’t postpone learning; you will not be able to “make up”
later. Topics get difficult quickly.
– Come regularly to discussion sections; you will learn a lot
by working out problems and learning from fellow students