Professional Documents
Culture Documents
Introduction To Theory of Computation
Introduction To Theory of Computation
Introduction To Theory of Computation
Course Objectives
1. To introduce the concepts and methods that underlie the formal (mathematical) study
of computing machines
– What is a computing machine?
– How can we characterise and classify computing machines?
2. To present some of the basic results concerning the capabilities and limits of
computing machines
3. To extend basic mathematical skills and to develop further logical and analytical skills
directly related to Computer Science.
Theory of computation is a branch of mathematics and computer science that deals with
whether and how efficiently problems can be solved on a model of computation using an
algorithm.
The first two questions can only be addressed after the last question is addressed, that is, how
to represent computation in forms that admit rigorous analysis and not merely execution.
3. This course helps you to learn problem solving skills. Theory teaches you how to think,
prove, argue, solve problems, express yourself clearly and precisely, and abstract.
How do we model computation?
Automata theory
Is concerned with the definitions and properties of mathematical models of computation
Unlike other models (SE, DB, DAA), computation models deal with all computers that exist,
will exist and that can ever be dreamed of.
Note that computational models may be accurate in some ways and not in other ways.
One model, called the finite automaton, is used in text processing, compilers, and hardware
design.
Another model, called the context – free grammar, is used in programming languages and
artificial intelligence.
Languages
The fact that our study is sometimes called theory of formal languages makes it imperative to
study languages. The word formal means that all the rules for the language are explicitly
stated in terms of what strings of symbols can occur. The other reason why we study
languages is that, languages are used to model computation. It has already been indicated that
TOC deals with asking the question, how do we model computation? And it has been
indicated that computation is modelled using languages and machines.
A language is defined as a game of symbols with formal rules. Natural languages like English
are made up of letters, words, sentences, paragraphs etc. Similarly, with computer languages,
certain character strings are recognisable as words (END, DO, WHILE etc), certain strings of
words are recognisable as commands and certain sets of commands become a program that
can be translated into machine commands.
Terminology
Are members of an alphabet and usually denoted by small letters but numbers can be part
of the symbols.
3. Words
Are strings containing the symbols in some alphabet. Two words are considered the same
if all their characters are the same and in the same order.
4. String over an alphabet- is a finite sequence of symbols from that alphabet written
adjacent to one another and not separated by commas. For example if Σ = {a,b,c,d} then
aadc, cdabb, and adcd are strings over Σ.
5. Empty string or null string
Is a string of length zero and is denoted by (Λ or ε). For clarity, the symbol Λ or ε is not
allowed to be part of the alphabet for any language.
6. String length (|x| or length(x))
Refers to the number of symbols in a string. For any word x in any language, if length(x)
= 0 then x = Λ
7. Language
Is a certain specified set of strings of characters from an alphabet. Is denoted by L.
8. Empty language (Φ)
Is a language that has no words or strings.
Points of thought
is there a difference between Φ and Λ (language without words and word without
symbols)
is L + Λ = L (+ is the union of sets operation)
is L + Φ = L
Answers
1. There is a subtle but important difference between the word that has no letters and the
language that has no words. It is false that Λ is a word in the language Φ since this
language has no words at all.
2. If a language L does not contain Λ, then L + Λ is not the same as L
3. L + Φ = L since no new words were added
Cohen (1991) posits that, anyone who thinks that Λ is not confusing has missed
something. It is already a problem and it gets worse latter.
Defining Languages
There are two types of language defining rules:
1. Can be used to test whether a word is valid
2. Used to construct all the words in the language by some clear procedures
Concatenation operation
Is used to join two or more strings and a concatenation is a string obtained by appending one
string to the end of another. For example L1 = {good} and L2 = {one}, L1 + L2 = {goodone}
Reverse function
If c is a word in some language L, then reverse(c) is the same string of letters spelled
backward, called the reverse of c even if this backward string is not a word in L. Example
reverse(eert) = tree
Palindrome
Assume a new language Palindrome is defined over the alphabet, Σ = {m, n} then Palindrome
= {Λ, and all strings y such that reverse(y) = y} so words in Palindrome are: {Λ, m, n,
mm, nn, mnm, nmn, mmm...}
Note that if you concatenate two words in Palindrome, the obtained word is sometimes in
Palindrome.
Valid words
If a word is contained in a given language it is valid otherwise it is invalid
Question:
Given the following languages:
L1= pxqyrx+y, where x and y range over all the natural numbers, 0,1,2... and p x denotes
the string containing x successive copies of the symbol p
L2= pxqyrx-y, where x and y range over all the natural numbers, 0,1,2... and p x denotes
the string containing x successive copies of the symbol p and x > y For each language, list 5
valid and invalid words.
The Kleene star is an operation that makes an infinite language of strings of letters out of an
alphabet. The term infinite language means, infinitely many words each of a finite length.
Lexicographic ordering
Means that strings must be arranged in size order (words of shortest length first) and words of
the same length must be put alphabetically. For example if Σ = {0, 1} then Σ * = {Λ, 0, 1, 00,
01, 10, 11, 000, 001, 010, 100, 110, 111 }
Example 1:
if S = {00, 1} then
Example 2:
if S = {x, xy} then
S* = { Λ plus any word composed of factors of x and xy} = { Λ plus all strings of x’s
and y’s except those that start with y and those that contain a double y}
= { Λ, x, xx, xy, xxx, xxy, xyx, xxxx, xxxy, xxyx, xyxx, xyxy, xxxxx, xxyxx, xxyxy,
xyxxx, xyxxy, xyxyx, ...}
For example if S = {aa, aaa} then S* = { Λ plus all strings of more than one a} or {an for n
= 0, 2, 3, 4, 5, ...} or { Λ, aa, aaa, aaaa, aaaaa, aaaaaa, ...} prove whether aaaaaaa is in S*.
The factors are:
(aa) (aaa) (aa) or (aaa) (aa) (aa) or (aa) (aa) (aaa).
Using example 1, prove whether or not the string 1000011001110001 is in the closure of S.
Given that S= {aa, aaa}, prove that S* contains all an for n ≠1.
We proceed as follows;
1. Assume that there are some powers of a we could not produce by concatenating
factors of (aa) and (aaa). Since we can produce a4, a5and a6 then strings that we
cannot produce must be large.
2. Determine the smallest power of a (> 1) that we cannot form out of factors of (aa) and
(aaa). Assume here that we start making a list of how to construct the various powers
of a. On this list we state how to form a,2 a 3 , a4, a5, a6 and so on. Assume that we
work our way successfully up to an-1 but then we cannot figure out how to form an
3. Establish how an-2 was formed and then concatenate another factor of aa in front of
this and then you will have an.
If Σ = {} then Σ* = {Λ} this is not the same as, if S = {Λ}, then S* ={Λ}which is also true but
for a different reason that is Λ= Λ Λ.
Sometimes the notation + instead of * is used to modify the concept of closure to refer to
only the concatenation of some (not zero) strings from a set S. If Σ = {a} then Σ+ = {a, aa,
aaa, aaaa, ...}
For any language S* = S+ + Λ if S does not contain Λ.
|Λ| = 0
Theorem 1
This is analogous to saying that if computers are made up of circuits and circuits are
made up of logic gates then computers are made up of logic gates.
Proof
Every word in S** is made up of factors from S*. Every factor from S* is made up of factors
from S. Therefore, every word in S ** is also a word in S*. This can be expressed as S** ⊂ S*. It
can be generalised that for any set A we know that A⊂A*, since in A* we can choose as a
word any one factor from A. So if we consider A to be our set S *, we have S* ⊂ S**.
Together the two inclusions prove that S** = S*.
2. Regular expressions
3. Finite automata
4. Transition Graph
5. Other
Recursive definition
2. Give rules for combining more objects in the set from the ones we already know
3. Declare that no objects except those constructed in this way are allowed in the set.
Examples:
Rule 1: 2 is in EVEN
Rule 1: 1 is in INTEGERS
Rule 1: 1 is in INTEGERS
Rule 2: if both x and y are in INTEGERS, then so are x+y and x-y
A polynomial is a finite set of terms, each of which is in the form: a real number times
a power of x (that may be x0=1).
Rule 3: if p and q are in POLYNOMIAL then so are p+q, p-q, pq and (p)
By rule 1, 2 is in POLYNOMIAL
By rule 2, x is in POLYNOMIAL
By rule 1, 3 is in POLYNOMIAL
REGULAR EXPRESSIONS
• Cohen (2001) defines REs as language defining symbols whereas Sipser (1996) defines
them as expressions describing languages.
• Languages defined by REs are referred to as Regular Languages
• REs are limited in capacity because there are some languages that cannot be defined by
REs
• A RL is one that can be defined by a RE
• The value of a RE is a language.
• Symbols that appear in REs include letters of the alphabet Σ, the symbol of the null string
Λ, the symbol for the empty language Φ, parenthesis, the star operator and the plus sign.
• The set of regular expressions is defined as follows:
o Rule 1: every letter of the alphabet Σ can be made into a regular expression by
writing it in bold face; Λ itself is a RE and so is Φ.
o Rule 2: if r1 and r2 are REs then so are:
(r1)
r1r2 -- concatenation (°) -- r1r2 = {𝑤 : 𝑤 = 𝑥𝑦, 𝑥 ∈ r1, 𝑦 ∈ r2}
Example: If 𝐴 = {good, bad} and 𝐵 = {boy, girl} we get
𝐴 ∘ 𝐵 = {goodboy, goodgirl, badboy, badgirl}.
Prepared by Zanamwe N Page 9 of 13
Page 10 of 13
Note that:
• L(a) = {a}
• L(a+b) =
• L(ab) =
• L(r*) = L(r*)
• Language of Λ = L(Λ) = Λ
Examples:
1.
2.
14. L(a*b*)- all strings over a and b in which all a’s (if any) precede all
b’s (also if any)
15. L(a*b*)* - all strings over a and b
16. L(a(a+b)*a +b(a+b)*b + a + b – all strings over a and b that start and
end with the same symbol
Page 11 of 13
Page 12 of 13
Note that:
• L(a*Φ)=Φ- concatenating an empty language to any non-empty language yields
an empty language.
• r + Φ = r- adding the empty language to any other language will not change it
• r =r – concatenating an empty or null string to any string will not change it.
But
• r+ may not equal r for example if r=b then L(r) = {b} but L(r+ ) ={b, }
• rΦ may not equal r if r=b, then L(r) ={b} but L(rΦ) = Φ // accepts L(r) when r=Φ
Notice that the use of the plus sign is far from the normal meaning of addition in the algebraic
sense, for plus as union or plus as choice the following all make sense.
b*=b* + b*
b*=b* + b* + b*
b* = b* + bbb
Also note that in algebra but in formal languages ac ca and also that (ab)* a*b*
Below are rules that define the language associated with any regular expression:
1. Rule 1: that language that is associated with the regular expression that is just a single
letter is that one-letter word alone and the language associated with is just { }, a
one word language.
2. Rule 2: if r1 is a regular expression associated with the language L1 and r2 is the
language associated with the language L2, then:
a. The regular expression (r1)(r2) is associated with product L1L2 that is the
language L1 times L2:
i. Language(r1)(r2) =L1L2
b. The regular expression r1 + r2 is associated with the language formed by the
union of the sets L1 and L2:
i. Language(r1+r2) =L1+L2
c. The language associated with the regular expression (r 1)* is L1*, the Kleene
closure of the set L1 as a set of words:
i. Language(r1*) = L1*
• Is there an algorithm for determining whether different REs describe the same
language?
• Is it true that every language can be described by a regular expression?
Page 12 of 13
Page 13 of 13
Theorem: if L is a finite language (a language with only finitely many words), then L can be
defined by a regular expression. In other words all finite languages are regular.
Proof: To make one RE that defines the language L, convert all words in L into bold face
type and insert plus signs between them. For example, the RE that defines the language
L = {a, aa, aaa, aaaa} the algorithm above gives the RE a + aa + aaa + aaaa
This trick only works for finite languages because with infinite languages the RE will be
infinitely long which is forbidden.
Page 13 of 13