Chapter 1

Introduction to the Theory of

Computation

1
 Theory of Computation
 Can we mathematically model the computation?
 Automata Theory

 Regular languages: DFA, NFA, Regular expressions

 Context-free languages: Context-free grammar,
pushdown automata
 Turing machines: the most powerful model of
computation
 Can the computer solve all problems?
 Computability Theory

 Halting problem
 How efficient the solution is?
 Complexity Theory

 Time and space complexity (Big O notation)

2
 1.1 Mathematical Preliminaries
Sets (1)
 A set is a collection of objects
 Order and repetition is irrelevant
 X = {1, 2, 3}
 X = {n | 1  n  3, n  N }
 Y = { } =   Y is an empty set

 The power set of X is the set of all subsets of X

 (X) = 2X = { Y | Y  X}
 |(X)| = 2|X|
 Example X = {1, 2, 3}
 2X = (X)={{},{1},{2},{3},{1,2},{1,3},{2,3}, {1,2,3}}
 {1,2}  X is a proper subset of X
 {1,2,3}  X is NOT a proper subset of X
 {1,2,3} is a superset of {1, 2}
3
 Sets (2)
 The  symbol signifies membership (belong to):
 x  X indicates that x is a member or element
of the set X
 x  X indicates that x is not a member or
element of the set X
 Assume X = {1, 2, 3}, then
 1  X, but 4  X
 Two sets are equal if they contain the same
members (elements)
 If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4}
 Then X = Y, but X  Z, and so Y  Z

4
 Sets (3)
 Assume X = {1,2,3} and Y = {2,4}

 Union
 X  Y = {z | z  X or z Y} = {1,2,3,4}

 Intersection
 X  Y = {z | z  X and z Y} = {2}

 Difference
 X - Y = {z | z  X and z  Y} = {1,3}

5
 Sets (4)
 Complement
 Let X be a subset of Y
 The complement of X ( X ) with respect to Y is
the set of elements in Y but not in X
 X = { z | z  Y and z  X } = Y - X
 Assume X = {1,2,3} is a subset of Y = {1,2,3,4}
 X = {4}
 DeMorgan’s Laws

 (X  Y) = X  Y

 (X  Y) = X  Y
6
 Inductive Proofs
 Prove a statement S(X) about a family of objects X
(e.g., integers, trees) in three parts:
 Basis Step:
 Prove for one or several small values of X directly
 Assumption step :
 Assume S(Y) for Y smaller than X
 Prove S(X) using that assumption

7
 Inductive Proofs (EX1)
 Prove by induction that
 S: 1 + 2 + 3 + ……+ n = n (n + 1) / 2
 Basis step:
 n=1

 1 ( 1 + 1) / 2 = 1
 Assumption step:
 Assume that S is true for all series of length < n
 Prove that S is true for a series of length n?
 The series of length n is

 1+2+3+…+n-1+n
 By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2
 Add n to the summation proves S.
8
 Inductive Proofs (EX2)
 Example
 A complete binary tree with n leaves has 2n - 1
nodes

9
 Inductive Proofs (EX2)
 Basis:
 If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so OK
 Assumption:
 Assume that the statement is true for all trees that have leaves
less than n
 Proof: Proof that the statement is true for trees that have n leaves
 T has two subtrees, U and V
 U has u leaves and V has v leaves
 According to the assumption: U has (2u – 1) of nodes and
V has (2v – 1) of nodes
 T must be a root plus two subtrees U and V. The number of
leaves in T equal number of leaves in U + number of leaves
in V = u + v
 The number of nodes in T = (2u – 1) + (2v – 1) + 1 = 2 ( u +
v) - 1

10
 If-And-Only-If Proofs
 Often, a statement we need to prove is of the form
 “X if and only if Y”
 We are then required to do two things:
 Prove the if-part: Assume Y and prove X
 Prove the only-if-part: Assume X, prove Y
 Remember:
 The if and only-if parts are converses of each
other
 One part, say “if X then Y” says nothing about
whether Y is true when X is false.
 An equivalent form to “if X then Y” is “if not Y
then not X”; the latter is the contra-positive of the
former.
11
 Equivalence of Sets
 Many important facts in language theory are of the
form that two sets of strings, described in two
different ways, are really the same set
 To prove sets S and T are the same, prove:
 x is in S if and only if x is in T. That is:

 Assume x is in S; prove x is in T .
 Assume x is in T ; prove x is in S.

12
 Functions
 Let f be a function defined on a set A and taking values to
a set B. Set A is called the domain of f; set B is called the
range of f.
 The arity of the functions
 The number and type of input arguments the number
and type of output
 Examples:
 The arity of the Multiplication: Num × Num  Num
 The arity of the SquareRoot : Num  Num

13
 1.2: Strings and Languages (1)
 Alphabet: Finite, nonempty set of symbols
 Examples:

  = {0, 1}: binary alphabet

  = {a, b, c, …, z}: the set of all lower case letters
 The set of all ASCII characters

 String: Finite sequence of symbols from an

alphabet 
 Examples:

 01101 where  = {0, 1}

 abracadabra where  = {a, b, c, …, z}

14
 1.2: Strings and Languages (2)
 Empty String: The string with zero occurrences of
symbols from  and is denoted or 

 Length of String: Number of symbols in the string

 The length of a string w is usually written |w|
 |1010| = 4
 || = 0
 |uv| = |u| + |v|

 Reverse : wR
 If w = abc, wR = cba

15
 1.2: Strings and Languages (3)
 Concatenation: if x and y are strings, then xy is the
string obtained by placing a copy of y immediately
after a copy of x
 x = a a …a , y = b b …b
1 2 i 1 2 j
 xy = a a …a b b …b
1 2 i 1 2 j
 Example: x = 01101, y = 110, xy = 01101110
 x = x = x

16
 1.2: Strings and Languages (4)
 Power of an Alphabet: k = the set of strings of length k
with symbols from 
 Example:  = {0, 1}
 1 =  = {0, 1}
 2 = {00, 01, 10, 11}
 0 = 

 Question: How many strings are there in 3?

 The set of all strings over  is denoted *

 * = 0  1  2  3  …
 Also
 + = 1  2  3  …
 * = +  
 + = * - 

17
 1.2: Strings and Languages (5)
 Substring: any string of consecutive characters in
some string w
 If w = abc
 , a, ab, abc are substrings of w

 Prefix and suffix:

 if w = vu
 v is a prefix of w
 u is a suffix of w
 Example
 If w = abc
 a, ab , abc are prefixes of w
 c, bc, abc are suffixes of w

18
 1.2: Strings and Languages (6)
 Suppose: S is the string banana
 Prefix : ban, banana
 Suffix : ana, banana
 Substring : nan, ban, ana, banana

19
 1.2: Strings and Languages (8)
 Language: set of strings chosen from some alphabet
 A language is a subset of *
 Example of languages:

The set of valid Arabic words

The set of strings consisting of n 0’s followed by n 1’s
– {, 01, 0011, 000111, …}
 The set of strings with equal number of 0’s and 1’s
– {, 01, 10, 0011, 0101, 1010, 1001, 1100, …}
 Empty language:  = { }
 The language  consisting of the empty string
 Note:  

20
 1.2: Strings and Languages (9)
 Can concatenate languages
 L L = {xy | x L , y L }
1 2 1 2
 Ln = L concatenated with itself n times
 L0 = {}; L1 = L

 Star-closure
 L* = L0 L1 L2 
 L+ = L* - L0

 Languages can be finite or infinite

 L = {a, aba, bba}
 L = {an | n > 0}

21
 1.2: Strings and Languages (10)

OPERATION DEFINITION
union of L and M L  M = {s | s is in L or s is in M}
written L  M
concatenation of L LM = {st | s is in L and t is in M}
and M written LM
Kleene closure of L L* =  Li , i=0,.., 
written L*
L* denotes “zero or more concatenations of “ L
positive closure of
L written L+ L+ =  Li, i=1,.., 
L+ denotes “one or more concatenations of “ L
L+ = LL*

22
 1.2: Strings and Languages (11)

L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9}

L  D = the set of letters and digits
LD = all strings consisting of a letter followed by a digit
L2 = the set of all two-letter strings
L4 = L2 L2 = the set of all four-letter strings
L* = { All possible strings of L plus  }, L+ = L* - 
D+ = set of strings of one or more digits
L (L  D ) = set of all strings consisting of a letter
followed by a a letter or a digit
L (L  D )* = set of all strings consisting of letters and
digits beginning with a letter
23
 1.2: Strings and Languages (12)
 The language L consists of strings over {a,b} in
which each string begins with an a should have an
even length

 aa, ab  L
 aaaa,aaab,aaba,aabb,abaa,abab,abba,abbb  L

 baa  L
 aL

24
 1.2: Strings and Languages (13)
 The language L consists of strings over {a,b} in
which each occurring of b is immediately
preceded by an a

 L
 aL
 abaab  L

 bb  L
 bab  L
 abb  L

25
 1.2: Strings and Languages (14)
 Let X = {a,b,c} and Y = {abb, ba}. Then
 XY = {aabb, babb, cabb, aba, bba, cba}
 X0 = {}
 X1 = X = {a,b,c}
 X2 = XX = {aa,ab,ac,ba,bb,bc,ca,cb,cc}
 X3 = XXX =
{aaa,aab,aac,aba,abb,abc,aca,acb,acc,baa,bab,ba
c,bba,bbb,bbc,bca,bcb,bcc,caa,cab,cac,cba,cbb,
cbc,cca,ccb,ccc}

26
 1.2: Strings and Languages (15)
 The language L = {a,b}*{bb}{a,b}* = *{bb}*
 consists of the strings over {a,b} that contain
the substring bb
 bb  L
 abb  L
 bbb  L
 aabb  L
 bbaaa  L
 bbabba  L
 abab  L
 bab  L
 bL

27
 1.2: Strings and Languages (16)
 Let L be the language that consists of all strings that
begin with aa or end with bb
 L = {aa}{a,b}*
1
 L = {a,b}*{bb}
2
 L = L  L = {aa}{a,b}*  {a,b}*{bb}
1 2
 bb  L
 abb  L
 bbb  L
 aabb  L
 bbaaa  L
 bbabba  L
 abab  L
 bab  L
 ba  L

28
 1.2: Strings and Languages (16)
 Let L1 = {bb} and L2 = {
{ , bb, bbbb} over b

 The languages L1* and L2* both contain precisely

the strings consisting of an even number of b’s.

  , with length zero, is an element of L1* and L2*

29
 1.2: Strings and Languages (17)
 What is the language of all even-length strings over
{a,b}
 L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)*

 What is the language of all odd-length strings over

{a,b}
 L = {a,b}* - {aa, bb, ab, ba}* or
 L = {a,b}{aa, bb, ab, ba}* or
 L = {aa, bb, ab, ba}* {a,b}

30