Theory of computation (IT 010 404) Course File Nov 12 May13 MODULE: I

Lecture Notes -Module : I

INTRODUCTION TO THEORY OF COMPUTATION MODULE DESCRIPTION This module describes the introduction to theory of computation At the end of the module students will be able to: Understand proving techniques and solve application problems LEARNING OUTCOME Understand different types of functions based on recursion and computability Understand formal languages and Chomsky Classification TOPICS
Sl. No. TOPIC TIME REQUIRED 1.5 hr 1.5 hr LECTURE #

01 02

Mathematical Preliminaries Set Theory, Relations, Functions Equinumerous sets, countable sets, infinite sets

LN 01 LN 02 LN 03 LN 04 LN 05 LN 06 LN 07 LN 08 LN 09

03 Proving techniques - Mathematical induction 04 Proving techniques - Diagonalization principle

05 06

02 hr 01 hr 01 hr 01 hr 02 hr 02 hr 02 hr

Proving techniques - Pigeonhole principle

Functions Primitive recursive and partial recursive functions 07 Computable and non computable functions 08 Formal representation of languages 09 Chomsky Classification REFERENCES 1.

Michael Sipser, Introduction to the Theory of Computation, Cengage Learning,New Delhi,2007 Peter Linz, An Introduction to Formal Languages and Automata ,Fourth Edition, 02. Narosa Kamala Krithivasan, Rama R, Introduction to Formal Languages,Automata 03. Theory and Computation, Pearson Education Asia,2009 04. S. P. Eugene Xavier, Theory of Automata Formal Language & Computation,New Age International, New Delhi ,2004

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 TEST ITEMS

Lecture Notes -Module : I

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 LECTURE # 01

Lecture Notes -Module : I

MATHEMATICAL PRELIMINARIES SET THEORY, RELATIONS, FUNCTIONS OBJECTIVE To revise the mathematical preliminaries of set theory, relations and functions EXPLANATION 1. Sets any collection of objects (individuals)

Notation: If a is a member of A, we write a A.If b is not a member of A, we write b A Describing sets a. by enumerating the elements of A for finite sets: {red, blue, yellow}, {1,2,3,4,5,6,7,8,9,0} for infinite sets we write: {1,2,3,4,5,.} b. by a formulating rule (recursive definition) Example: defining the set of all positive integers (natural numbers): 1. 1 N 2. if a N then a + 1 N 3. no other elements belong to N c. by property, using predicate logic notation (setbuilder notation) Let P(x) be a property, D - universe of discourse . The set of all objects in D, for which P(x) is true, is : A = {x : P(x)} We read: A consists of all elements x such that P(x) is true) Example: The set of all female students can be described in the following way: Let female(x) be true if x is a female, student(x) be true if x is a student. Then A = {x: female(x) and student(x)}

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 Universal set: U - the set of all objects under consideration Empty set: - the set without elements. 2. Subsets

Lecture Notes -Module : I

A is a subset of B (A B) if all elements of A belong to B. However B may contain elements that do not belong to A A B if and only if x ( x A x B) Example: A = {2,4,6}, B = {1,2,3,4,5,6}, A B Proper subsets: A is a proper subset of B, A B, if and only if A is a subset of B and there is at least one element in B that is not in A. The empty set is a subset of all sets.All sets are subsets of the universal set U. 3. Operations on sets a. Intersections Let A and B be two sets. The set of all elements common to A and B is called the intersection of A and B Formal definition: A B = {x : (x A ) (x B)} (The symbol denotes conjunction and is pronounced "and".) Example: A = {2,4,6}, B = {1,2,5,6}, A B = {2,6} Other properties: A B A, A B B, the intersection of two sets A and B is a subset of A, and a subset of B A = ,

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 the intersection of any set A with the empty set is the empty set A U = A,

Lecture Notes -Module : I

the intersection of any set A with the universal set is the set A itself. Intersection corresponds to conjunction in logic. b. Unions The union of two sets A and B consists of all elements that belong to A or B. An element may belong both to A and B A B = {x : (x A) V (x B)} The union of A and B is the set of all elements x such that x belongs to A B.The symbol V denotes disjunction and is pronounced "or". Example: A = {2, 4, 6}, B = {1, 2, 3, 4}, A B = {1, 2, 3, 4, 6} Other properties: A A B, B A B, or x belongs to

A is a subset of the union of A and B, B is a subset of the union of A and B A = A, the union of any set A with the empty set is A A U = U, the union of any set A with the universal set U is the universal set. Union corresponds to disjunction in logic. c. Differences Let A and B be two sets. The set A B, called the difference of A and B, is the set of all elements that belong to A and do not belong to B.

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 Notation: A B or A \ B

Lecture Notes -Module : I

Formal definition: A B = { x : ( x A) ( x B)}

Example: A = {2, 4, 6}, B = {1, 5, 6}, A B = {2, 4} Basic set identities A = A, the difference of A and the empty set is A A U = , the difference of A and the universal set is the empty set. U A = ~A , ~A is called the complement of set A, sometimes denoted as AC A ~A = , the intersection of any set A and its complement is the empty set. A ~A = U, the union of any set with its complement is the universal set.

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

A ~A = U A ~A = AU=A A=A AU=U A= AA=A AA=A AB=BA AB=BA (A B) C = A ( B C) (A B) C = A ( B C) A (B C) = (A B) (A C) A (B C) = (A B) (A C) (A B) A = A (A B) A = A ~(A B) = ~A ~B ~(A B) = ~A ~B A - (B C) = (A - B) (A - C) A - (B C) = (A - B) (A - C) A - B = A ~B

Complementation Law Exclusion Law Identity Laws

Domination Laws

Idempotent Laws

Commutative Laws

Associative Laws

Distributive Laws

Absorption Laws

De Morgan's Laws

Alternate representation of set difference Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

1.Disjoint sets, Cartesian product, Power sets, Partitions 1.1. Union and intersection of more than two sets Let S be a collection of sets, i.e. a set of sets S = {S1, S2, . Sn,} Union of all sets in S is denoted by US = S1 S2 S3 Sn Intersection of all sets in S is denoted by S =S1 S2 S3 Sn

1.2. Disjoint sets Sets that have empty intersection are called disjoint sets. Two sets S1 and S2 are disjoint sets iff S1 S2 = 1.3. Cartesian product of two sets A and B The Cartesian product of A and B is defined as the set Example: A = {a,b,c}, B = {1,2} A x B = {(x,y) : x A y B}

A x B = {(a,1), (a,2), (b,1), (b,2), (c,1), (c,2)

Similarly, we can build the Cartesian product of three or more sets: A x B x C = {(x,y,z): x A y B z C} 1.4. Power set of a set A The set of all subsets of A is called power set of A. The power set of A is denoted by 2A Example: A - { a,b,c,d} 2A = { , {a},{b},{c},{d},{a,b},{a,c},{a,d},{b,c},{b,d},{c,d},{a,b,c},{a,b,d},{a,c,d},{a,b,c,d}}

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

Let A be a set. |A| is the number of the elements in A. Note that |2A| = 2|A| 1.5. Partition of a nonempty set A A partition of a nonempty set A is a nonempty set that is a subset of the power set of A, such that a. , i.e. each element of is nonempty; b. distinct members of are disjoint; c. U = A. i.e. each element of the partition is nonempty, each element of A belongs exactly to one element of the partition, and the union of all elements of the partition is A. Example: Let A = {a, b, c, d}. The following sets are partitions of A: {{a}, {b, c, d}} {{a}, {b, c}, {d}} {{a, b}, {c}, {d}} The following sets are not partitions of A: {{a}, {b, c}} - element d does not belong to any set. {{a, b}, {a, c, d}} - element a belongs to two sets. Relations Let A and B be two sets. A relation R from A to B is any set of ordered pairs (x,y), x A, y B. Thus R A x B, i.e. R is a set defined as a subset of the Cartesian product of A and B. The notation xRy is used to denote that two elements x and y are in relation R. Let R be a relation from A to B. The inverse relation R-1 of R is defined as: R-1 = {(b,a) : (a,b) R} Thus if xRy, then yR-1x, i.e. if x and y are in relation R, then y and x are in the inverse relation R-1 .

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 Functions

Lecture Notes -Module : I

A function from a set A to a set B is a relation R with the property: Each element of A is a member of only one ordered pair of the relation. Notation: Let f be a function from A to B. We write: f : A B, f(a) = b , a A, b B Definitions: Let f be a function from A to B. 1. 2. 3. Domain of f: the set A Range of f: {b: b B and there is an a A, f(a) = b} Image of a under f: f(a)

Types of Functions a.one-to-one: distinct elements have distinct images if a1 a2, then f(a1) f(a2) a.onto:Every element in B is an image if some element in A b. bijection: f is bijection iff f is a one-to-one function and f is a onto function

If f is a bijection, f-1 is a function, also a bijection. Otherwise f-1 fails to be a function. Properties: f-1(f(a)) = a, f(f-1 (b)) = b, a A, b B,

Examples of relations from a set A to a set B Relation, not a function: Let A = {a,b,c}, B = {1,2,3}

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

The relation R = {(a,1),(a,2),(b,3),(c,3)} is not a function, because the element a has two images. Function, not one-to-one, not onto: Let A = {a,b,c}, B = {1,2,3} The relation R = {(a,1),(b,1),(c,3)} is a function.

The function is not one-to-one because two distinct elements in A, a and b, have

one and the same image in B. The function is not onto, because the element 2 in B is not an image under the function. Function, one-to-one, not onto

Let A = {a,b,c}, B = {1,2,3,4,5} The relation R = {(a,1),(b,2),(c,5)} is a function.

The function is one-to-one because every two distinct elements in A have distinct images in B. The function is not onto, because the element 3 and 4 in B are not images of elements in A under the function. Function, onto, not one-to-one Let A = {a,b,c,d}, B = {1,2,3} The relation R = {(a,1),(b,1),(c,2),(d,3)} is a function. The function is not one-to-one because two distinct elements in A, a and b, have one and the same image in B. The function is onto, because each element in B is an image of some element in A Function, bijection: Let A = {a,b,c}, B = {1,2,3} The relation R = {(a,1),(b,2),(c,3)} is a function.

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

The function is one-to-one because every two distinct elements in A have distinct images in B. The function is onto, because each element in B is an image of some element in A. Hence the function is a bijection. Its inverse function is also a bijective function. Binary Relations Binary relation: a relation between two sets: aRb, a A, b B, Binary relations on a set and itself: aRb, a A, b A Binary relations on a set can be represented by a directed graph: Let A = {a,b,c,d,e,f} R = {(a,b),(a,d),(b,a),(b,b),(b,f),(c,d)}

The set of graph vertices is {a,b,c,d,e,f}, the set of edges is {(a,b),(a,d),(b,a),(b,b),(b,f),(c,d)} Definition: Let Q and R be two binary relations. The composition Q R (or just QR) is defined as: Q R = {(a,b): for some c, (a,c) Q and (c,b) R}. Composition of functions: Let f : A B, g : B C be two functions. The composition h = f g is a function from A to C such that h(a) = g(f(a)). Example: Let f (x) = x +1, g(x) = x2. The composition h(x) = f (x) g(x) is the function g(f(x)) = (x+1)2 The composition p(x) = g(x) f(x) is the function f(g(x)) = x2 + 1 Properties of binary relations

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 Reflexive relations: For all a A, (a,a) R Irreflexive: For all a A, (a,a) R neither:

Lecture Notes -Module : I

There is a such that (a,a) R, There is b such that (b,b) is not in R

Symmetric relations For all (a,b) R, a b, (b,a) R

Antisymmetric For all (a,b) R, a b, (b,a) R

neither: There is (a,b) R, a b, such that (b,a) R, There is (c,d) R, c d, such that (d,c) is not in R

Transitive relations: If (a,b) R and (b,c) R, then (a,c) R

Not transitive relations: There are a, b, c such that (a,b) and (b,c) are in R, however (a,c) is not in R

Relations of Equivalence: R is a relation of equivalence iff the relation is reflexive, symmetric and transitive A relation of equivalence specifies a partition of the set. Each element in the partition is called a class of equivalence. Example: R = "having same parents" defines a partition on the set of all persons Partial orders: R is a partial order relation iff the relation is reflexive, antisymmetric and transitive Example: R = "being a subset of a set" defines partial order on powersets. A = {a,b,c} 2A = { ,{a}, {b}, {c},{a,b}, {a,c}, {b,c},{a,b,c} }

Minimal element: a, such that for no b, (b,a) R In the above example is the minimal element. Maximal element: b, such that for no a (b,a) R

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 In the above example {a, b, c} is the maximal element

Lecture Notes -Module : I

Total order: a partial order relation that is defined for any two elements in the set. Example: R = "being less or equal to" defines a total order on the set of real numbers. LECTURE # 02 EQUINUMEROUS SETS, COUNTABLE SETS AND INFINITE SETS OBJECTIVE To learn about finite and infinite sets, countable and equinumerous sets EXPLANATION Finite and infinite sets Finite sets - finite number of elements - can be counted Infinite sets - infinite number of elements Equinumerous sets: Sets A and B are equinumerous if there is a bijection f : A B E.G.: A = {1,2,3}, B = {a,b,c} Countably infinite sets: A set is countably infinite if it is equinumerous with N (the set of natural numbers) Countable sets Finite or countably infinite sets are countable.Sets that are not countable, are called uncountable. Examples of countably infinite sets are: 1. 2. 3. The set of all squares - countably infinite A U B U C, A = {a1,a2,a3, .} B = {b1,b2,b3,.}, C = {c1,c2,c3} NxN

Proving countability/uncountability of sets

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 1. Finding a bijection to the set of natural numbers

Lecture Notes -Module : I

By definition, if a set is countably infinite, there is a bijection between the set in consideration and the natural numbers (the set N). This means that the elements of the set can be ordered - we can map each element uniquely to a natural number. 2. Applying the dovetailing technique If it is difficult to represent the bijection explicitly as a function, we can prove that a set is countably infinite by showing a method to order the elements in the set so that if we follow the method of ordering we can visit each element in the set. One such method widely used is the dovetailing technique.The basic idea is to put the elements of the set in a table, following a specific rule. The table may have infinite number of rows, or infinite number of columns, or it may be infinite in both directions. LECTURE # 03 PROVING TECHNIQUES - MATHEMATICAL INDUCTION OBJECTIVE To explain the principle of mathematical induction EXPLANATION The induction principle is often used when we are trying to prove that some claim holds for all positive integers. When we use induction we do not need to explicitly refer to the set A from the statement of the induction principle. Generally, this set is the set of numbers for which the claim that we are trying to prove holds. Suppose we want to show that some claim holds for all positive integers. A general template for proving this by induction: (a) State the method of proof. For example, The proof proceeds by induction. (b) Prove the induction basis. That is, prove that the number 1 satisfies the claim. (c) Assume the induction hypothesis. That is, state the assumption that the claim holds for some positive integer k. (d) Prove, using the induction hypothesis, that the claim holds for k+1. The proof should consist of a chain of clear statements, each logically following from the previous ones combined with our shared knowledge base. The final statement in the chain should state that the claim holds for k + 1. (e) Conclude the proof. For example, This completes the proof by induction. 1.Prove that all natural numbers of the form n3 +2n are divisible by 3 using principle of induction. Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 P (n) = n3 +2n Induction base: P (0) = 0 Induction hypothesis: P (k) = k3 +2k Induction step: P (k+1) = (k+1)3 +2(k+1) = (k3 +3k2 +3k+1) + (2k+2) = (k3+2k) + (3k2 +3k+3) = (k3+2k) + 3 (k2 +k+1) which is a multiple of 3.
This completes the proof by induction.

Lecture Notes -Module : I

LECTURE # 04 PROVING TECHNIQUES DIAGONALIZATION PRINCIPLE OBJECTIVE To explain diagonalization principle EXPLANATION The diagonalization principle is based on a simple observation.Let A be a finite set, and R be a binary relation on A. We can represent the relation by a square table, rows and columns representing the elements, and cells having 1s if there is a link between the corresponding elements. For example, if A = {a,b,c,d}, and R = {(a,b), (a,c), (b,b), (b,d), (c,b), (c,d), (d,a)}, the table would be: a a b c d 0 0 0 1 b 1 1 1 0 c 1 0 0 0 d 0 1 1 0

The principle says that the complement of the diagonal is different from each row. The reversed diagonal in the example is 1,0,1,1 and it is different from each row. Explanation: The reversed diagonal differs from the first row in the first element (we have taken the complement in the diagonal), it differs from the second row in the second element, etc. Application of the diagonalization principle to prove uncountability of sets.

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13 Basic idea:

Lecture Notes -Module : I

In order to apply the principle the elements of the set have to be represented as infinite sequences of 0 and 1, and any infinite sequence of 0 and 1 has to be a representation of some element in the set. Assume that we can order the elements of the set in some way. The binary representation will result in an infinite table, where each row will correspond to an element in the set. Take the complement of the diagonal - it is a representation of some element in the set, so it should appear somewhere among the rows of the table. It differs from the first row in the first element, it differs from the second row in the second element, etc, and hence it is not equal to any row in the table. This contradicts the assumption that all elements can be ordered, and each element corresponds to a row in the table. Hence the set is uncountable. Example 1: The set 2N is uncountable Proof: Let 2N = {R1,R2, } is the power set of N, it contains all possible subsets of N. We can represent each subset as a sequence of 0s and 1s, where the ith position is 1 if i is in the subset, and 0 otherwise. Let us assume that 2N is countable. This means that there is a bijection between 2N and N, so we can order the elements of 2N accordingly. In the ordered sequence we represent each Rk by means of 0s and 1s. Thus we obtain an infinite table filled with 0s and 1s. Consider now the reverse diagonal D = { d1,d2,.} of that table. It is a sequence of 0s and 1s, so it corresponds to some set of natural numbers D = {n : dn = 1} This set obviously is a subset of N, so it should be in the power set of N, and should appear somewhere in the ordered elements of 2 N , i.e. should be equal to some row in our table. D cannot be the first row as it differs from the first row in the first position. It cannot be also equal to the second row as it differs there in the second position, etc. So D cannot be anywhere among the rows of the table. However we have assumed that the table contains all possible subsets of N. This is a contradiction, following from our assumption, that the elements of 2Ncan be ordered. Hence 2N is uncountable.

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

Example 2: The set of all real numbers in the interval [0,1] is uncountable. Real numbers in [0,1] that are not rational, have infinite decimal expansions. We can assume that rational numbers also have infinite expansions consisting of zeros after the rightmost non-zero digit. Decimal digits have binary representations, so we can represent the numbers by binary sequences. Let us assume that the numbers can be ordered.The ordering will give an infinite table consisting of 0 and 1. The complement of the diagonal of this table is an infinite sequence of 0 and 1, and as such it represents a real number that has to be somewhere in the ordering. But this cannot be the first number, since the complement of the diagonal differs from the first row in the first position. Similarly, it cannot be the second number, it cannot be the third number, etc. Thus the number represented by the complement of the diagonal does not occur in the ordering. This contradicts the assumption that the numbers can be ordered. Therefore the set of all real numbers in [0,1] is uncountable. LECTURE # 05 PROVING TECHNIQUES PIGEONHOLE PRINCIPLE OBJECTIVE To understand pigeonhole principle. EXPLANATION If we put more than n pigeons into n pigeonholes, at least one pigeonhole will have two or more pigeons. Basic form: If m objects are put in n boxes and n < m, then at least one box contains at least two objects. The one-line proof is by contradiction: If every box contains at most one object, there are at most n 1 = n objects. A more rigorous formulation of the principle is as follows: Given two sets A and B, with |A| = m > n = |B|, for any function f : AB there exists b B such that |{x A : f(x) = b}|> 1. General form: If m objects are put in n boxes, then at least one box contains at least [m/n] objects. The proof is again by contradiction: If every box contains atmost [m/n] - 1 < m/n objects, there are less than n(m/n) = m objects. The more rigorous formulation is: Given two sets A and B, for any function f : AB there exists b B such that |{x A : f(x) = b}| [m/n] LECTURE # 06 PRIMITIVE RECURSIVE AND PARTIAL RECURSIVE FUNCTIONS OBJECTIVE To explain about primitive recursive functions and partial recursive functions EXPLANATION Primitive recursive functions and Partial recursive functions

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

The class of primitive recursive functions is defined in terms of base functions and closure operations. Let = {a1, . . . , aN}. The base functions over are the following functions: (1) The erase function E, defined such that E(w) =, for all w *; (2) For every j, 1 j N, the j-successor function Sj , defined such that Sj(w) = waj , for all w *; (3) The projection functions Pni , defined such that Pni (w1, . . . , wn) = wi, for every n 1, every i, 1 i n, and for all w1, . . . , wn *.. P11 is the identity function on *. Projection functions can be used to permute the arguments of another function. A crucial closure operation is (extended) composition. Let = {a1, . . . , aN}. For any function g: * * * m and any m functions * hi : *
*

n the composition of g and the hi is the function * f: *

*

n denoted as g o (h1, . . . , hm), such that f(w1, . . . , wn) = g(h1(w1, . . . , wn), . . . , hm(w1, . . . , wn)), for all w1, . . . , wn *. As an example, f = g o (P22 , P21 ) is such that f(w1, w2) = g(w2, w1). Another crucial closure operation is primitive recursion. Let = {a1, . . . , aN}. For any function g: * * * m-1 where m 2, and any N functions hi: * * * m+1 the function f: * * * m is defined by primitive recursion from g and h1, . . . , hN, if f(, w2, . . . , wm) = g(w2, . . . , wm), f(ua1, w2, . . . , wm) = h1(u, f (u, w2, . . . , wm), w2, . . . , wm), ...=... f(uaN, w2, . . . , wm) = hN(u, f(u, w2, . . . , wm), w2, . . . , wm), for all u,w2, . . . , wm *..

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

When m = 1, for some fixed w *, we have f() = w, f(ua1) = h1(u, f(u)), ...=... f(uaN) = hN(u, f(u)), for all u * For numerical functions (i.e., when = {a1}), the scheme of primitive recursion is simpler: f(0, x2, . . . , xm) = g(x2, . . . , xm), f(x + 1, x2, . . . , xm) = h1(x, f(x, x2, . . . , xm), x2, . . . , xm), for all x, x2, . . . , xm N. The successor function S is the function S(x) = x + 1. Addition, multiplication, exponentiation, and super-exponentiation can be defined by primitive recursion as follows : add(0, n) = n, add(m + 1, n) = S(add(m, n)), mult(0, n) = 0, mult(m + 1, n) = add(mult(m, n), n), rexp(0,m) = 1, rexp(m + 1, n) = mult(rexp(m, n), n), exp(m, n) = rexp . (P22 , P21 ), supexp(0, n) = 1, supexp(m + 1, n) = exp(n, supexp(m, n)). As an example over {a, b}*, the following function g: * * * is defined by primitive recursion: g(, v) = P11 (v), g(uai, v) = Si o P32 (u, g(u, v), v), where 1 i N. It is easily verified that g(u, v) = vu. Then, f = g o (P22 , P21 ) computes the concatenation function, i.e. f(u, v) = uv. Let = {a1, . . . , aN}. The class of primitive recursive functions is the smallest class of functions (over *) which contains the base functions and is closed under composition and primitive recursion. Every primitive recursive function is a total function. For an alphabet = {a1, . . . , aN}, every primitive recursive function is Turing computable. Partial Recursive Functions The operation of minimization (sometimes called minimalization) is defined as follows. Let = {a1, . . . , aN}. For any function g: * * * m+1 the function f: * * * Department of Information Technology where m 0, for every j, 1 j N,

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

m is defined by minimization over {aj}* from g, if the following conditions hold for all w1, . . . , wm *: (1) f(w1, . . . , wm) is defined iff there is some n 0 such that g(ap j, w1, . . . , wm) is defined for all p, 0 p n, and g(anj , w1, . . . , wm) = . (2) When f(w1, . . . , wm) is defined, f(w1, . . . , wm) = anj where n is such that g(anj , w1, . . . , wm) = and g(apj , w1, . . . , wm) for every p, 0 p n -1. We also write f(w1, . . . , wm) = minju[g(u, w1, . . . , wm) = ]. LECTURE # 07 COMPUTABLE AND NON COMPUTABLE FUNCTIONS OBJECTIVE To introduce theory of computation To learn about computable and non computable functions EXPLANATION Theory of computation is a course of abstractions about what we can compute. It is the purest of all computer science topics attempting to strip away any details about the real computer and replacing in it with abstractions that give a hierarchy of capabilities culminating in a Turing Machine, the abstract model of our modern computer, invented by Alan Turing. Abstractions give us a way of saying something about what we are doing in a rigorous formal way, rather than flailing randomly through a haze of informality. There is great value in abstraction. Applications Finite state machines are used in string searching algorithms, compiler design, control unit design in computer architecture, and many other modeling applications. Context free grammars and their restricted forms are the basis of compilers and parsing. NP-Complete theory helps us distinguish the tractable from the intractable. Overview A problem is a set of strings (often binary strings) that we wish to distinguish. For example, one problem might be the set of all binary strings divisible by three. To solve this problem we need to be able to say yes or no to each candidate binary string as to whether it is in the set or not. If we have an algorithm to do this, then we have solved the problem. These kinds of problems are called decidable.

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

More complicated inputs over larger alphabets can be considered as well. For example, we can consider lists of numbers as the input, where each list has numbers written in binary with a # symbol separating each one from the other. A problem is the set of all such strings where the numbers are in ascending order. Consider the set of all syntactic legal Java programs that never go to into an infinite loop on any input. There is no way to solve this problem. This problem is undecidable. Consider a machine, or automaton, as a guard at the door of an exclusive club that admits only specific kinds of binary strings. The machine is given a program by the boss. It looks at each candidate string trying to enter the club, and it executes the program by looking at deciding for each one: yes or no. Computability and Non-computability A function or a (decision type) problem is said to be computable, if there exist a Turing machine which computes the function or answer the problem. [i.e., the TM will halt on all the inputs and gives the correct output for all the input].Otherwise, the function is said to be non-computable. There exist Turing Machines to compute the initial functions i. Z (x) = 0, for all x N (the zero function) ii. S (x) = x+1, for all x N (the successor function) iii. Uin(x1, x2, x3, , xi, ..., xn) = xi (the projection function) There also exist Turing machines to compute composition, recursion and minimization operations. As every partial function is obtained from the initial functions by a finite number of applications of the operations of composition, recursion and minimization, thus, there exist Turing machine that can compute every partial recursive function. Thus, every partial recursive function is computable. On the other hand it can be proved that every Turing computable function is partial recursive. Here are some examples of non-computable problems. "Halting problem" for Turing machine is one of the popular examples for noncomputability of Turing machines. This is the problem of finding a decision procedure which would enable one to determine in a finite number of steps, given any Turing machine T and premarked tape 't', whether or not the machine will ever make d(sj, aj) = HALT. Equivalently the problem is to construct a Turing machine, which will accept just those pairs (T, t) that will come to HALT, and reject just those pairs (T, t) that will not come to HALT. It can be proved theoretically that there

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

cannot exist any Turing machine, which will perform this job. Thus, the "HALTING PROBLEM IS NONCOMPUTABLE". Other interesting example of non computable functions are the following: For any programming language, to determine whether or not: (a) a given program that can loop for ever on some input, (b) a given program that can ever produce an output on given input, (c) a given program that eventually halts on the given input. LECTURE # 08 FORMAL REPRESENTATION OF LANGUAGES OBJECTIVE To introduce formal representation of grammar and the basic terminologies EXPLANATION Formal representation of languages Language Symbol An atomic unit, such as a digit, character, lower-case letter, etc. Sometimes word. Alphabet A finite set of symbols, usually denoted by . = {0, 1} = {0, a, 4} = {a, b, c, d} String A finite length sequence of symbols, presumably from some alphabet. u= w = 0110 y = 0aa x = aabcaa z = 111 special string: (also denoted by ) concatenation: wz = 0110111 length: |w| = 4 |x| = 6 but |u| = 0 R reversal: y = aa0 Some special sets of strings: *All strings of symbols from

+ = * - {}
Example: = {0, 1} * = {, 0, 1, 00, 01, 10, 11, 000, 001,} + = {0, 1, 00, 01, 10, 11, 000, 001,} A formal language is: 1) A set of strings from some alphabet (finite or infinite), in other words 2) any subset L of * Some special languages: {} The empty set/language, containing no strings {} A language containing one string, the empty string. Examples: = {0, 1} L = {x | x is in * and x contains an even number of 0s}

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

= {0, 1, 2,, 9, .} L = {x | x is in * and x forms a finite length real number} = {0, 1.5, 9.326,} = {a, b, c,, z, A, B,, Z} L = {x | x is in * and x is a Pascal reserved word} = {BEGIN, END, IF,} = {Pascal reserved words} U { (, ), ., :, ;,} U {Legal Pascal identifiers} L = {x | x is in * and x is a syntactically correct Pascal program} = {English words} L = {x | x * and x is a syntactically correct English sentence} LECTURE # 09 CHOMSKY CLASSIFICATION OBJECTIVE To explain Chomsky classification of grammar EXPLANATION THE CHOMSKY HIERARCHY The Chomsky hierarchy, named after linguist Noam Chomsky who studied formal grammars, describes a hierarchy of languages, ranging from the very simple regular languages up through the very general recursively enumerable languages. The hierarchy of languages can be described as a hierarchy of languages (sets), or a hierarchy of grammars, or a hierarchy of machines

Chomsky Hierarchy Languages Recursively enumerable languages Context-Sensitive Languages Grammars Type 0: Unrestricted grammars Machines Turing Machines (deterministic or nondeterministic) Linear-Bounded Automata (nondeterministic) Pushdown Automata (nondeterministic) Finite Automata (deterministic or nondeterministic)

Type 1: Context-Sensitive Grammars Context-Free Languages Type 2: Context-Free Grammars Regular Languages Type 3: Right-Linear Grammars Chomsky Classification Type 0

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

Definition - A phrase-structure grammar or a type 0 grammar is a 4-tuple G = (N , T , P, S ), where N is a finite set of non terminal symbols called the non terminal alphabet, T is a finite set of terminal symbols called the terminal alphabet, S N is the start symbol and P is a set of productions (also called production rules or simply rules) of the form u v, where u (N T ) N (N T ) and v (N T ) . Derivations are defined as follows: If u is a string in (NT ) and u v is a rule in P , from u , we get v by replacing u by v.This is denoted as u v . If 1 2 , 2 3 , . . . , n1 n , the derivation is denoted as 1 2 n or 1 n is the reflexive, transitive closure of .

The language generated by a grammar G = (N , T , P, S ) is the set of terminal strings derivable in the grammar from the start symbol. L ( G) = {w/w T Example Consider the grammar G = (N , T , P, S ) where N ={S, A}, T={a, b, c}, production rules in P are S aS c, S aAc, A b A typical derivation in the grammar is S aSc aaScc aaaAccc aaabccc n n The language generated is L(G) = {a b c /n 1}. Chomsky Classification Type I ,S w}

Department of Information Technology

Theory of computation (IT 010 404) Course File Nov 12 May13

Lecture Notes -Module : I

+ If the rules are of the form A , , (N T ) , A N , (N T ) , the grammar is called context-sensitive grammar. If in the rule u v, |u| |v|, the grammar is called length increasing grammar. Example Let G = (N , T , P, S ) where N = {S, B },T= {a, b, c}, P has the following rules: 1. 2. 3. 4. S aSB c S abc cB B c bB b b

Chomsky Classification Type II If in a grammar, the production rules are of the form, A , where A N and (N T ) , the grammar is called a type 2 grammar or context-free grammar. The language generated is called a type 2 language or context-free language. Chomsky Classification Type III If the rules are of the form A B, A , A, B N, , T , the grammar is called a right linear grammar or type 3 grammar and the language generated is called a type 3 language or regular set. We can even put the restriction that the rules can be of the form A aB, A b, where A, B N, a T , b T . This is possible because a rule A a 1 . . . ak B can be split into A a B , 1 1 B1 a2 B2 , . . . , Bk1 ak B by introducing new non terminals B1 , . . . , Bk .

Department of Information Technology