Professional Documents
Culture Documents
Ok Chomsky
Ok Chomsky
Ok Chomsky
Nom Chomsky
Department of Modern Languages and Research Laboratory of Electronics
Massachusetts Institute of Technology
Cambridge, Massachusetts
Abstract
There are two central problems in the (1) John ate a sandwich
descriptive study of language. One primary (2) Sandwich a ate John.
concern of the linguist is to discover simple
and srevealing* grammars for natural languages. In this case, we can test the adequacy of a
At the same time, by studying the properties of proposed linguistic theory by determining, for
such successful grammars and clarifying the basic each language, whether or not the clear cases
conceptions that underlie them, he hopes to are handled properly by the grammars constrncted
arrive at a general theory of linguistic in accordauce with this theory. For example, if
structure. We shall examine certain features of a large corpus of English does not happen to
these related inquiries. contain either (1) or (2), we ask whether the
grammar that is determined for this corpus will
The grammar of a language can be viewed as project the corpus to include (1) and exclude (21
a theory of the structure of this language. Any Even though such clear cases may provide only a
scientific theory is based on a certain finite weak test of adequacy for the grammar of a given
set of observations and, by establishing general language taken in isolation, they provide a very
laws stated in terms of certain wpothetical strong test for any general linguistic theory and
constructs, it attempts to account for these for the set of grammars to which it leads, since
we insist that in the case of each language the
-.- clear cases be handled properly in a fixed and
4Thi8 work was supported in part by the Army predetermined manner. We can take certain steps
(Signal Corps), the Air Force (Office of Scientific towards the construction of an operational
Research, Air Research and Development Command), characterization of ngrammatical sentences that
and the Navy (Office of Naval Research), and in will provide us with the clear cases required to
part by a grant from Eastman Kodak Company. set the task of linguistics significantly.
Observe, for example. that (1) will be read by an of linguistic structure in terms of the possi-
English speaker with the normal intonation of a bility and complexity of description (questions
sentence of the corpus, while (2) will be read (31, (4)). Then, in 6 6, we shall briefly
with a falling intonation on each word, as will consider the same theories in terms of (5) , and
any sequence of unrelated words. Other dis- shall see that we are Independently led to the
tinguishing criteria of the same sort can be same conclusions as to relative adequacy for the
described. purposes of linguistics.
(4) Can we COnStNCt reasonably simple 2.2. Suppose that we take the set A of transition
grammars for all interesting languages? symbols to be the set of English phonemes. We
(5) Are such grammars n reveali@ In the can attempt to construct a finite state grammar G
sense that the syntactic structure that they which will generate every string of English
exhibit can support semantic analysis, can provide phonemes which is a grammatical sentence of
insight into the use and understanding of langusge, English, and only such strings. It is immediately
etc. 7 evident that ths task of constructing a finite-
state rammar for English can be considerab
We shall first examine various conception6 simpli fi ied if we take A as the set of Englis 1%
ll4
morpheme3 or words. and construct G so that it exemple, the languages k L2, 3 described in
will generate exactly the grammatical stringsof (12) are not describable y any inite-state
these units. We can then complete the grammar
by giving a finite set of rules that give the gTj(i) L contains anb. aoar\ bn b ,
phonemic spelling of each word or morpheme in A
a ananbnbnb,..., and in
each context in which it occurs. We shall general, all sentences consisting
consider briefly the status of such rules in of n occurrences of a followed by
4 4.1 and 4 5.3. exactly n occurrences of b, and only
these ;
Before inquiring directly into the problem (fi) L2 contains afia, bnb, anbnbna.
of constructing a finite-state grammar for
Englieh morpheme or word sequences, let US bnananb, ananbnbnana,. ..,
lnvestlgate the absolute limite’of the set of and in general, all nmlrror-imagen
finite-state languages. Suppose that A is the sentencee consisting of a string X
alphabet of a language L, that al,. . *an are followed by X In reverse, and on4
these ;
eymbole of thir alphabet, and that S=aln.. .nan (W contains ana, b-b, anbnanb,
is a sentence of L. We sey that’s has an (i, j)- ZA
b anbna, ananbnananb, ...,
dependency with respect to L if and only if the and in general, all sentencee con-
following conditions are met: eietlng of a string X followed by
the identical string X. and only
(9)(i) lSi<j<n these.
(ii) -there are symbols bi,b cAwith the In 12, for example, for agy m we can find a
property that S1 is no d asentence of sentence with a dependency set.
L, and S2 is a sentence of L, where Sl Dm=((1,2m),(2.2m-1) ,.., (m,m+l)) .4
is formed from S by replacing the ith
symbol of S (namely, ai) by bi, and S2 2.3. Turning now to English, we find that there
is formed from Sl by replacing the are infinite sets of sentences that have depenw
jth symbol of Sl (namely, aj) by bj. sets with more than any fixed number of terms.
For exemple, let Sl,S2, . . . be declarative eentsrces.
fn other words, S has an (I, j)-depenency with Then the following are all English sentencee:
reepsct to L if replacement of the I symbol a. (13)(i) If Sl, then S2.
1
of S by bi ( bi#ai) requires a corresponding (ii) Either S3, or S4.
(iii) The men who said that S5, is
replacement of the jth symbol aj of S by b j arriving today.
(b #a ) for the resulting string to belong to L. These sentencee have dependencies between sifv-
J il neither”-“or”, ‘mans-siss. But we csn
“then”,
We sey that D- (q,@l) , . . .(a,,B,)}ie a choose Sl, S3, S5 which appear between the inter-
i dependent words, as (ISi), (13ii), or (13111) them-
dependency set for S in L if and only if the selves. Proceeding to construct eentencee in this
following c;;;;;iitions are met: way we arrive at subparts of English with just the
mirror image properties of the languages Ll and L2
(10)(i) For’l<i<m, S hae an (ci,Si)- of (12). Consequently, English falls condition x
dependency with respect to L (11). English is not a finite-state language, aha
we are forced to reject the theory of language
(II) for each l,j, ui< pj under discussion as failing condition (3).
(iii) for each i,j such that i#j. uibj We might avoid this consequence ti an
arbitrary decree that there is a finite upper
aha B,#Bj. limit to sentence length in English. This would
Thus, in a dependency set for S in L every two serve no useful purpose, however. The point is
dependencies are distinct in both terms and each that there are processes of sentence formation
sdetermininge element in S precedes all ede- that this elementary model for language is
terminede elements, where we picture a as intrinsically incapable of handling. If no
determining the choice of a ai finite limit is set for the operation of these
%’ processes, we can prove the literal lnapplica-
Evidently, if S has an m-termed dependency set bility of this model. If the processes have a
in L, at least 2” states are necessary In the limit, then the construction of a finite-state
finite-state grammar that generates the grammar will not be literally impossible (since
langnage L. a list is a trivial finite-state grammar), but
this grammar will be so complex as to be of little
Thie observation enables ue to state a use or interest. Below, we shall study a model
necessary condition for finite-state languages. for grammars that can handle mirror-image lan-
guagee. The extra power of such a model in the
(11) Suppoee that L is a finite-state infinite case is reflected in the fact that it is
bsguaSe. Then there is an m such that no much more ueeful and revealing if an upper limit
sentence S of L has a dependency set of more is set. In general, the aeeumption that a
than m terms in L. are infinite Is made for the purpose of simpli-
With thie condition in mind, we can easily fying the description.5 If a g-r has no
conetNct many nonfinite-state languages. For recursive steps (closed loops, in the model
diacuaaed above) it will be prohibitively complex- that there is apparently no significant approach
it will, in fact, turn out to be little better to the problems of grammar in this direction.
then a list of strings or of morpheme class
sequences in the case of natural languages. If it Botice that although for every n, a process
does have recursive devices, it will produce of n-order approximation can be represented as a
infinitely many sentencee. finite-state Markov process, the converse is not
true. For example, consider the three-state
2.4 Although we have found that no finite-state proceee with (So.S1), (S1,S1) .(S1,So>.
Markov process that produces sentencee from left
to right can serve as an English grammar, we (So,S2) ,(S2,S2) ,(S2,So) as its only connected
might inquire into the possibility of constructing states, and with a, b, a, c, b, c as the reapact-
a sequence of such devices that, in some nontrivial ive transition symbols. This process can be
way. come closer and closer to-matching the outnut represented by the following state diagram:
of-a satisfactory English grammar. Suppose, foI
example, that for fixed n we construct a finite-
state grammar in the following manner: one a tate
of the grammar is associated with each sequence of
English words of length n and the probability that
the word X will be produced when the system is in
the state Si is equal to the conditionalproba-
bility of X, given the sequence of n words which C a
defines S . The output of such grammar is
cuatomariiy called an n+lat order approximation to
English. Evidently, as n increases, the output of This process can2 roauce the sentencee ana,
such grammars will come to look more and mora like anbrra, a”bc’b a, a”b-bOb-a ,..., c-c,
English, since longer and longer sequences bavea cfibnc, chbnb*c cc\b”b’\bnc,..., but not
high probability of being taken directly from the ahbnbnc, chbnb’a, etc. The generated
sample of English in which the probabilities were language has sentencee with dependencies of any
determined. This fact has occasionally led to finite length.
the suggestion that a theory of linguistic
structure might be fashioned on such a model. In 8 2.4 we argued that there is no
significant correlation between order of approxi-
Whatever the other interest of statistical mation and gremmaticalneaa. If we order the
approximation in this sense may be, it is clear strings of a given length in terms of order of
that it can shed no light on the problems of approximation to English, we shall find both
grammar. There is no general relation betweenthe grammatical and ungrammatical strings scattered
frequency of a string (or its component parts) and throughout the list, from top to bottom. Hence
its grammaticalneas. We can see this moat clearly the notion of statistical approximation appears
by considering such strings as to be irrelevant to grammar. In 4 2.3 we pointed
out that a much broader class of processes,
(14) colorleaa green ideaa sleep furiously nemely, all finite-state Markov processes that
produce transition symbols. does not include an
which is a grammatical sentence, even though It is English grammar. That la, if we construct a
fair to assume that no pair of ita words may ever finite-state gremmar that produces only English
have occurred together in the past. Botice that a sentencee, we know that it will fail to produce
speaker of English will read (14) with the an infinite number of these sentences; in par-
ordinary intonation pattern of an English sentence, ticular, it will fail to produce an infinite
while he will read the equally unfamiliar string number of true sentences. false sentencee,
reasonable questions that could be intelligibly
(15) furiously sleep ideas green colorleaa asked, and the like. Below, we shall investigate
e still broader claes of processes that might
with a falling intonation on each word. as In provide us with an English gremmar.
the case of any ungrammatical string. Thus (14)
differs from (15) exactly as (1) differs from (2); 3. Phrase Structure.
our tentative operational criterion for grem-
maticalneas supports our intuitive feeling that 3.1. Customarily, syntactic description is
(14) is a grammatical sentence and that (15) is given in terms of what is called nimmediate
not. We might state the problem of grammar, in constituent analyeie.n In description of this
pert, as that of explaining and reconstructing sort the words of a sentence are grouped into
the ability of an English speaker to recogniae phrases, these are grouped into smaller conati-
(l), (14), etc., as grammatical, while rejecting tuent phrases and so on, ‘until the ultimate
(2) , 05.). etc. But no order of approximation constituents (generally morphemes3) are reached.
model can distinguish (14) from (15) (or an These phrases are then classified as noun
indefinite number of similar pairs). As n phrases (EP), verb phrases (VP), etc. For
increases, an nth order annroximation to Eneliah example, the sentence (17) might be analyaed as
will exclude (as more and-more imnrobabla) an in the accompanying diagram.
ever-increasing number of arammatical sentencee _
while it still-contain2 vast numbers of completely
ungrammatical strings. We are forced to conclude
(17) that are derivable from some L x ,F] grammar,
and we eey that L is a terminal lm if it is
the set of terminal strings from some system
c 2 4’1.
In every Interesting case there will be a
terminal vocabulary VT (VT C VP) that
exactly characteriaer the terminal strings, in
Evidently, description of aentencea in such tenaa the sense that every terminal string la a string
permita considerable simplification over the in VT and no symbol of VT la rewritten in any of
word-by-word model, since the composition of a the rules of F. In such a case we can interpret
complex claaa of expreealona such as XP Fan be the terminal strings as constituting the law
stated just once in the grammar, and this class under analysis (with Y aa its vocabulary), and
can be used as a building block at various the derivations of thege strings as providing
points in the construction of sentencee. We now their phrase structure.
aak what fow of grammar corresponds to this
conception of lingulatic structure. 3.3. Aa a simple example of a system of the form
(18). consider- the foliowing smell part of Pngliah
3.2. A phrase-structure grammar is defined by a grammar:
finite vocabulary (alphabet) Y , a finite aet 2
of initial strings in Y end E finite aet Fof (20)
C : WSentencen#
rules of the form: X +?i, where X and Y axe I: Sentence - l$VP
strings in Y . Path such rule is Interpretedas VP - Verb*NP
the instruct P on: rewrite X as Y. For reaaona NP - the-man, the” book
that will appear directly, we require that In Verb - took
each such [ 2 ,F] grammar Among the derivations from (20) we have, in
particular :
(18) I: : xl,.., cn
(21) Dl: $~%nfit;~~;#
F: ..- - - -..
x1 - y1
.
. #?henmannYerbnXPn#
#“thenmannYerbn thenbookn#
‘rn - ‘rn #“the” man” tookm thenbook” #
D2 : #“Sentence”#
Yi is formed from Xi by the replacement of a WXPnYPn#
single symbol of Xi by some string. Neither #C\thenmannYPP”#
#%renmanr\VerbnXPn#
the replaced symbol nor the replacing string #*the* mann tooknl?Pn#
may be the identity element U of footnote 4. #nthenmanntookn thefibookn#
Given the [ c ,F] grammar (18). we say that: These derivations are evidently equivalent; they
differ only in the order in which the rules are
(19)(i) a atring BRollowa from a string a applied. We can represent this equivalence
if a&xi W end $4*YinW, for graphically by constructing diagrams that
some i I rni7 correapondd, in an obvious wey, to derivations.
a derivation of the string S is a Both Dl and D2 reduce to the diagram:
(ii)
sequence D=(Sl,. . ,St) of atr 1 ngn,
where Sle c and foreach i< t, Si+l (22) #*Sentencefi #
In f 2.2 we found that Ll, L2 and 5 of Wote that the latter is more abstract than
the finite-state model in the sense that symbols
(12) were not finite-state langnsges. Ll and Lzl that are not included in the vocabulary of a
however, are terminal languages. For LIB e-e., language enter into the description of this
language. In the terms of 4 3.2, V P properly
we have the [ I: ,F] grammar
includes VT. Thus in the cs.se of (29) , we
(29) x : Z c\b describe Ll in terms of an element Z which is not
F: Z-a ; and in .the case of (20)-(24). we introduce
Z -anZhb in L1 symbols as Sentence, NP, VP, etc., which are
such
not words of English. into the description of
This establishes (271). English structure.
Suppose that L4 is a derivable language with 3.6. We can interpret a [ c ,F] grammar of the
form (18) as 8. rather elementary finite-state
process in the following way. Consider a system
that has a finite number of states So,..,S .
a -cb i, where the bins are not in VP and are 9
i When in state So, it can produce any cP the
all distinct. Then this new grammar gives a
terminal language which is simply a notational strings of X , thereby moving into a new state.
variant of L . Thus every derivable language Its state at aq point is determined by the sub-
is also term f nal. set of elements of Xl,..,Xm contained as sub-
As an example of a terminal, nonderivable strings in the last produced string, and it moves
language cohsider the languege L5 containing just to a new state by applying one of the rules to
the strings this string, thus producing &new string. The
system returns to state S with the production
(30) anb, chaObnd, chcnanbhdhd, of a terminal string. Th& system thus produces
0-c “c*a-b^d^d”d,... derivations, in the sense of 5 3.2. The process
is determined at any point by its present state
An infinite derivable language must contain an and by the last string that has been produced,
infinite set of strings that can be arranged in a and there is a.finite upper bound on the amount
sequence Sl,S2,... in such a way that for saeo~ of inspection of this string that is necessary
before the process can continue, producing a new
rule X-T, Si follows from Si-1 by application string that differs in one of a finite number of
of this rule, for each i >l. And Y in this rule ways from its last output.
must be formed from X by replacement of a alnple
symbol of X by a string (cf. (18)). This is
evidently impossible in the case of L . This It is not difficult to construct language8
language is, however, the terminal la d guage given that are beyond the range of description of
by the following grammar: [ x ,F] grammars. In fact, the language L3 of
(12111) is evidently not a terminal language. I
do not know whether English is actually a terminal
language or whether there are other actual
Z -cnZ”d languages that are literally beyond the bounds of
phrase structure description. Hence I see no wsy
An example of a finite-sta.te, nonderivable to disqualify this theory of linguistic structure
langoage is the language L6 containing all and on the basis of consideration (3). When we turn
only the strings consisting of 2n or 3x1 to the question of the complexity of description
occurrences of a, for nP1.2,. . . . Language Ll (cf. (4)), however, we find that there are ample
of (12) is a derivable, nonfinite-state language. grounds for the conclusion that this theory of
with the Initial string anb and the rule: linguistic structure is fundamentally inadequate.
a-b -ra”a-b* b. We shall now investigate a few of the problems
that arise when we attempt to extend (20) to a ,Applying this rule to each of the three Afnv
full-scale grammar of English. sequences in the last line of (33), we derive
4. Inadeauacies of Phrase-Structure Grammar (35) #rrthenmannhavenpastn #%enenn #^
takening” #nthenbook”#.
4.1. In (20) we considered on4 one wey of
developing theelement Verb, namely, as “tookn. In the first paragraph of 8 2.2 we mentioned
But even with the verb stem fixed there are that a grammar will contain a set of rules (called
a great many other forms that could appear in morphophonemic rules) which convert strings of
the context sthe man -- the book,s e.g., stakeq” morphemes into strings of phonemes. In the
“has taken,” shas been taking,” sfs teking,s morphophonemics of English, we shall have such
“has been taken,” swill be takiog,n and se 011. rules as the following (we use conventional,
A direct description of this set of elements rather than phonemic orthography) :
would be fairly complex, because of the heavy
dependencies among them (e.g., shas taken” but (36) havenpast - had
not “has taking.5 Ye being taken5 but not “1s benen - been
being taking,s etc.). We can, in fact, give a take-ing - taking
very simple analysis of “Verb” as a sequence of will” past - would
independent elements, but only by selecting as can~past -) could
elements certain discontinuous strings. For M*present - M
example, in the phrase “has been takiDg5 we can walk-past - walked
separate out the discontinuous elements shas..er$ takenpast - took
Nba..ing,5 and stakes, and we can then say that etc.
these eleeents combine freely. Following this
course systematically, we replace the last rule Applying the morphophonemic rules to (35) we
in (20) by derive the sentence:
(32) (i) Verb -AuxillarynV (37) the man had been taking the book.
(ii> V-take, eat,...
(iii) Auxiliary --?!8Mz Qd (be-w Similarly, with one major exception to be
discussed below (and several minor ones that we
( -1 M-will*, can, shall, mey, must shall overlook here), the rules (32), (34) will
(v) C - past, present give all the other forms of the verb in
declarative sentences, and only theee forms.
The notations in (32111) are to be inter-
preted .a5 follows: in developing sAuxiliar+ This very simple ana4eis, however, goes
in a derivation we must choose the unparenthe- beyond the bounds of [ x .F] grammars in several
sised element C, and we may choose sero or more respects. The rule (34). although it is quite
of the parenthesised elements, in the order simple, cannot be incorporated within a [ 2 ,F]
given. Thus, in continuing the derivation grammar, which has no place for discontinuous
D of (21) below line five, we might proceed elements. Furthermore, to apply the rule (34)
a t follows: to the last line of (33) we must know that stakes
is a V, hence, a Y. In other words, in order to
(33)#nthenmannVerbn thenbook ^# apply this rule It is necessary to inspect more
[from Dl of (21)] than just the string to which the rule applies;
#^ the;(~~,Auxiliary” V” the* book * # it is necessary to know some of the constituent
structure of this string, or equivalently
(cf. 3 3.3), t 0 i nspect certain earlier lines in
#~thenmannAuxiliaryntakenthe”bookn# Its derivation. Since (34) requires knowledge of
1:(3211)1 the ‘history of derivation’ of a string, it violates
#^then mann C” have” enc\ hen ing n take n the elementary property of [ 2 ,F] grammars
discussed in f 3.6.
thenbookn #
C(32fii). choosing the elements C,
4.2. The fact that this simple analysis of the
havenen, and bening] verb phrase as a sequence of independently chosen
#nthenman*past”5ha~e’5en”beningntake’5 units goes beyond the bounds of-[ c .F] grammars,
thenbook” # suggests that such grammars are too limited to
C(32dl give a true picture of linguistic structure.
Further study of the verb phrase lends additional
Suppose that we define the class Af as containing support to this conclusion. There is one major
the-afflxes *ens, s I@, and the C’s; and the limitation on the independence of the elements
class v as includine all V's.. MIS. shaves,and she .m introduced in (32). If we choose an Intransitive
We can then convert-the last.llne-of (33) into a verb (e.g., ncome,n 5occur.s etc.) as V in (32).
properly ordered sequence of morphemes by the we cannot select be-en as an auxiliary. We can-
following rule : not have such phrases as “John has been come,s
“John is occurred ,” and the like. Furthermore,
(34) Afnv -vnAfn # the element beAen cannot be chosen independently
of the context of the phrase sVerb.n If we have
320
the element “Verb” in the context *the man -- the 5.Transformational Grammar.
food,” we are constrained not to select benen in 5..1. Each grammatical transformation T will
applying (32). although we are free to choose any essentially be a rule that converts every sentence
other element of (32). That is, we can have “‘the with a given constituent structure into a new
man is eating the food,s “the man would have been sentence with derived constituent structure. The
eating the food,’ etc., but not n the man is eaten transform and its derived structure must be related
the food,* “the man would have been eaten the food,* in e fixed and constant wey to the structure of
etc. On the other hand, if the context of the the transformed string, for each T. We can
phrase “Verb” Is, e.g., sthe food - by the man,” characterize T by stating, in StNCtUrd terms,
we are required to select benen. We can have the
_-- domain
-.--. of strings to which it applies and the
s the food is eaten by the man,” but not sthe food cmee that it effeke on any such string.
is eating by the man,” etc. In short, we find that Let us suppose in the following discussion
the element be-en enters into a detailed network that ve have a [ H ,P] grammar with a vocabulary
of restrictions which distinguish it from all the VP and a terminal vocabulary VT C VP, as in 4 3.2.
other elements introduced in the analysis of “Verb”
in (32). This complex and unique behavior of In 5 3.3 we showed that a [ 2 ,F] grammar
benen auggests that it would be desirable to permits the derivation of terminal strings, and we
exclude it from (32) and to introduce passives into pointed out that In general a given terminal string
the grammar in some other way. will have several equivalent derivations. Two
derivations were said to be equivalent if they 9
There is, in fact, a very simple way to reduce to the same diagram of the form (22), etc.
incorporate sentences with benen (i.e., passives) Suppose that Dl, . . , Dn constitute a maximal set
into the grammar. notice that for every active of equivalent derivations of a terminal string
sentence such as sthe man ate the foods we have a S. Then we define a phrase marker of S as the set
corresponding passive “the food was eaten by the of strings that occur as lines in the derivations
man” and conversely. Suppose then that we drop the A string will have than one phrase more
D1.. l ,Dn.
element be-en from (32111). and then add to the
grammar the following rule: marker if and only if it has nonequivalent
derivations (cf. (24)).
I21
T if the restricting class II associated with T tpol J2 WY3;y3 J4) = y3
contains a sequence ($, . . ,X$ into which (S,K) is tp(Y1. a. ,Y4;Y4> = by”Y1
The domain of a tra sformation is
% % “%*of ordered pairs (S,Kf of a string S t,(Yl,.. ,Yn;Yn ,.., Yr) = Yn for all x&r++.
and a phrase marker K of S. A transformation rosy
be applicable to S with one phrase marker, but not The derived transformation tIf thus has the follow-
with a second phrase marker, in the case of a string ing effect:
S with ambiguous constituent structure. (WJ(U t$Y,,.. ,Y,) = Y1 - Y2%inen - I3 - l$Yl
In particular, the paSBiVe transformation (ii> tG( tb”man, past, eat, then food) =
described in (38) has associated with it a the”food - past” be” en - eat - bynthen
restricting class Il, Containing just one sequence: man.
The rule8 (34),(36) carry the right-hand side of
(42) BP= { (&. Auxiliary, (4&i) into “the food was eaten by the men,@ just
as they carry (43) into the corresponding active
This transformation can thus be applied to eny sthe man ate the food.”
string that is analyzable into an SF followed by The pair (R ,t ) as in (42),(47) completely
an Auxiliary followed by a V followed by an HP. characterizes thg p&.eive transformation as
For example, it can be applied to the string (43) described in (38). B tells us to which strings
analyzed into substrings sl,..,s4 in accordance this transforxation applies (given the phrase
with the dashes. markers of these strings) and how to subdivide
these strings in order to apply the transformation,
(43) the man - past - eat - the food. and t tells us what structural change to effect
on thg subdivided string.
5.2. In this way, we can describe in structural
texms the set of strings (with phrase markers) to A grammatical transformation is specified
which eny transformation applies. We must now completely by a restricting class R and an
specify the structural change that a transformation elementary transformation t, each of which is
effects on any string in its domain. An element- finitely characterizable, as in the case of the
transformation t is defined by the following paesive. It is not difficult to define rigorously
property : the manner of this specification, along the lines
she tched above. To complete the development of
(44) for each pair of integers n,r (nsr), transformational grammar it is necessary to show
there is a unique sequence of integers (ao,al,. . ,cg) how a transformation automatically assigns a
derived phrase marker to each transform and to
and a unique sequence of strings in VP (Z,,. . ,Zk+l) generalize to transformations on sets of strings.
ruchthat (i)ao=O; k>O;llajl r for 15 j<k;Yoi[J1l (These and related topiqs are treated in reference
[3].) A transformation will then carry a string S
(ii) for each Tl,. . ,Yr, with a phrase marker K (or a Bet of such pairs)
into a string St with a derived phrase marker Kc.
t(Y1,..,Yn;Yu,..,Yr)=Ya* znY”***Y* cYn
al 2 a2 0 5.3. From these considers.tions we are led to a
Thus t can be understood as converting the picture of gl’emmars as pOBBeSSing a t,riwrtitS
occurrence of Yn in the context structure. Corresponding to the phrase structure
WbBiS We have a sequence of rules of the form
(45) Y1
A
. . - Ynwl* --^Yn+l^ . .hYr X-Y, e.g.. (20). (23). (32). Following this we
have a se Pence of transformational rules such a*
(34) and ?38). Finally, we have a sequence of
into a certain string
n Zlh . .n Y n Zk+l Y morphophonemic rules such as (36). sgain of the
aO % form X--Y. To generate a sentence from such a
which is unique, given the sequence of terms grammar we construct an extended derivation
0 Y ) into which Ylh ..nY, is subdivided. beginning with an initial string of the phrase
t bbliier the string Y -. . -Yr into a new string structure grammar, e.g., #^Sentence^#, as in
wl-. . qr which is rela t ed in a fixed way to (20). We then rnn through the rules of phrase
Y1-. .-Yr. More precisely, we associate with t the structure, producing a terminal string. We then
derived transformation t*: apply certain transformations, giving a string of
morphemes in the correct order, perhaps quite a
(46) t* is the derived transformation of t if different string from the original terminal string
and only if for all Yl,.., Y,, t*(Yl,.., Yr)=Wln ..^s, Application of the morphophonemic rules Converts
where W,==t(Yl, . . ,Yn;Tn,. . ,Y,) for each n L r. this into a string of phonemes. We might run
through the phrase structure grammar Several times
We now associate with each transformation T and then apply a generalized transformation to the
an elementary transformation t. For example, with resulting set of terminal strings.
the passive transformation (38) we associate the In ! 3.4 we noted that it is advantageous to
elementary transformation t defined es follows: order the rules of phrase structure into a
P eequence, and to distinguish obligatory from
optional rules. The aeme is true of the trans-
(47) tpul;Y1’ ‘Y4) = ‘4 * l
I.22
sequence affix-verb into the sequence verb-effix, in terms of such essentially formal criteria. as
and the passive transformation (38). Notice that simplicity. In d 1 we SIIgg85ted that there are
(34) must be applied in every extended derivation, Other relevant consideration5 of adequacy for
or the result will not be a grammatical sentence. such theories. We can ask (cf.(S)) whether or
Rule (3b), then, is sn obligatory transformation. not the syntactic structure revealed by these
The passive transformation, however, may or may theories provides insight into the use and under-
not be applied; either way we have a sentence. The standing of language. We can barely touch on
passive is thus an optional transformation. This this problem here, but even this brief discussion
distinction between optional and obligatory trans- will suggest that this criterion provides the
formation5 leads us to distinguish between two same order of reletive adequacy for the three
classes of sentences of the language. We have, on models we have considered.
the one hs.nd, a kernel of basic sentences that are
derived from th8xnal strings Of the phrase- If the grammar of a langoege is to provide
structure grammar by application of Only insight into the way the language is Understood,
obligatory transformations. We then have a set of it mUst be true, in particular, that if a sentence
derived 58nt8nC85 that are generated by applying Is ambiguous (Understood in more than one way),
optional transformations to the strings underlying then this sentence is provided with alternative
kern81 Sentences. analyses by the grammar. In other words, if a
When we actually carry out a detailed study certain sentence S is ambiguous, we can test the
of English StrUcture, we find that the grammar can adequacy of a given linguistic theory by asking
be greatly simplified if we limit the kernel to a whether or not the simplest grammar constructible
very small eet of simple, active, declarative in terms of this theory for the languege in
sentence5 (in fact, probably a finite set) such as question automatically provide6 distinct ways of
"the man ate the food,5 etc. We then derive generating the sentence S. It is instructive to
questions, paseivee, sentences with conjunction, compare the Markov prOC8Sf3, phrase-structure, and
sentence5 with compound noun phrases (e.g., transformational models in the light of this test.
"proving that thsorem was difficult.5 with the NP
aproving that theoremn),li! etc., by transformation. In 53.3 we pointed out that the simplest
Since the result of a transformation is a sentence [ x ,F] grsmmsr for English happens to prOVid8
with derived constituent etructure, transfofiaaticns nonequivalent derivations for the sentence nthey
can be compounded, ana we can form question5 from are flying planes,H which is, in fact, ambiguous.
~SSiV8S (e.g.) "was the food eaten by the man"), Thie reasoning does not appear to carry over for
etc. The actual 58nt8nc85 of real life are finite-state grammars, however. That IS, there
usually not kernel sentences, but rather is no obvious motivation for assigning two
complicated transforms of these. We find, however, different paths to this ambigaous sentence in any
that the tr5nsformations are, by and large, meanin& finite-state grammar that might be proposed for
preserving, so that we can view the kernel a part of English. Such examples of construction-
sentences Underlying a given sentence as being, in al homonymity (there are many others) constitute
some sense, the elementary 5content elements" in independent evidence for the superiority of th8
terms of which the actual transform i~%nderstood.~ phrase-etructure model over finite-state grammars.
We discuss this problem briefly in 4 6, more
extensively in references Cl], [2]. Ptrrther investigation of English brings to
IZI 83.6 W8 pointed Out that a grammar of light examples that are not easily explained in
phrase strUCtur8 is a rather elementary type of terms of phrase structure. Consider the phrase
finite-State process that is determined at each
point by its present state and a bounded amount, of (49) the shooting of the hunters.
its last output. W8 discovered in $! 4 that this
limitation is too severe, and that the grammar can
b8 simplified by adding tran5formational l’,l18S We can understand this phrase with nhuntersII as
that take into account a certain amount of the subject, analogously to (50), or as the
constituent structure (i.e., a certain history of object; analogously to (51).
derivation). Hovever, each transformation is- still
finitely charactarieable (cf. $5 5.1-2). and th8 (50) the growling of lions
finite restrlctin class (41) aSSOCiat8d with a
transformation in f icates how mUch information
about a string is needed in order to apply this (51) the raising of flowers.
transformation. The grammar can therefore still
be regarded
- as an elementary finite-state process Phrases (50) and (51)) however, are not similarly
Of the type Corresponding t0 phrase strUCtUr8. ambiguous. Yet in terms of phrase structure, each
There is still a bound, for each grammar, on how of the58 phrases is represented as: the - V*ing-
nruch of the past output must be inspected in order of9P.
for the process of derivation to Continue, even
though more than just the last output (the last Careful analysis of English show5 that we can
line of the derivation) must be known. simplify the grammar if we strike the phrases
(49)-(51) out of the kern81 and reintroduce them
6. Explanatory Power of Linguistic Theories transformetionally by a transformation Tl that
carries such sentences as "lions growl" into (50).
We have thus far COnSid8red the relative and a transformation T2 that carries such sentences
adequacy of theories of linguistic structure only
I.23
as "they raise flowerss into (51). T1 and T2 will replace "is" by nare 'I in (13iii), and choose
be similar to the nominalizing transformation S5 of any required length).
described in fn.12, when they are correctly
But both shunters shootH and zthey 7.2 or W may be the identity element U (cf..fn.4)
constructed. in this case. Note that since we limited (18)
shoot the hunterss are kernel sentences; and
so as to exclude U from figuring significantlv
application of Tl to the former and T2 to the on either the right- @r the left-hand side
latter yields the result (49). Hence (49) has of a rule of B, and since we reouired that
two distinct transformational origins. It is a only a single symbol of the left-hand side
case of constructional homonymity on the trans- msy be-replaced in any rule, it follows that
formational level. The ambiguity of the grammat- Yi must be at least as long as Xi. Thus we
ical relation in (49) is a consequence of the fact
have a simple decision procedure for deriv-
that the relation of "shoot" to lfhuntersz
differs in the two underlying kernel sentences. ability and terminality in the sense of
We do not have this smbiguity in the case of (50). ( 19iii) , (19p) .
(51)s since neither "they growl lions" nor
8. See [3] for a detailed development of an
"flowers raise" is a grammatical kernel sentence.
algebra of transformations for linguistic
There are many other examples Of the same description and an account of transforma-
tional grammar. For further application of
m-ma1 kind (cf. [11,[21), and to my mind, they this type of description to linguistic
provide quit6 convincing evidence not only for
material. see Cl], [2], and from a somewhat
the greater adequacy of the transformational
different point of view, ['cl.
conception Of linguistic structure, but also for
the view expressed in $5.4 that transformational
analysis enables us to reduce partially the 9. It is not difficult to give a rigorous
definition of the equivalence relation in
pro3lem of explaining howwe understand a sentence
to that of explaining how we understand a kernel question, though this is fairly tedious.
sentence.
10. The notion "is an should actually be
In summary, then, we picture a language as relatioized further to a given occurrence
having a small, possibly finite kern& of basic of si in S. We can define an occurrence of
sentences with phrase structure in the sense of ai in 5 as an ordered pair (zi,X), where X
331 along with a set of transformations which
is an initial substring of.S, and si is a
Can be applied to kernel sentences or to earlier
transforms to produce new and more complicated final substring of X. Cf. 651, p.297.
sentences from elementary components. We have 11. Where U is the identity, as in fn. 4.
seen certain indications that this approach may
enable us to reduce the immense complexity of 12. Notice that this sentence requires a
actual language to manageable proportions and; in generalized transformation that operates on
addition, that it may provide considerable insight a pair of strings with their phrase markers.
into the actual use and understanding of language. Thus we have a transformation that converts
Footnotes Sl,S2 of the forms NP-VPl, it-VP2, respect-
1. Cf. c73. Finite-state grammars can be ivekf , into the string: ingnVPl - VP2. It
represented graphically by state diagrams, as converts Sl= "they - prove that theorem",
in C7]. P.13f. s2= "it - was difficultI' into "ing prove
2. See [6], Appendix 2, for an axiomatization Of that theorem - was difficult," which by (34)
concatenation algebras. becomes sproving that theorem was difficu1t.z
Cf. il], [3] for details.
3. By 'morphemes' we refer to the smallest Bibliogranny
grammatically functioning elements of the 611 Chomsky, N., The Logical Structure of
language, e.g., "boys, "run", "ing" in Linguistic Theory (mimeographed).
"mining" , flsll in "bookss, etc. c21 Chomsky, N., Syntactic Structures, to be
published by Mouton & Co.. 'S-Gravebe. - ~
4. In the case of Ll, bJ of (9ii) can be taken Gtherlands;
as an identity element U which has the 633 Chomskv. M.. Transformational Analyzis,Ph. D.
Dissertation, University of Pennsylvania,
property that for all X, UAX=X"U=X. Then
June, 1955.
Dm will also be a dependency set for a Harris ,Z.S., Discourse Analysis, Language
c41
sentence of length 2m in Ll. 2891 (1952).
c51 Quine.
- . W.V.. Mathematical Logic, revised
5. Note that a grammar must reflect and explain edition,HarvardUniversity Press., Cambridge,
the ability of a speaker to produce and underc- 1951.
stand new sentences which msy be mush longer C61 Rosenbloom, P.. Elements of Mathematical
than any he has previously heard. Logic, Dover, New York, 1950
6. Tnus we can always fina sequences of n+l c73 Shannon & Weaver, The Mathematical Theory of
words whose first n words and last n words Communication, University of Illinois Press,
msy occur, but not in the same sentence (e.g. Urbana, 1949.