Connectionism and a single-route account

Connectionist models are computer simulations that aim to emulate the language learning
behaviours of the child. The more closely the computer model echoes child language data, the more
plausible it becomes that the way in which the computer learns reflects the way in which the child learns.
The startling discovery by connectionist modellers was that a single route for learning the past tense may
suffice. The single system is a pattern associator, which detects and exploits the numerous regularities and
quasi-regularities that exist across the board among both regular and irregular verbs. The fallout from the
single-route model has been dramatic. It calls into question the distinction between regular and irregular in
inflectional morphology. It further suggests that morphemes are not explicitly represented as discrete
entities. And the need for linguistic symbols like V, together with the rules for combining symbols, are
dispensed with. These are all foundational assumptions of modern (and ancient) linguistic theory, so it is
no wonder that the advent of connectionism, about 30 years ago, was heralded as the dawn of
a new age in linguistics and cognitive science.
Irregular past tense forms are neither chaotic nor idiosyncratic, in the way characterized by Pinker
(1999). Arguably, only two verbs in English are genuinely irregular, the so-called suppletive forms be (past
tense was or were) and go (past tense went). For the remaining irregulars we can detect nine different
clusters of verbs, each with its own pattern for the formation of past tense forms (McClelland & Patterson,
2002). For example, 28 verbs which end in /d/ or /t/ have identical forms in both present and past tense,
including cut, hit and bid. Another group, also ending in either /d/ or /t/ simply change the vowel in the past
tense, for example hide/hid, slide/slid and fight/fought. Yet another group replaces word-final /d/ with /t/ to
produce the past tense form: send/sent, bend/bent and build/built. By this point, your cunning eye will have
detected a pattern, based on the popularity of /d/ and /t/ for past tense endings. Fully, 59 per cent of the 181
irregular verbs listed by Pinker & Prince (1988) take word-final /d/ or /t/. And you will recall that these two
phonemes constitute the realizations of the regular past tense morpheme -ed. Connectionist models exploit
the systematic patterns that exist among sub-groups of irregulars, including the frequent appearance of /d/
and /t/. They treat the learning problem for these patterns as identical to the problem of finding the pattern
in regular verbs. Hence, all verbs are learned by the same model in the same way at the same time, by
establishing the various patterns of association that exist across present and past tense forms in different
sub-groups of verbs.
In the connectionist approach, learning is achieved by a network of units that correspond,
metaphorically, to neurons in the brain. Each unit is connected to many other units, in a way that mirrors
the synaptic connections between neurons. The level of activation for a given unit is computed from the
activation levels of all the units feeding into it. If the overall activation level is sufficiently high, the unit
will be switched on, causing it to send a signal via all of its connections to the output layer. If, on the other
hand, the activation level of a unit does not exceed a critical threshold, it remains switched off.
The sample network in Figure 7.3 looks complicated enough with its numerous connections. But
the real thing is even more bamboozling. In Rumelhart & McClelland (1986), the input and output layers
each contain 460 separate units, each of which is switched either on or off for a given learning trial. The
task of the network is to match the root form of a verb with the correct past tense form. It does this by
gradually forming patterns of association between the features of words. The association is not between the
whole words, drive and drove, but between parts of words: dr- and dr-, driand dro-, -ive and -ove, and so
on. At the same time, other associations are inhibited, for example, between -rive and -rived, to discourage
drived as the output for drive. The focus on features allows the network to detect the patterns that exist
among different families of verbs. For example, most of the units that are turned on when shrink is entered
will also be activated when stink is entered. The output from the network gradually converges on the
correct form.
In Rumelhart & McClelland’s (1986) original model, 420 verbs were each entered into the model
200 times, yielding a total of 84,000 trials. The end result of all this activity was very impressive. The model
converted most of the input verbs into their correct past tense counterparts, both regular (seem/seemed,
melt/melted) and irregular (make/made, sing/sang). The next challenge was to see if the model could
generalize beyond the verbs it had been trained on and generate appropriate past tense forms for 86
unfamiliar verbs. The model managed to add -ed to three quarters of the new regular verbs. It did so by
analogy with the verbs in its training set. The model also generated some overregularization errors, similar
to those made by children, like gived and digged. Intriguingly, these errors emerged after an initial
period when the correct irregular form was produced, somewhat like children (though see below). The
model also did well in recognizing clusters of irregular verbs. It could therefore cope with a new verb like
cling/clung by analogy with ring/rung and sing/sung. The model also produced blended forms, like gaved
and stepted, which we have seen are a feature of child language, but which are not easily accommodated
by Words and Rules theory. Overall, then, the achievements of connectionist models provide a serious
challenge to the idea that we need two systems for producing past tense forms. Maybe just one system – a
simple pattern associator – is all that is required.
Problems with connectionist models
All is not entirely rosy in the connectionist garden. Early models suffered from a number of
deficiencies (see Pinker & Prince, 1988, for a full account). One problem was the presence of a ‘teacher’
in the model, essentially a way to compare its output against the target and adjust learning where the two
did not match. This ‘teacher’ can be seen as a form of corrective input, a concept which, you will recall, is
anathema to many theorists, both nativist and nonnativist (but see Chapter 4 for an alternative view).
Another problem was with homophones like brake and break. Connectionist models that input
only information about phonology will treat these two verbs as identical, because they sound the same. Yet
one is regular, while the other is irregular (brake/braked versus break/broke). A third problem is with novel
and very low frequency regular verbs. Connectionist models have difficulty in generalizing to novel words
that differ from those in their training set. They either produce odd blends (like trilb/treelilt) or no output
at all. The failure of connectionist models in this regard may be inherent in their design (Marcus, 2001).
Children and adults, in contrast, can cope with the extremes of novelty and inflect whatever verb comes
their way. Even odd concoctions that violate the phonological constraints of English, like ploamph, can be
accommodated by people (ploamphed), but not by connectionist models.
A fourth problem for connectionism, which we can mention briefly, is the modelling of U-shaped
development, a phrase that captures the dip in performance we mentioned earlier: errors with irregulars
creep in after a period of correct performance, before being phased out again. The U-shape appears if we
make a graph with percentage correct use on the vertical axis, and time on the horizontal axis. Unhappily,
Rumelhart & McClelland based their model on inaccurate descriptions of this U-shape. They assumed a
simple developmental sequence for irregulars: correct → error → correct. But correct uses never disappear.
In fact, even when overregularization errors emerge, they remain in the minority, with correct forms
predominating. Hence, the kind of U-shaped behaviour generated by Rumelhart & McClelland’s model is
not the kind actually exhibited by children. We have made brief mention of just four problems with early
connectionist models: the presence of a teacher; homophones; low frequency and/or unusual verbs; and the
course of U-shaped development. Most of these problems have, to some extent, been addressed in
subsequent versions of connectionist models. But no model has accommodated all of these problems in a
single, supersize solution to all connectionist headaches. Instead, individual models have chipped away at
individual problems.
From: Child Language: Acquisition and Development by Matthew Saxton