Download as pdf or txt
Download as pdf or txt
You are on page 1of 23

This article was downloaded by: [University of Arizona]

On: 09 January 2013, At: 07:01


Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office:
Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Cognitive Neuropsychology
Publication details, including instructions for authors and subscription
information:
http://www.tandfonline.com/loi/pcgn20

The maps problem and the mapping problem:


Two challenges for a cognitive neuroscience
of speech and language
a
David Poeppel
a
Department of Psychology, New York University, New York, NY, USA
Version of record first published: 27 Sep 2012.

To cite this article: David Poeppel (2012): The maps problem and the mapping problem: Two challenges for a
cognitive neuroscience of speech and language, Cognitive Neuropsychology, 29:1-2, 34-55

To link to this article: http://dx.doi.org/10.1080/02643294.2012.710600

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Any substantial
or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or
distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the
contents will be complete or accurate or up to date. The accuracy of any instructions, formulae,
and drug doses should be independently verified with primary sources. The publisher shall not
be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or
howsoever caused arising directly or indirectly in connection with or arising out of the use of this
material.
COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 – 2), 34 –55

The maps problem and the mapping problem: Two


challenges for a cognitive neuroscience of speech and
language

David Poeppel
Department of Psychology, New York University, New York, NY, USA

Research on the brain basis of speech and language faces theoretical and empirical challenges. Most
Downloaded by [University of Arizona] at 07:01 09 January 2013

current research, dominated by imaging, deficit-lesion, and electrophysiological techniques, seeks to


identify regions that underpin aspects of language processing such as phonology, syntax, or semantics.
The emphasis lies on localization and spatial characterization of function. The first part of the paper
deals with a practical challenge that arises in the context of such a research programme. This maps
problem concerns the extent to which spatial information and localization can satisfy the explanatory
needs for perception and cognition. Several areas of investigation exemplify how the neural basis of
speech and language is discussed in those terms (regions, streams, hemispheres, networks). The
second part of the paper turns to a more troublesome challenge, namely how to formulate the
formal links between neurobiology and cognition. This principled problem thus addresses the relation
between the primitives of cognition (here speech, language) and neurobiology. Dealing with this
mapping problem invites the development of linking hypotheses between the domains. The cognitive
sciences provide granular, theoretically motivated claims about the structure of various domains (the
“cognome”); neurobiology, similarly, provides a list of the available neural structures. However, expla-
natory connections will require crafting of computationally explicit linking hypotheses at the right
level of abstraction. For both the practical maps problem and the principled mapping problem, develop-
mental approaches and evidence can play a central role in the resolution.

Keywords: Localization; Computation; Cognitive neuroscience; Linking hypothesis.

Research on the neurobiological foundations of neuroscience: maps of the brain and maps of brain
cognition, in general, and speech and language activation. This maps problem concerns the extent
processing, in particular, faces a variety of interest- to which spatial information about brain activity
ing empirical and theoretical challenges. Two pro- provides satisfactory descriptions of the neural
blems are discussed here, a practical one and a basis of perception and cognition. The techniques
principled one. The practical problem has to do that currently dominate the field (whether spatially
with how we should conceive of (one of) the specialized, such as fMRI, or temporally special-
main forms of data that lie at the basis of cognitive ized, such as MEG) predominantly characterize

Correspondence should be addressed to David Poeppel, Department of Psychology, NYU, 6 Washington Place, New York, NY
10003, USA. (E-mail: david.poeppel@nyu.edu).

34 # 2012 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business
http://www.psypress.com/cogneuropsychology http://dx.doi.org/10.1080/02643294.2012.710600
MAPS PROBLEM AND MAPPING PROBLEM

data in terms of spatial attributes (such as local and function, in a sense instrumentalizing the
topographic organization like retinotopy or soma- computational theory of mind (e.g., Pinker,
totopy, processing streams like dorsal versus 1997) more aggressively. We will need to seek
ventral pathways, or networks of interconnected theoretically well-motivated, computationally
brain regions). Thinking and talking about brain explicit, and biologically realistic characterizations
activity in spatial terms is very intuitive (spot A of function to take the work to the next level.
“does”/houses/executes function X, spot B does Some of the issues that arise in this central part
Y, etc.), is often more or less correct (at some of cognitive neuroscience, making maps, are dis-
level of description), and has captured both the cussed in the next section, with a view towards
professional the popular imaginations. Visually how maps implicate functions, and how a develop-
compelling publications such as Images of Mind mental perspective can sharpen the issues and help
(Posner & Raichle, 1997), Mapping the Mind construct productive links between structural
(Carter & Frith, 2000), or Portraits of the description and functional analysis.
Mind (Schoonover, 2010) elegantly summarize The principled problem deals with the “align-
Downloaded by [University of Arizona] at 07:01 09 January 2013

the successes of the research programme. They ment” between the putative primitives of cognition
also clearly reflect (in their titles as well as their and neurobiology and constitutes a more abstract
substantive content) the deeply held, implicit pre- challenge. Addressing this mapping problem—
supposition that making a visual, spatial map of what is the relationship between the “parts list”
mental faculties is a critical step in a proper expla- of cognition and the “parts list” of neurobiol-
natory account of the brain basis of psychological ogy?—is considerably more difficult than it
function. This stance is likely to be correct in might seem at first, ultimately requiring the devel-
part, but at best will reflect an incomplete under- opment of appropriate linking hypotheses between
standing; uncontroversially, localization and the different domains of study. I use the expression
spatial mapping are not explanation (e.g. mapping here not to refer to the assignment of
Poeppel, 2008). In a typical cognitive neuroscience putative linguistic or psychological function to
study of speech, lexical, or sentence-level proces- brain areas, be they distributed or localized, micro-
sing, participants will engage in some naturalistic scopic or macroscopic. Rather, I mean by mapping
or artificial task while their brain activity is moni- the investigation of the (ultimately necessary)
tored. The analyses show that some area or areas formal relations between two sets of hypothesized
are selectively modulated, and it is then argued inventories, the inventory constructed by the
that activation of a given area underpins, say, pho- language sciences and that constructed by the
nological processing or lexical access or syntax. neurosciences: How do the primitive units of
Although this superficial characterization under- analysis of the cognitive sciences map on to the
estimates the state of the field, it is fair to say primitive units of analysis of the neurosciences?
that the canonical results—very much at the The cognitive sciences, including linguistics and
center of current research—are correlational; that psychology, provide detailed analyses of the onto-
is to say, systematic relations consistently occur logical structure of various domains (call this the
between brain areas and some functions that “human cognome,” i.e. the comprehensive list of
reappear across studies, but we have no expla- elementary mental representations and oper-
nation, no sense in which properties of neuronal ations), and neurobiology provides a growing list
circuits that we understand account for the of the available neural structures. To exemplify,
execution of function. How to proceed? I suggest the infrastructure of linguistics—building on for-
that even high-resolution data from (existing or mally specified concepts such as syllable or noun
to be developed) new techniques will remain phrase or discourse representation—provides a struc-
inadequate unless we decompose the cognitive tured body of concepts that permit linguists and
tasks under investigation into computational pri- psychologists to make a wide range of precise gen-
mitives that can be related to local brain structure eralizations about knowledge of language that

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 35


POEPPEL

speakers bring to bear, about language acquisition, level computational and low-level implementational
online language processing, historical change, and characterizations.
so on. Similarly, the infrastructure of the neuro- In the next section, on maps, I outline four
sciences—drawing on units of analysis such as den- types of spatial organization that form the basis
drite or cortical column or long-term potentiation— for much of contemporary research on the brain
captures a variety of structural and functional fea- basis of speech and language: local regional organ-
tures of the brain, with profound consequences for ization of a given brain region (e.g. “Broca’s
the neural basis of cognition and perception. region”), the notion of processing streams (e.g.
However, how do the hypothesized units of analy- “dorsal” versus “ventral” streams), hemispheric
sis relate? Simple reductionist assumptions cannot asymmetries, and putative networks of connected
even be stated in any sensible way (e.g., “neuron ¼ brain areas. This “brain mapping” programme of
syllable”). Such alignments may seem potentially research, emphasizing different “functional ana-
pleasing at first glance, but that pleasure subsides tomic units of analysis” ranging from the micro-
quickly if one actually has to develop such a view scopic to the network level, has been productive
Downloaded by [University of Arizona] at 07:01 09 January 2013

and do the work of representation and compu- and successful and remarkably popular in the
tation. The fact of the matter is that we have public understanding of cognitive neuroscience—
very little to no idea as to how the stuff of but it needs to be considered what the next steps
thought relates to the stuff of brains, in the case could be to flesh out the promise of specifying
of speech and language—and virtually all other such spatial maps. After discussing the issues sur-
cases (Gallistel & King, 2011; Mausfeld, 2012). rounding the maps problem and suggesting some
In the third section, I argue that to link cognition questions for which developmental evidence can
and neurobiology in an explanatory fashion will and should play a critical role for progress, I turn
require the formulation of computationally explicit to the mapping (or alignment) problem in the
linking hypotheses at the right level of abstraction. third section, developing the argument as to why
With respect to this principled issue, too, develop- it is so difficult to establish the principled relations
mental data provide critical insights, for example between linguistic or psychological primitives and
in specifying the parts lists attributed to the neurobiological primitives. The paper closes with
infant from the start (innate, precompiled some tentative suggestions about how one might
toolbox) versus the representations and operations proceed in a way that could bridge the maps and
argued to be acquired and constructed. mapping problems, endorsing a computational
The guiding perspective throughout is that of a approach that attempts to identify a level of analy-
Marr-style approach (Marr, 1982). Separating sis that might facilitate at least the formulation of
computational, algorithmic, and computational linking hypotheses.
levels of analysis, rather than being anachronistic,
will be helpful in better defining the nature of the
challenges and pointing to some tentative answers, THE STANDARD RESEARCH
or at least plausible approaches to possible STRATEGY IN COGNITIVE
answers. Cognitive neuroscience has made NEUROSCIENCE: MAKING MAPS
immense progress in the past 20 years, and the
grounds for optimism are justified. However, Defining the spatial layout of brain organization
what constitutes an explanation is rarely has a long and successful history. In fact, perhaps
addressed. Adapting the position articulated by the first large-scale model of brain organization
Marr (and others) on the distinct but tightly in systems neuroscience was the so-called “connec-
yoked levels of description, I suggest that a focus tionist model” for language based on the seminal
on the algorithmic and representational level can insights by Broca (1861) and Wernicke (1874),
provide a potentially productive perspective in arguing that there are two localized centers that
seeking hypotheses that bridge between high- underpin language production and comprehension

36 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM

(Geschwind, 1970). (Though this model has been functions) and subsequent composition into
extremely influential to this day, especially in clini- higher-order, complex visual representations.
cal contexts, it is widely accepted that it remains (The model of vision espoused is ultimately under-
woefully underspecified, from both the linguistic specified and no longer the dominant view;
and neurobiological points of view. For discussion, however, it has been a remarkably productive
see, e.g., Poeppel & Hickok, 2004; Grodzinsky & empiricist paradigm.) Local features (e.g., retinal
Amunts, 2006.) Spatial organization of one form position, edges, luminance) are aggregated along
or another has been investigated profitably feedforward pathways, leading to the construction
at many levels of description, from determining of more complex visual representations (e.g., faces,
(i) at a microscopic level the features that specify as an obvious example). Three generalizations
local neural analysis within a brain region emerged from this work. First, decades of visual
(e.g., retinotopic mapping in the visual system) neurophysiology confirm that there is localization
to (ii) information processing streams (e.g., the of function: Neurons and ensembles of neurons
what versus where systems in vision) to (iii) soph- perform operations that are regionally specific.
Downloaded by [University of Arizona] at 07:01 09 January 2013

isticated network analyses (Sporns, 2010) that Cells in an early visual area such as primary
model how distributed regions are connected in visual cortex (V1) have firing properties that
support of function and (iv) attempts to under- reflect strikingly different receptive field size and
stand the human connectome (Seung, 2012); response selectivity than cells in a higher order
that is, the comprehensive mapping of how all area, say in the infero-temporal cortex. The oper-
areas are connected, cell by cell: the total wiring ations underlying visual decomposition (and com-
diagram of the human brain. position) are localized and specific. Second, the
Why does the search for finding a map with feedforward pathways along which the visual
local topographic organization induce such yearn- information is aggregated suggest hierarchical pro-
ing in the field? What underlies the seductive cessing. Neurons along a processing stream reflect
appeal of map-making? Maps of one form or ever more integrated (in space, time) and abstract
another are, of course, on target for sensory and properties of the external world. Third, there
motor systems, revealing fundamental organiz- exist concurrent pathways for information proces-
ational principles that have helped model aspects sing that underlie different larger-scale functions.
of perception and motor control. Searching for Originally, these were conceived as “what versus
topographies has been extremely successful in the where” streams (Mishkin, Ungerleider, &
case of vision (e.g., retinotopic organization of Macko, 1983); subsequently the discussion has
numerous visual areas), in the case of the somato- focused on object identification versus action, sen-
sensory system (the sensory homunculus), widely sorimotor transformation, and other visual tasks
attested in auditory processing (tonotopic (Goodale & Milner, 1992). Cumulatively, this
mapping), and so on. So, in a strategic sense, research on the neurobiological foundations of
aiming to understand the spatial organization of vision has profoundly influenced the way of
cognitive functions builds on well-established neu- asking questions about the neural basis of percep-
roscientific observations. tion and cognition.
The study of the visual system has formed the But it is not at all clear how such spatial think-
conceptual basis for many aspects of the standard ing plausibly extends to cognition, and especially
paradigm in cognitive neuroscience. Much of the language. Research on the brain basis of language
original work is elegantly summarized in a mono- has (implicitly) adopted the approach of vision,
graph by David Hubel, one of the chief architects going as it were “from the outside in,” assuming
of visual neuroscience: Eye, Brain, and Vision that sounds (or signs) are analysed in one region,
(Hubel, 1995). In that domain, it was assumed aggregated into morphemes in another, followed
that the external visual world is represented by by some sequential syntactic processing in yet
decomposition into elementary features (or basis another region, and so on. In fact, an enormous

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 37


POEPPEL

part of the neuroscience of language literature is grained computational decomposition of linguistic


dedicated to functional localization at a very high domains into elementary representations and com-
level, seeking to identify segregated regions for putations will be required.
speech perception, phonology, morphology,
syntax, semantics, etc. But it is an empirical ques-
Maps of regions
tion whether any spatial mapping principles as
they appear in sensory systems extend to cognition Two examples of brain regions are discussed. First,
or language. (Verbs here, nouns there? the canonical language region, Broca’s area, is
Phonemeotopy? Spatial distribution of syntactic examined. Second, we consider the contribution
categories? It is not even clear if such questions to language processing of visual areas located
can be properly formulated in a theoretically early in the processing hierarchy.
motivated context—although precisely these Subsequent to the original definition by Broca
examples have been discussed in the cognitive (the third frontal convolution of the inferior
neuroscience literature; Pulvermueller, frontal gyrus), the first detailed anatomic analysis
Downloaded by [University of Arizona] at 07:01 09 January 2013

Lutzenberger, & Preissl, 1999; Obleser, Elbert, of the inferior frontal gyrus was provided by
Lahiri, & Eulitz, 2003; Hillis & Caramazza, Brodmann (1909). He argued for a division of that
1995.) We have, of course, learned a great deal region into two or three subregions; typically, the
about the likely functional contribution of certain cytoarchitectonically distinct Brodmann areas 44
brain regions (e.g., the superior temporal gyrus), and 45 are included (some authors also count area
processing streams (e.g., dorsal and ventral 47). Roughly until 2000, generalizations about
streams in speech), hemispherically organized Broca’s area—whether deriving from imaging data
brain structures, and so on. The examples of or from neuropsychological deficit–lesion corre-
spatial organization raised in the next sections lation data—were made at best at that level of analy-
are intended to be forward looking, hoping to sis (see Grodzinsky & Amunts, 2006, for an
extend what spatial maps of the future might tell extensive review). That is to say, researchers made
us. The point to bear in mind throughout is that claims about “Broca’s area”—or on occasion about
the spatial organization of information-processing areas 44 versus 45—in the interpretation of their
systems can be a useful, and even necessary, inter- experimental data. The resolution of the analysis
mediate step in explaining a system, but even fan- was very rarely higher than that. There exist some
tastic localization of function, incorporating the interesting connectivity studies (e.g., Anwander,
newest techniques (e.g., multivoxel pattern classi- Tittgemeyer, von Cramon, Friederici, & Knösche,
fication of imaging data or various new approaches 2007), but the functional anatomic characterization
to electrophysiology), does not constitute an expla- has been coarse.
nation (Poeppel, 2008). The cartographic impera- New anatomic techniques (imaging as well as
tive does not suffice. It is the mechanistic immunocytochemistry) have been applied more
understanding of the function that we seek, and recently, and data in an influential paper by
that is not going to be tractable by localization of Amunts et al. (2010) show that the organization
function and spatial topographic mapping alone. of this one cortical region is much more
In what follows, I discuss four examples that complex, incorporating, depending how one
describe progress in speech and language research counts, 5 – 10 local areas that are distinguished by
in the context of spatial mapping studies and their cellular properties. Figure 1, taken from
that exemplify the challenge of relating brain from Amunts et al., illustrates the currently
maps to language processing. One lesson from hypothesized organization of Broca’s region, high-
the perspective developed here is that a traditional, lighting anterior and posterior as well as dorsal and
quasimodular view (assigning phonology here, ventral subdivisions of known fields and also
syntax there, etc.) cannot succeed, in part because pointing to the additional opercular and sulcal
the spatial studies suggest that much more fine- tissue that has been implicated.

38 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM
Downloaded by [University of Arizona] at 07:01 09 January 2013

Figure 1. The organization of Broca’s region (Amunts et al., 2010). Reproduced with permission. To view a colour version of this figure,
please see the online issue of the Journal.

Now, it is extremely likely that each subregion data structures native to the linguistic cognitive
performs at least one different local compu- system. We know next to nothing about precisely
tation—after all, they are anatomically distinct. what kind of conditions need to be met to be pro-
And it is worth remembering that we have not cessed by Broca’s region, and in particular by one
even scratched the surface of possible laminar of the many areas that constitute this part of the
differences within and across these cortical fields. inferior frontal cortex.
(Recall that cortex is a sheet of layers, each of So how to proceed? If other examples from the
which is distinct in terms of cell types, receptor central nervous system can serve as a guide (say the
distribution, inputs and outputs, etc. We know retina with its many cell types and associated func-
from the study of other regions, using animal tions, or primary visual cortex with its ocular dom-
models, that there are intricate feedforward and inance columns and orientation pinwheels),
feedback projection schemes, canonical cortical Broca’s region is likely either (i) to support a
circuits (Douglas & Martin, 1991), and a rich number of different types of operations/compu-
infrastructure mediating locally distinct organiz- tations, or (ii) to support similar operations on
ation.) Suppose then, for the sake of argument, very different input data structures to yield differ-
that there are merely five anatomic subdomains ent outputs. In either case, it is our job to acquire
in Broca’s region, and suppose that each anatomic and describe data at a much higher resolution than
subdomain supports two types of operation. These has been customary to do justice to the known
are rather conservative numbers, but we must then organizational complexity, at least at the level of
still identify 10 types of computation. (This is known anatomic distinctions as shown by
likely to underestimate vastly the actual complex- Amunts et al. (2010). (Imagine, for comparison,
ity.) Some of the computations will be applicable studying visual perception without acknowledging
to any input data structure (structured linguistic the role primary visual cortex plays in eye-specific
representation, speech signal, or other), since it is and orientation-selective processing; it would be
well established that Broca’s region is engaged by bizarre.) How can this be achieved? In the next
numerous cognitive tasks (Embick & Poeppel, decades, we will presumably have access to new
2006). Other computations may be dedicated to techniques with higher spatial resolving power.

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 39


POEPPEL

And existing but relatively new analytic techniques that visual areas play a role in processing visually
(say multivoxel pattern classification in the context presented language. There exist, of course,
of fMRI) also will provide a way to make necessary relations between linguistic representations and
anatomic distinctions. In the meantime, we will sensory representations in vision (text), but also
rely on invasive approaches (e.g. Sahin, Pinker, in hearing (speech), and touch (Braille). At stake
Cash, Schomer, & Halgren, 2009), animal is, rather, whether the visual activations observed
studies (which are, of course, limited in their in some recent studies can help illuminate the
value with respect to questions of speech and computational subroutines and constitutive rep-
language), and the coarser techniques that we cur- resentations that are invoked during language pro-
rently use in cognitive neuroscience. But notwith- cessing. The relevant concept that connects the
standing any technical progress, it is quite clear studies is predictive processing.
that generalizations of the type “Broca’s region How does language processing generate neural
supports syntax” or “Broca’s region supports hier- activity in early visual areas? Unlike Broca’s region,
archical processing” or “Broca’s region underpins early visual areas are decidedly not counted among
Downloaded by [University of Arizona] at 07:01 09 January 2013

verbal working memory” are idealizations or sim- the typical language areas. Yet recent MEG
plifications that are not just insufficient but poss- studies by Dikker, Pylkkanen, and colleagues
ibly misleading. What appears to be required is a show that syntactic and lexical – semantic cues in
much finer decomposition, and the question is the input affect neural activity in occipital cortex
what the most appropriate level of decomposition between 100 and 130 ms—that is to say, in very
might be. This is an issue discussed in the early visual areas (Dikker & Pylkkänen, 2011;
context of the mapping problem below, but the Dikker, Rabagliati, Farmer, & Pylkkänen, 2010;
challenges of the maps problem already point to Dikker, Rabagliati, & Pylkkänen, 2009). These
the nature of the matter. authors experimented with expected versus unex-
In acknowledging the architectural complexity pected word categories (syntactic predictions) in
of Broca’s region as depicted in Figure 1, it sentential contexts and showed enhanced activity
becomes clear that the classical model of brain in visual cortex by 120 ms. Similarly, lexical –
and language—Wernicke’s region (e.g., for com- semantic predictions affected early visual proces-
prehension), Broca’s region (e.g., for production), sing, again by 100 ms. The authors interpret
and the arcuate fasciculus connecting them— these findings, plausibly, in the context of predic-
must be abandoned (see Poeppel & Hickok, tive models of language processing: The brain gen-
2004 for discussion). Virtually any angle from erates (syntactic, semantic) predictions about
which one examines the classical model (linguistic, upcoming input whenever any prediction is poss-
psycholinguistic, neurobiological, computational) ible; some of these predictions might be actualized
undermines its utility. as form-based estimates available to sensory areas
Next, let us turn to another brain region, occi- (say, like an orthographic or even more basic
pital cortex, and a different type of “local region” code). In other words, these regions are not
issue. The early visual areas are not typically associ- language areas but do reflect consequences of the
ated with language processing, but recent research predictive nature of online language processing.
reveals how processing linguistic materials impli- A different type of predictive effect, one driven
cates very early visual regions and how language by linguistic experience and reflecting rather pro-
experience shapes early visual processing. found developmental plasticity, is demonstrated
Contrary to intuition, the data from neurophysiol- in electrophysiological results by Almeida,
ogy suggest that even unquestionably domain- Poeppel, & Corina (2012). They studied the
general regions such as early visual cortex partici- MEG responses in visual cortex elicited by poss-
pate in rather abstract aspects of processing, not ible and impossible gestures in signers and non-
just the superficial, feedforward perceptual analysis signers. Both subject populations showed
of input signals. To be clear: It is not surprising completely canonical responses to faces and

40 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM

inverted faces. However, when viewing signs and processed in visual cortex in these participants,
gestures, although both groups were able to suggesting that these areas can be recruited in
detect carefully crafted biological anomalies in addition to classical areas of the language
the gestures, the congenitally deaf signers network. Whatever the properties of the local
showed marked changes and facilitation (in visual neural circuits are, they can operate over lin-
terms of sensitivity, reaction time, and neural acti- guistic representations in a way that distinguishes
vation). That work provides new evidence that language processing from other tasks, suggesting
high-level visual – perceptual properties related to that the computations that comprise language pro-
human form processing can be detected within cessing are sufficiently generic to interface with
the first 100 ms of visual processing—and that those types of circuit.
this early processing has functional significance. What are we to conclude from such examples?
The data show modulatory effects within the Both classical language areas (Broca’s region) and
vicinity of primary visual cortex (V1), a neural putatively domain-general areas (early visual
region that historically has been considered to be cortex) are implicated in different forms of
Downloaded by [University of Arizona] at 07:01 09 January 2013

reactive to only low-level visual properties. The language task. Critically, it is not just the visual
altered response properties are driven by linguistic form of the materials that drive the low-level
experience. The findings derive from comparing activation. In the cases mentioned here, early
the visual processing capabilities of normal visual cortex is (re)organized to reflect sensitivity
hearing listeners with profoundly deaf individuals, to high-order properties of high-level category
whose necessary reliance on visual information information in one case and human-form-based
makes them plausible candidates for significant analysis in the other. In neither case do feedfor-
cortical reorganization of the visual system. ward analyses account for the patterns; rather,
Profoundly deaf signers show exquisite sensitivity predictive aspects of processing, based on knowl-
to the detection of these gestural manipulations, edge of language, appear to condition the neural
unlike normally hearing subjects. These data response properties. Similarly, (the subregions of)
suggest that sophisticated, rather abstract visual Broca’s region are not just driven by high-level
constraints may be realized remarkably early in language constructs; relatively low-level discrimi-
sensory processing streams. It is argued that nations implicate Broca’s region as well.
these constraints may be a reflection of the encod- Presumably it follows from this discussion of
ing of linguistic properties of signed languages, maps of regions that we must decompose linguis-
reflecting a visually driven internal forward tic computation into constituent elements that
model for human gestures that recruits even are much finer (and perhaps of a different kind)
primary visual areas. As in the other examples of than we are used to doing. For instance,
visual cortex activation, the interpretation centres arguing that a brain region forms the basis for
on the predictive nature of the language system, “syntax” or “phonology” fails to capture the com-
whether it is studied in written English or in putational structure that actually comprises some-
signed ASL. thing as richly structured as syntax (which is
In addition to accounts that are motivated by obviously not monolithic). Surely we will not
predictive operations in language processing and want to argue that, say, visual cortex “hosts”
their consequences for perceptuo-motor acti- syntax. But we are then required (as linguists,
vations, as above, there is intriguing imaging evi- psychologists, cognitive neuroscientists) to
dence for language processing proper being provide an analysis that captures which com-
executed in early visual areas, in congenitally ponent of syntactic computation in fact is compa-
blind adults (Bedny, Pascual-Leone, Dodell- tible with what is executed in visual cortex. In the
Feder, Fedorenko, Saxe, 2011; Bedny & Saxe, cases sketched out above, aspects of predictive
this issue). They show that phonological, seman- coding that lead to sensorimotor predictions are
tic, and syntactic information is differentially the likely suspects.

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 41


POEPPEL

Maps of streams speech acquisition. If the characterization of such


processing streams is on the right track, develop-
In vision, the concept of processing streams has
mental data will illuminate the role of these struc-
been immensely influential in shaping research
tures. For example, consider that the identification
since the 1980s. The existence of concurrent
of speech sounds and early lexical processing far
what, where, and how streams showed how
precedes their articulation. In fact, performance
certain large-scale computational challenges
in relatively complex language tasks—such as
(localization, identification, sensorimotor trans-
mapping novel word forms to visual referents,
formation/action– perception linking) could be
exploiting some remarkably high-level intona-
supported in parallel by streams consisting of hier-
tional cues—can be documented by six months
archically organized cortical regions. This archi-
of age (Shukla, White, & Aslin, 2011). Does
tecture provided a functional anatomic blueprint
that mean that the ventral stream is mature prior
for supporting parallel processes as well as the hier-
to the dorsal stream? More provocatively, does
archical elaboration of visual representations
this mean that articulatory information is not cau-
Downloaded by [University of Arizona] at 07:01 09 January 2013

within a stream (Goodale & Milner, 1992;


sally necessary for the establishment of phonologi-
Mishkin et al., 1983).
cal and lexical representations? Is dorsal stream
This key idea from systems neuroscience was
information merely modulatory? That speech
adopted and adapted for speech and language pro-
motor information plays an important modulatory
cessing in the past 10 years (Hickok & Poeppel,
role in perception is not under debate; however,
2004, 2007; Friederici, 2012; Rauschecker &
data on the development of these processing
Scott, 2009). Figure 2 illustrates one such model,
streams will sharpen the questions about what
constructed to account for a range of phenomena
information is necessary and sufficient for estab-
in speech processing. A ventral stream can be con-
lishing the speech system, in the learner.
sidered responsible for supporting the transform-
ation from auditory input to lexical
representations. A dorsal stream is primarily criti-
Maps of hemispheres: Lateralization of
cal for sensorimotor transformations. One note-
function
worthy feature, and a clear departure from
previous models, is that current data suggest Zooming even further out (than regions and
much more bilateral support for speech and streams), arguably the coarsest level of brain
language processing than was previously mapping is at the level of cerebral lateralization.
suggested. The ventral stream structures, in par- However, coarse or not, hemispheric asymmetry
ticular, are robustly bilateral. has been an issue at the very foundation of cogni-
As is true for vision, identifying objects is one tive neuroscience, and particularly in the context of
fundamental computational challenge in speech language processing. Since the very first insights by
perception. The other computational role Broca, Wernicke, Lichtheim, and others, the
(in speech), in the sense of Marr, is to provide strong lateralization of “language” processing to
the infrastructure for production. That is to say, the dominant hemisphere has been one of the hall-
the words we know must be able to make contact marks of brain organization.
with both articulatory representations and auditory If the analysis proceeds at the level of
representations. The dorsal stream regions appear “language,” as if it were an unanalysed whole, an
to host the operations that permit the seamless equivalently coarse biological analysis is commen-
translation from auditory type (e.g., spectrogram) surate—and well deserved. As soon as one begins
to articulatory type (e.g., motor code) to decompose language into its constituent
representations. domains and operations, the question of cerebral
Note how both of these computational chal- asymmetry becomes more complicated and
lenges link to some foundational questions of nuanced. For example, there is now emerging

42 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM
Downloaded by [University of Arizona] at 07:01 09 January 2013

Figure 2. Dorsal and ventral streams (Hickok & Poeppel, 2007). Reproduced with permission. To view a colour version of this figure, please
see the online issue of the Journal.

consensus that speech perception—roughly the linguistic information may be encoded in various
mapping from acoustic waveform to lexical rep- cortical regions in the left and right hemispheres,
resentation—is a bilaterally mediated subroutine but the computations that operate over the puta-
of language comprehension (cf. Figure 2). There tive representations appear to reflect lateralized
is also a growing body of evidence to suggest specializations.
that some aspects of lexical level processing One area of investigation in which there has
(especially lexical semantics) are executed in both been vigorous debate concerns the relative contri-
hemispheres (Federmeier, Wlotko, & Meyer, butions of left and right hemispheres to speech
2008). In contrast, syntactic processing and pro- perception proper. Given that speech perception
duction appear to be rather lateralized to the appears to be bilaterally mediated, what are the
dominant hemisphere. To caricature the pattern, two sides doing? Are they fully redundant?
the linguistic operations that appear to be robustly Completely different? Are there relative
associated with left hemisphere mechanisms are specializations?
“COM – COM –PRE – PRO”: combinatorics, To get a flavor for some of the recent hypoth-
composition, prediction, and production. Notice eses, it is helpful to recall what the perceptual
that these terms refer to operations or compu- system must ultimately achieve, with speech or
tations, not to representational primitives. To other auditory inputs. On one hand, the system
date there exists no compelling evidence that, must extract sequential items at a time resolution
say, distinctive features, or morphemes, or roots, that permits identifying the correct order of
or phrasal types are selectively lateralized. Stored parts. Words, in particular, comprise a sequence

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 43


POEPPEL

of relatively short segments, and correctly identify- two separate studies, the same lateral asymmetry
ing their order is a prerequisite for recognition (the was observed: The higher modulation rate
pest versus pets issue). The high time resolution gamma band responses correlated with left-hemi-
necessary to succeed demands neural circuits opti- sphere auditory structures, the lower rate theta
mized to integrate information over short time responses with the right auditory cortex
scales (on the order of tens of milliseconds). On (Figure 3). These data demonstrate that there
the other hand, listeners must be able to analyse exists some intrinsic asymmetry such that auditory
and use small frequency deviations, for example neuronal ensembles “hear” the world at different
in the context of suprasegmental prosody (the time scales simultaneously. This (implementational)
lunch! versus lunch? issue). Such changes demand asymmetry provides the basis for the (algorithmic)
a high frequency resolution and are often associ- multitime resolution approach that solves the
ated with longer time constants. Again, neural cir- (computational) integration – resolution tension
cuitry must be responsive to slower time constants for auditory signals.
and high spectral sensitivity. The output of both
Downloaded by [University of Arizona] at 07:01 09 January 2013

types of perceptual analysis must be in a form


Maps of the whole network
that can interface with the language system—let
us say for the sake of argument, lexical Recently, the neurosciences have witnessed sub-
representations. stantial enthusiasm for trying to map across local
It has been suggested, in the context of two as well as distant areas, or even the entire brain,
slightly different but closely related theories using increasingly sophisticated network analyses
(Poeppel, 2003; Zatorre, Belin, & Penhune, (e.g. Sporns, 2010), and building on the new tech-
2002), that right hemisphere auditory mechanisms niques and their improved ability to reveal connec-
show a higher sensitivity to spectral change and are tions at both fine and coarse scales. The appetite
optimized for longer time constants, whereas left for this type of work culminates in connectomics
lateralized mechanisms may reflect better temporal (Seung, 2011), the presupposition of which is
resolving power. The spectral versus temporal that understanding all the connections will lead
theory by Zatorre and colleagues and the asym- to understanding of how the mind works.
metric sampling in time (AST) theory by Attempting to map the connections of the brain
Poeppel appeal to slightly different attributes of is, surely, a laudable goal. What lies behind this
cortical neurons, but the models by and large surge in research effort? There are at least two
agree that some type of split must be performed strands of work that fuel the desire to map every-
(Zatorre & Gandour, 2008). It is fair to say that thing—and connect everything. One motivating
there is emerging consensus about right hemi- factor presumably derives from the remarkable
sphere properties. Whether the left hemisphere insights gleaned from the cutting-edge techniques
is optimized for rapid temporal integration or in neuroscience. Two-photon microscopy (Homma
short temporal sampling remains quite controver- et al., 2009) and optogenetics (Yizhar, Fenno,
sial (e.g., McGettigan & Scott, 2012). To test Davidson, Mogri, & Deisseroth, 2011) are gener-
whether such asymmetries were intrinsic properties ating stunning new data on local circuitry and con-
of the auditory system (and not just associated with nectivity. These are, it should go without saying,
processing overt stimuli), Giraud and colleagues not techniques easily applied to undergraduates
(2007) recorded concurrent EEG and fMRI data or babies . . . They are, however, unveiling in
while subjects were resting (no stimulus input). animal preparations obscenely high-resolution
They filtered the EEG data into bands commen- views on the structure and function of small local
surate with the time scales of interest (theta, regions of central nervous system. If it is possible
gamma) and used that electrophysiological to map local circuits with such clarity—and,
pattern (convolved with the hemodynamic indeed, such aesthetic beauty—then it stands to
response function) to query the fMRI data. In reason that it would be exciting to scale up such

44 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM
Downloaded by [University of Arizona] at 07:01 09 January 2013

Figure 3. FMRI experiments (Giraud et al. 2007). Reprinted from Neuron, 56(6), Anne-Lise Giraud, Andreas Kleinschmidt, David
Poeppel, Torben E. Lund, Richard S. J. Frackowiak & Helmut Laufs, Endogenous cortical rhythms determine cerebral specialization for
speech perpception and prodcution, pp. 1127– 1134, Copyright 2007, with permission from Elsevier. To view a colour version of this
figure, please see the online issue of the Journal.

approaches to whole brains. A second motivating 2000, respectively). But beyond being numerically
factor appears to lie in the belief that acquiring awesome, will a comprehensive map, the connec-
enormous amounts of data and mining them stat- tome of the human brain, form the basis for the
istically will yield new types of insight. There is an explanation of the human mind? Will the wiring
unending thirst for more and more data (the same diagram suffice? It seems unlikely. For example,
cannot be said for more and more theory) (for an the worm C. elegans has a nervous system that is
engaging perspective on the problem of data col- completely known and described, yet our under-
lecting versus theory constructing, in physics and standing of the function is embarrassingly
in linguistics, see Weinberg, 2009 and Chomsky, inadequate (Bargmann, 1993).

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 45


POEPPEL

It is therefore possible that that even someth- language (and other aspects of cognition)—or
ing as sophisticated and data-driven as the connec- causes? Are different subregions more or less sus-
tome will be at best an intermediate step—unless ceptible to modification by experience? Can we
we also make serious, theory-driven progress on use data from developmental cognitive neuro-
the cognome (the specification of the represen- science to begin to unpack the potential contri-
tational and computational parts list of human bution of distinct regions? Given the enormous
cognition) and function. Genomics has been a appetite for data from the infant brain—including
triumph of computational biology in terms of cap- for speech and language processing (e.g. Dehaene-
turing something truly numerically awesome, but Lambertz, Hertz-Pannier, & Dubois, 2006;
its real utility won’t be revealed until we under- Telkemeyer et al., 2009; Perani et al., 2011)—
stand the coding at a much more comprehensive one viable research strategy is to use knowledge
level (including, say, the proteome). of development to understand features of the
In conclusion, can making maps or defining brain (rather than the other way around).
topography ever meet the criterion of explanatory Turning from Broca’s region to the dual path-
Downloaded by [University of Arizona] at 07:01 09 January 2013

adequacy? Spatial maps are not explanations, so ways, there is now a growing body of fibre tracking
there is an important ingredient missing, a detailed evidence (largely from diffusion tensor imaging)
functional analysis. We need to begin to address that suggests that in two-day-old infants, many
the difficult link between maps and computation. ventral pathway structures are in place in an
This presumably requires computational theories adult-like configuration, while the dorsal stream
of a certain granularity; that is to say, what is loca- is anatomically underspecified (Friederici, 2012b;
lized is computations for which a certain circuit is Perani et al., 2011). Figure 4 illustrates some rel-
specialized (perhaps “canonical computations,” as evant data. This pattern is certainly consistent
recently discussed by Carandini, 2012). The with an interpretation that dorsal stream regions
maps we make will therefore be maps of have a critical role for production, or the linking
computations. of representations from temporal to frontal
regions by sensorimotor transformation. Note
that more than one dorsal stream projection is
The maps problem and developmental data
depicted in Figure 2. It has been argued that the
A variety of developmental questions intersect projection targeting Broca’s region is not in
with the map perspective outlined above (admit- place, whereas the more dorsal structures are avail-
tedly, from the naı̈ve point of view of a nondeve- able earlier in ontogeny. If this finding is correct, it
lopmentalist). I describe just three in brief below, reinforces the issue raised in the section above on
focusing on data that speak to three of the four Broca’s region: Does the fine-grained spatial struc-
map types. However, it is worth emphasizing ture visible there arise in development with the
that data collection in a developmental context “arrival” of the fibre tracts from temporal lobe
will also not suffice without theory building. In origins—and can we document concomitant per-
particular, computational theories will provide formance changes—or is it in place and merely
the most useful framework for addressing the modified as the fiber tracts mature? Many more
development of the cognitive and neural foun- normative data sets on the developing brain will
dations of perception and cognition (Aslin & be necessary to sort out such issues.
Fiser, 2005). Finally, consider hemispheric asymmetries.
Figure 1 depicts the organization of the adult There now exists a growing body of evidence
Broca’s region subsequent to normal language that right-hemisphere mechanisms are develop-
acquisition. An obvious question is whether the mentally slightly advanced. This developmental
infant brain is similarly structured. More specifi- asymmetry appears to be the case both for the
cally, are such cytoarchitectural subdivisions con- auditory and visual systems. Given that, by
sequences of the acquisition of speech and hypothesis, the local circuitry in the right

46 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM
Downloaded by [University of Arizona] at 07:01 09 January 2013

Figure 4. PNAS, dorsal stream developmen (Perani et al. 2011). Reproduced with permission. To view a colour version of this figure, please
see the online issue of the Journal.

hemisphere is different with respect to its proces- Given some of the properties that have been
sing preferences (not necessarily different in discussed concerning hemispheric asymmetry, the
kind, just in allocation of amount of neural ensem- dorsal and ventral streams, and the anatomic spe-
bles with certain properties, say slow temporal cification of Broca’s region, one can advance a
integration), this implementational difference pro- (simplified) developmental hypothesis arising
vides a way to address a few perceptual phenomena from these cognitive neuroscience descriptions.
that require explanation. For example, in the audi-
tory domain, it would be plausible for infants to 1. The right-hemisphere mechanisms, especially
show enhanced sensitivity to syllabically mediated in the superior temporal lobe portions of the
information as well as other, suprasegmental pro- ventral stream, are available slightly (weeks to
sodic phenomena, even in utero. The “universal” a few months) earlier in ontogenesis, perhaps
listener may start out with a syllable-based even in utero. Given the hypothesized specializ-
segmentation strategy. In general, larger units, ation of right temporal lobe mechanisms for
preferentially processed by right hemisphere longer integration time constants, the percep-
mechanisms, would be favored in early develop- tual information attended to preferentially
ment. Similarly, configural perceptual information by the learner occurs on these longer time-
should be preferred to locally detailed, high- scales. Units of approximately syllabic size/
resolution cues. duration—which corresponds closely to the

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 47


POEPPEL

envelope of spoken language—have epistemo- projections to the inferior frontal gyrus, includ-
logical priority at this stage; perceptual analysis ing Broca’s region, show a delayed develop-
at this temporal granularity is especially effec- mental trajectory (Perani et al., 2011,
tive at helping the learner segment the input Friederici, 2012). The structures of the dorsal
into units that correspond well to certain lin- stream play a critical role in sensorimotor trans-
guistic grouping principles (stress, conso- formation for speech. The seamless interaction
nant – vowel alternation, possibly even rhythm of the perception and production systems
class) (cf. Bertoncini & Mehler, 1981). is arguably coordinated by dorsal stream
2. Convergence by the learner on the phonetic structures, and it is therefore plausible that
repertoire of her language over the first year there is a tight association between the neural
of life (see, e.g., Kuhl, 2004 for review) development of the dorsal stream and
coincides with the increasing specification of the developments of speech production.
the structures underlying segmental phonol- Uncontroversially, perception precedes pro-
ogy. Since left-hemisphere mechanisms have duction, and it is reasonable to hypothesize
Downloaded by [University of Arizona] at 07:01 09 January 2013

been argued to reflect some degree of specializ- that this delay in part reflects the constraints
ation for short timescale processing, and dorsal imposed by a developing sensorimotor dorsal
stream structures are implicated in processing stream.
segmental phonology, this accords well with 4. If inferior frontal gyrus regions, and in par-
the requirement to analyse segmental and sub- ticular Broca’s region, receive a changing
segmental phonetic information. On balance, it innervation pattern over development because
is plausible to assume that the ventral stream the dorsal stream projecting there is not fully
structures are in place by approximately one developed, it is reasonable to conjecture that
year. The learner is now optimally positioned the functional specification of some of these
to analyse input at both syllabic and segmental regions changes (perhaps sharpens) over
timescales, concurrently. This combination time. It is not possible to conclude whether
constitutes critical ingredients for successful or not the many cytoarchitectural subregions
decoding of words—the second big milestone of the IFG (Figure 1) are available at birth,
the learner has to achieve (i.e., the incremental or whether this region becomes increasingly
construction of the mental lexicon). That is not specified over development. One can,
to say that younger infants cannot discriminate however, conclude that the computational
segmental information or fail to map auditory infrastructure is still changing over time, as
word forms to visual referents (see, e.g., the inputs to this rich and complex structure
Shukla et al., 2011). These tasks, though, change quite dramatically. A thoughtful
could be achieved by the learner based on rep- analysis of the development of speech and
resentations that are underspecified with its cerebral lateralization is provided by
respect to the linguistic inventory that is ulti- Minagawa-Kawai, Cristià, & Dupoux (2011).
mately specified for their native language; They discuss the interaction between signal-
that is to say, more acoustically driven, episodic driven lateralization versus lateralization as a
representations could form the basis for execut- function of learning biases and derive a
ing such tasks in the first few months of model of the acquisition of speech that
development. covers many of the phenomena raised here.
3. The structures of the dorsal stream—or at least It is worth noting that several of the subre-
one of the dorsal streams—are argued to gions of the IFG play critical functional role
develop later than those of the ventral stream. in the processing of complex sentential rep-
The dorsal stream projection from the tem- resentations, as well as in other domains of
poral lobe to premotor areas has been suggested language and nonlanguage processing.
to be present at birth. In contrast, the dorsal (Similarly, the thorny issues of semantics

48 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM

remain untouched in this conjectured scen- investigating. This is no different in kind from
ario.) The sequence outlined here is narrowly physics, where the field aims, for example, to
focused on the possible development of specify the elementary particles as well as the
speech processing. forces that condition their interaction—and these
two features of a theory provided an explanation
THE FUTURE OF COGNITIVE of the phenomena. Similarly, as neuroscientists
NEUROSCIENCE: MAKING we seek to determine the infrastructure of the
MAPPINGS nervous system, from subcellular components to
large-scale systems.
We now turn from making maps, which lies at the Let us make the problem a bit more concrete,
very foundation of cognitive neuroscience, to the building on the notion of the parts list. Suppose
problem of mappings. This latter problem refers we write to a large (even exhaustive) list of
to a very different way of thinking about the chal- speech and language researchers, asking what
lenges that the field faces. What is at stake is they consider to be the primitive elements of
Downloaded by [University of Arizona] at 07:01 09 January 2013

whether we are able to identify the possible their domain without which they could not
formal relations (mappings) between the putative account for the elementary phenomena of their
primitives of cognitive science, and in particular fields. In other words, what are the set of primi-
language research, and the putative primitives of tives that are absolutely required to provide an
neurobiology. Whereas the maps problem raises explanatory account of the canonical phenomena
a number of practical questions—for example, in speech perception, language comprehension,
what might be the right techniques to generate and production? An answer to such a query will
topographic maps at the right level of analysis— yield a parts list of representational elements as well
the mapping problem raises principled questions as a list of elementary functions, or computations.
about the nature of the relation between psychol- Figure 5 (left column) illustrates a hypothetical
ogy and neuroscience. This issue has been answer. An enumeration of primitives might
addressed in some detail by Poeppel and Embick include concepts such as distinctive feature, sylla-
(2005) for language and is, of course, raised by ble, morpheme, small clause, and variable. The
many investigators who grapple with the chal- elementary functions might include concatenation,
lenges at the interface of mind and matter recursion, variable binding, etc. Note that such a
(Smolensky & Legendre, 2006). list will be theory-dependent. A particular theory
might dispute the fundamental nature of distinc-
tive features or the relevance of recursion.
Granularity mismatch and the ontological
However, specific commitments of a given theory
incommensurability: Misaligned parts lists
are beside the point; any theory (of linguistics, psy-
One way to develop an intuition for the problem is cholinguistics, neurolinguistics) must be able to
to think about it in the context of specifying the provide an inventory of its constituent represen-
“parts lists” of the mind and brain. More formally tations and computations. Obviously, it is of par-
speaking, this refers to the goal of specifying the set ticular interest to develop a consensus position
of primitives, or the ontological structure, of the on what the “linguistic cognome” must necessarily
two domains in question. As cognitive scientists, contain to begin to get a grip on what is the onto-
we aim to determine an exhaustive list of the com- logical infrastructure of language research.
ponents of the human mind—call it the human Now, this very same exercise can be performed
cognome. We use the inventory of components to with neuroscientists. The right column in Figure 5
explain fundamental features of perception and illustrates the type of responses one might expect.
cognition; this is done by appealing to the hypoth- The goal there, too, is to identify an emerging con-
esized primitives and the rules that govern their sensus about the neurobiological units that form
interaction to yield the phenomena that we are the basis of central nervous system structure and

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 49


POEPPEL
Downloaded by [University of Arizona] at 07:01 09 January 2013

Figure 5. Sets of primitives for linguistics and neuroscience. First published in Poeppel, D., & Embick, D. (2005). Defining the relation
between linguistics and neuroscience. In A. Cutler (ed.), Twenty-first century psycholinguistics: Four cornerstones (pp. 103– 118).
Mahwah, NJ: Lawrence Erlbaum Associates. (Figure 1, p. 105.)

function. For the sake of argument, Figure 5 now computational cognitive science and compu-
provides a parts lists for the language sciences and a tational neuroscience. The central question will
parts list for the neurosciences. The evidence for be: What are plausible linking hypotheses
the primitives is extensive. That is to say, there is between the two domains; what form can they
a rich body of empirically based as well as theoreti- (or must they) take? Can cognitive neuroscience
cal argumentation to support every given member of speech and language processing move beyond
of these two “alphabets.” a purely correlational state of affairs?
The mapping problem is now quite obvious: There are (at least) two reasons why the
What is the relation between the ontological struc- formulation of linking hypotheses has been
ture of the language sciences and the neuro- slow—beyond the fact that it is rarely if ever
sciences? More colloquially, what are the lines/ acknowledged that there is a problem to begin
arrows that one can draw from one side to the with. One reason that it has proven difficult
other in Figure 5 that can capture the systematic derives from the level of analysis that is typical of
mapping from one set of elementary units to the different fields. Linguistic research comes at
another? The answer, too, is obvious: there is absol- the questions of interest with a toolbox that
utely no mapping to date that we understand in even permits a very high “resolution.” An area of
the most vague sense. There are no equivalence research as conceptually straightforward as word
relations, no isomorphisms, no easy mappings recognition approaches the issue by incorporating
from the theoretical infrastructure of the cognitive subtle features of phonology, the statistical adja-
sciences to that of neurobiology. On the assump- cency relations between sounds, the role of mor-
tion that one is adopting a nondualist research phological structure, the role syntactic categories
strategy, it is entirely unclear how to link the two may play for lexical access, nuanced theories
approaches (beyond stating some relatively gross about the semantics of words, theories exploiting
correlations). In what follows, a brief diagnosis the predictive nature of language processing, etc.
for this state of affairs is provided. Then a sugges- to understand aspects of facilitated or impeded
tion is made, in the spirit of Marr (1982) and lexical processing, and so on. That is to say, even
Gallistel & King (2011) about how to move a process as allegedly straightforward as recogniz-
forward, adopting an optimistic stance about ing a spoken word interfaces in a variety of ways

50 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM

with phonology, morphology, syntax, lexical 2005) suggests that the primitive elements of the
semantics, compositional semantics, and aspects two domains cannot be mapped onto each other
of discourse. In contrast, neurobiological research at all given the current formulation of the parts
on language attempts to address much broader lists. On such a view, the concepts that form the
questions. Typical issues that are investigated in basis of cognition and neurobiology are impossible
imaging studies, for example, include whether to align in principle. If that is the case, then there
syntax is localized or where the mental lexicon is is no opportunity for identifying plausible iso-
localized. This is, to be sure, not a problem that morphisms and certainly no opportunity for
cannot be overcome. It is a practical challenge and reduction (if that were a goal). Given—plausi-
simply reflects what has been called the granularity bly—that there is no hope to reduce perception
mismatch problem (Poeppel & Embick, 2005), to and cognition to neurobiology at present (and in
highlight the fact that the granularity of analysis the most primitive sense of reductionism), an
is too different to permit links that are more than alternative goal is to aim for unification
correlational. The techniques available to the (Chomsky, 2000) or consilience (Wilson, 1999).
Downloaded by [University of Arizona] at 07:01 09 January 2013

brain sciences investigate the issues in a manner The larger question thus becomes: What steps
commensurate with what can be measured and ana- need to be taken in terms of how we talk about a
lysed; the techniques available to the cognitive given domain in order for it to be unified with a
sciences attempt to capture generalizations about different domain? A historical example that illus-
speech and language processing at the highest poss- trates such a case comes from the relation
ible “conceptual resolution.” The practical conse- between chemistry and physics at the beginning of
quence of such a granularity mismatch is that the twentieth century. Chemistry was, contrary to
issues at very different levels of representation end intuition, not reduced to the principles of physics.
up being addressed. This is not intrinsically bad, Rather, the conceptual structure (ontology) of
but it is limiting in that it fails to move the field physics had to change to adjust to the new insights
forward in developing mechanistic linking hypoth- coming from the (putatively higher order) area of
eses between brain structure and function and the chemistry. Provocatively, a similar situation might
organization of this particular cognitive system. In present itself in the relationship between the brain
recent years, this has been changing in a positive and cognitive sciences (Chomksy, 2000; Gallistel
direction in the context of the ever closer relations & King, 2009). Perhaps what are considered
between theoreticians, cognitive scientists, and neu- elementary functional units in the neurosciences
robiologists applying state-of-the-art techniques. will change through closer consideration of what
In contrast to the granularity mismatch problem, the demands of perception and cognition are.
which can be overcome by forging closer links Conceivably, a representational primitive such as
between language researchers and brain researchers “neuron” may not end up playing as foundational
to better coordinate the nature of the questions that a role as some other functional units yet to be deter-
can be profitably addressed, there is another con- mined, even if it is presumably constructed out of
sideration that is a bit more insidious, and for the existing parts (say, for example, a canonical cor-
which a solution is more elusive. We seek to tical microcircuit; Douglas & Martin, 1991).
account for the phenomena of cognition by appeal- To summarize, the granularity mismatch
ing to the neurobiological infrastructure that lies at problem is a practical problem, a pragmatic
its basis. However, as is illustrated in Figure 5 and approach to which will play a productive role in
intimated above, there is no comprehensible way dealing with the maps problems addressed in the
in which we can link the two sides beyond corre- first section. The ontological incommensurability
lation—and this may reflect more than just a mis- problem reflects a principled problem, and to
match in the granularity of analysis. It may reflect make progress on this challenge, practitioners
a principled incommensurability. The ontological may need to consider how to reformulate the
incommensurability problem (Poeppel & Embick, nature of interdisciplinary research in this area—

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 51


POEPPEL

and question the commitments we make to the is as foundational and motivate research to seek
primitives of representation and computation. a neural circuit can execute the type of operation
known to be necessary for a variety of cognitive
tasks that are beyond debate. Why might such a
A tentative strategy for progress
(more muscular) stance of cognition be helpful?
One approach to explore can be called the “radical The field has taken for granted that the elemen-
decomposition” strategy. Adopting such a strategy tary units of neurobiology constitute the fact of
means questioning whether the primitives we hold the matter, the ground truth, from a functional
to be foundational are possibly decomposable into point of view. Neurons, dendrites, dendritic
finer-grained elements (that may themselves not spines, cortical columns, long-term potentiation
seem like natural units at first glance). For . . . The extremely impressive parts list of the
example, from the perspective of the cognitive neurosciences is not under debate. What one
sciences, consider a concept such as the might question, though, is whether the arrange-
“phoneme.” Many decades of research show that ment of units as they are currently discussed
Downloaded by [University of Arizona] at 07:01 09 January 2013

the phoneme is a bundle of more elementary units reflects the functionally appropriate arrangements
(features) and that generalizations about knowledge to account for the phenomena we know to require
of phonology are not made at the level of phonemes explanation. The interaction between the evi-
but at the level of features. Similarly, consider the dence coming from cognitive science theory and
tension between a theoretically poorly motivated experimentation, on one hand, and the practice
concept such as “word” compared to a theoretically and interpretation of neurobiology, on the
richly supported concept such as “morpheme.” other, needs to be significantly more synergistic
Whereas morphemes play an essential role in to go beyond correlation and develop genuinely
accounting for a range of properties of lexical struc- explanatory models.
ture and lexical access, an informal notion such as At this point it is worth considering once more
“word” has been largely unsuccessful. By analysing the research strategy outlined by Marr and others.
traditional concepts at a higher resolution, the The computational level of analysis, provided by
field has successfully identified smaller (or different) linguistics, psychology, and aspects of computer
units that appear to do a better job at capturing the science, must be linked to the implementational
phenomena under investigation. level of analysis, neurobiology. What was suggested
What kind of smaller—or different—units by Marr is that an intermediate (algorithmic) level
might be plausible? One strategy is to identify of description specifies the representations and
representations and operations that can be computations that are executed by the implementa-
linked to the types of operation that simple elec- tional circuitry to form the basis of the compu-
trical circuits can execute. One might begin, for tational level of characterization. If this strategy is
example, with theoretically well-motivated units on the right track, current research should in part
of representation or processing deriving from focus on the operations and algorithms that under-
cognitive science research (here, say linguistics); pin language processing. The commitment to an
then one attempts to decompose these into algorithm or computation in this domain
elementary constituent operations that are for- commits one to representations of one form or
mally generic (something like “concatenation,” another with increasing specificity and also pro-
for example). Now, in a role reversal, linguists vides clear constraints for what the neural circuitry
should challenge neurobiologists to define and must accomplish. The kinds of operation that
characterize the neural circuitry that can underpin might provide the basis for investigation include
something as elementary as concatenation. concatenation, segmentation, combination, label-
Instead of seeking validation of the hypothesized ling, and other elementary (and generic) operations
units from cognition by putative reduction to that could be implemented quite straightforwardly
neuroscience, assume that the cognitive evidence in neural circuits.

52 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM

A developmental wedge properties of the brain account for perception and


cognition—not just descriptions. The state of the
On the view sketched here, a prerequisite to
art is impressive, and perhaps even descriptively ade-
develop principled linking hypotheses between
quate, but our accounts of the relation between brain
(linguistic) cognition and neurobiology is the defi-
and cognition remain in large part correlational.
nition of the parts list, or the cognome.
Cognitive neuroscience ought to be more ambitious
Developmental research has a central role to play
than that, striving for explanations. A big issue in
in this research endeavor. No area of psychological
this context is whether we are investigating cogni-
investigation deals more explicitly with the ques-
tion at the right granularity, if our goal is to
tion of what the elementary constituents of the
develop linking hypotheses to neurobiology. If we
mind are. An essential perspective on the question
stick to cognition per se, and we achieve adequate
of the ontological primitives comes from the
explanations, then perhaps the analyses are on the
developmental literature, and specifically the ques-
right track. However, a more radical decomposition
tion “what is the parts list at the start?” It would be
approach might be necessary to develop successful
Downloaded by [University of Arizona] at 07:01 09 January 2013

of tremendous value to the enterprise of mapping


linking hypotheses.
from cognition to neurobiology if developmental
Many of the issues that lie at the centre of current
approaches could integrate questions about the
research are extremely relevant, and only a slight
computational constituents. For example, instead
shift in perspective can provide a different view on
of framing a developmental linguistic question in
how they help address the linking challenge. For
terms of the domain (e.g., speech, phonology) or
example, making maps, or localization of function,
the tasks employed (e.g., is phonetic categorization
continues to be extremely influential; but localiz-
evident in the neonate?), if such questions could be
ation of function might better be spelled out as local-
characterized in terms of their elementary con-
ization of local computation—where computation is
stituents it would license a claim about the
now construed to be the midlevel, generic set of
elementary computational infrastructure. Much
operations that underlie cognitive representation of
as in adult cognitive neuroscience research,
computation. Similarly, the relentless enthusiasm
describing the nature of the perceptual and cogni-
for prediction in perception and cognition, now
tive tasks not as tasks but as a set of primitive oper-
largely subsumed under Bayesian approaches, exem-
ations and representations will allow the
plifies the types of elementary computation that
formulation of linking hypotheses that translate
might hold in different parts of perception and cog-
more seamlessly to computational neuroscience.
nition, linguistic or otherwise.
There are, to be sure, good examples of the direc-
tion that can be taken. A lot of excellent work in
vision research demonstrates how tightly one can
REFERENCES
link theoretically motivated and computationally
explicit theories of visual perception to detailed Almeida, D., Poeppel, D., & Corina, D. (2012). The
neural models that link these domains. My own brain tuning to the body: Cortical plasticity and the per-
favorite example comes from the auditory research ceptual encoding of gestures. Manuscript submitted for
area of sound localization. The avian (e.g., barn publication.
owl) versus mammalian (e.g., hamster) brain solve Amunts, K., Lenzen, M., Friederici, A. D., Schleicher,
the same sound localization problem (computational A., Morosan, P., Palomero-Gallagher, N., &
level of analysis) using different algorithms (delay Zilles, K. (2010). Broca’s region: Novel organiz-
ational principles and multiple receptor mapping.
line model versus phase) using specialized circuits
PLoS Biology, 8, e1000489.
that implement only one kind of computation and Anwander, A., Tittgemeyer, M., von Cramon, D. Y.,
not another—that is to say, the circuits explain the Friederici, A. D., & Knösche, T. R. (2007).
computation (Grothe, 2003). The overall goal, pre- Connectivity-based parcellation of Broca’s area.
sumably, remains to provide explanations of how the Cerebral Cortex, 17, 816–825.

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 53


POEPPEL

Aslin, R. N., & Fiser, J. (2005). Methodological chal- Federmeier, K. D., Wlotko, E. W., & Meyer, A. M.
lenges for understanding cognitive development in (2008). What’s “right” in language comprehension:
infants. Trends in Cognitive Sciences, 9, 92–98. ERPs reveal right hemisphere language capabilities.
Bargmann, C. I. (1993). Genetic and cellular analysis of Language and Linguistics Compass, 2, 1–17.
behavior in C. elegans. Annual Review of Neuroscience, Friederici, A. D. (2012a). The cortical language circuit:
16, 47–71. From auditory perception to sentence comprehen-
Bedny, M., Pascual-Leone, A., Dodell-Feder, D., sion. Trends in Cognitive Sciences, 16, 262–268.
Fedorenko, E., & Saxe, R. (2011). Language proces- Friederici, A. D. (2012b). Language development and
sing in the occipital cortices of congenitally blind the ontogeny of the dorsal pathway. Frontiers in
adults. Proceedings of the National Academy of Sciences Evolutionary Neuroscience, 4. Advance online publi-
of the United States of America, 108, 4429–4434. cation. Retrieved May 31, 2012. doi:10.3389/
Bertoncini, J., & Mehler, J. (1981). Syllables as units in fnevo.2012.00003
infant speech perception. Infant Behavior and Gallistel, C. R., & King, A. P. (2011). Memory and the
Development, 4, 247–260. computational brain: Why cognitive science will trans-
Broca, P. (1861). Perte de la parole, ramollissement form neuroscience. Hoboken, NJ: Wiley.
Downloaded by [University of Arizona] at 07:01 09 January 2013

chronique et destruction partielle du lobe antérieur Geschwind, N. (1970). The organization of language
gauche du cerveau. Bulletin de la Société and the brain. Science, 170, 940.
d’Anthropologie de Paris, 2, 235–238. Giraud, A. L., Kleinschmidt, A., Poeppel, D., Lund, T. E.,
Brodmann, K. (1909). Vergleichende Lokalisationslehre Frackowiak, R. S., & Laufs, H. (2007). Endogenous
der Grosshirnrinde in ihren Prinzipien dargestellt auf cortical rhythms determine cerebral specialization for
Grund des Zellenbaues. Leipzig: Johann Ambrosius speech perception and production. Neuron, 56,
Barth Verlag. 1127–34.
Carandini, M. (2012). From circuits to behavior: A Goodale, M. A., & Milner, A. D. (1992). Separate
bridge too far? Nature Neuroscience, 15, 507–509. visual pathways for perception and action. Trends in
Carter, R., & Frith, C. D. (2000). Mapping the mind. Neurosciences, 15, 20–25.
Berkeley, CA: University of California Press. Grodzinsky, Y., & Amunts, K. (2006). Broca’s region
Chomsky, N. (2000). New horizons in the study of (illustrated ed.). Oxford, UK: Oxford University Press.
language and mind. Cambridge, UK: Cambridge Grothe, B. (2003). New roles for synaptic inhibition in
University Press. sound localization. Nature Reviews Neuroscience, 4,
Dehaene-Lambertz, G., Hertz-Pannier, L., & Dubois, J. 540–550.
(2006). Nature and nurture in language acquisition: Hickok, G., & Poeppel, D. (2004). Dorsal and ventral
Anatomical and functional brain-imaging studies in streams: A framework for understanding aspects of
infants. Trends in Neurosciences, 29, 367–373. the functional anatomy of language. Cognition, 92,
Dikker, S., & Pylkkanen, L. (2011). Before the N400: 67 –99.
Effects of lexical –semantic violations in visual Hickok, G., & Poeppel, D. (2007). The cortical organ-
cortex. Brain and Language, 118, 23–28. ization of speech processing. Nature Reviews
Dikker, S., Rabagliati, H., Farmer, T. A., & Pylkkänen, Neuroscience, 8, 393–402.
L. (2010). Early occipital sensitivity to syntactic Hillis, A. E., & Caramazza, A. (1995). Representation
category is based on form typicality. Psychological of grammatical categories of words in the brain.
Science, 21, 629–634. Journal of Cognitive Neuroscience, 7, 396–407.
Dikker, S., Rabagliati, H., & Pylkkänen, L. (2009). Homma, R., Baker, B. J., Jin, L., Garaschuk, O.,
Sensitivity to syntax in visual cortex. Cognition, Konnerth, A., Cohen, L. B., et al. (2009). Wide-
110, 293–321. field and two-photon imaging of brain activity with
Douglas, R. J., & Martin, K. A. (1991). A functional voltage- and calcium-sensitive dyes. Philosophical
microcircuit for cat visual cortex. Journal of Transactions of the Royal Society B: Biological
Physiology, 440, 735–769. Sciences, 364, 2453–2467.
Embick, D., & Poeppel, D. (2006). Mapping syntax Hubel, D. (1995). Eye, brain, and vision. New York, NY:
using imaging: Problems and prospects for the Scientific American Library.
study of neurolinguistic computation. In K. Brown Kuhl, P. K. (2004). Early language acquisition: Cracking
(ed.), Encyclopedia of language and linguistics (2nd the speech code. Nature Reviews Neuroscience, 5,
ed.). New York, NY: Elsevier. 831–843.

54 COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2)


MAPS PROBLEM AND MAPPING PROBLEM

Marr, D. (1982). Vision: A computational approach. event-related potentials and high-frequency cortical
San Francisco, CA: Freeman. responses. Cerebral Cortex, 9, 497–506.
Mausfeld, R. (2012). On some unwarranted tacit Rauschecker, J. P., & Scott, S. K. (2009). Maps and
assumptions in cognitive neuroscience. Frontiers in streams in the auditory cortex: Nonhuman primates
Psychology, 3, 67. illuminate human speech processing. Nature
McGettigan, C., & Scott, S. K. (2012). Cortical asym- Neuroscience, 12, 718–724.
metries in speech perception: What’s wrong, what’s Sahin, N. T., Pinker, S., Cash, S. S., Schomer, D., &
right and what’s left? Trends in Cognitive Sciences, Halgren, E. (2009). Sequential processing
16, 269–276. of lexical, grammatical, and phonological infor-
Minagawa-Kawai, Y., Cristià, A., & Dupoux, E. (2011). mation within Broca’s area. Science, 326, 445–449.
Cerebral lateralization and early speech acquisition: Schoonover, C. E. (2010). Portraits of the mind:
A developmental scenario. Developmental Cognitive Visualizing the brain from antiquity to the 21st
Neuroscience, 1, 217–232. century. New York, NY: Abrams.
Mishkin, M., Ungerleider, L. G., & Macko, K. A. Seung, S. (2011). Connectome: How the brain’s wiring
(1983). Object vision and spatial vision: Two makes us who we are. Boston, MA: Houghton
Downloaded by [University of Arizona] at 07:01 09 January 2013

cortical pathways. Trends in Neurosciences, 6, Mifflin Harcourt.


414–417. Shukla, M., White, K. S., & Aslin, R. N. (2011).
Obleser, J., Elbert, T., Lahiri, A., & Eulitz, C. (2003). Prosody guides the rapid mapping of auditory word
Cortical representation of vowels reflects acoustic forms onto visual objects in 6-mo-old infants.
dissimilarity determined by formant frequencies. Proceedings of the National Academy of Sciences of the
Cognitive Brain Research, 15, 207–213. United States of America, 108, 6038.
Perani, D., Saccuman, M. C., Scifo, P., Anwander, A., Smolensky, P., & Legendre, G. (2006). The harmonic
Awander, A., Spada, D., et al. (2011). Neural mind: From neural computation to optimality-theoretic
language networks at birth. Proceedings of the grammar. Cambridge, MA: MIT Press.
National Academy of Sciences of the United States of Sporns, O. (2010). Networks of the brain. Cambridge,
America, 108, 16056–16061. MA: MIT Press.
Pinker, S. (1997). How the mind works. New York, NY: Telkemeyer, S., Rossi, S., Koch, S. P., Nierhaus, T.,
Norton. Steinbrink, J., Poeppel, D., et al. (2009). Sensitivity
Poeppel, D. (2003). The analysis of speech in different of newborn auditory cortex to the temporal
temporal integration windows: Cerebral lateraliza- structure of sounds. Journal of Neuroscience, 29,
tion as ‘asymmetric sampling in time’. Speech 14726–14733.
Communication, 41, 245–255. Weinberg, S. (2009). Lake views: This world and the
Poeppel, D. (2008). The cartographic imperative: universe. Cambridge, MA: Harvard University Press.
Confusing localization and explanation in human Wernicke, C. (1874). Der aphasische Symptomencomplex:
brain mapping. In Bildwelten des Wissens: Vol. 6.1. Eine psychologische Studie auf anatomischer Basis.
Bildwelten des wissens (p. 13). Berlin, Germany: Breslau, Poland: Cohn & Weigert.
Akademie Verlag. Wilson, E. O. (1999). Consilience: The unity of knowl-
Poeppel, D., & Embick, D. (2005). Defining the edge. New York, NY: Random House Digital.
relation between linguistics and neuroscience. In A. Yizhar, O., Fenno, L. E., Davidson, T. J., Mogri, M., &
Cutler (ed.), Twenty-first century psycholinguistics: Deisseroth, K. (2011). Optogenetics in neural
Four cornerstones (pp. 103–118). Mahwah, NJ: systems. Neuron, 71, 9–34.
Lawrence Erlbaum Associates. Zatorre, R. J., & Gandour, J. T. (2008). Neural specializ-
Poeppel, D., & Hickok, G. (2004). Towards a new ations for speech and pitch: Moving beyond the
functional anatomy of language. Cognition, 92, 1–12. dichotomies. Philosophical Transactions of the Royal
Posner, M. I., & Raichle, M. E. (1997). Images of mind. Society of London B: Biological Sciences, 363, 1087–1104.
New York, NY: Scientific American Library. Zatorre, R. J., Belin, P., & Penhune, V. B. (2002).
Pulvermüller, F., Lutzenberger, W., & Preissl, H. (1999). Structure and function of auditory cortex: Music
Nouns and verbs in the intact brain: Evidence from and speech. Trends in Cognitive Sciences, 6, 37–46.

COGNITIVE NEUROPSYCHOLOGY, 2012, 29 (1 –2) 55

You might also like