Part Six: Formal Neurosemantics: Neurolinguistics

NEUROLINGUISTICS
lambdas
part six:
Formal Neurosemantics
logic, meaning and composition in the brain
Plato
Aristotle
Montague
Panizza, 2011
LANGUAGE
a code exploited by humans in order to communicate in a very fast and efficient way
linguistic tokens
(words)
more complex structures

(sentences)
syntactic
rules
agents of communication share
meaning of
each word
(lexicon)
extra-linguistic concepts about the world

(conceptual and world knowledge)
is something missing in this picture?
linguistic tokens
(words)
Panizza, 2011
syntactic
rules
meaning of
each word
(lexicon)

(sentences)

linguistic tokens
(words)
syntactic
rules
meaning of
each word
(lexicon)
Panizza, 2011

(sentences)

set of representation and operations
grammar-driven
derived from syntactic structure
abstract
not tied to specific entity or sensorial modality
symbolic
manipulated through logical (mathematical, formal)
operations
Panizza, 2011
Alfred Tarski
To know the meaning of a sentence is to know its truth conditions
the snow is white is true if and only if the snow is white

sentence
Donald Davidson
formal theories of meaning

reveal very little about the
conditions under which
individual sentences are true
goal: to relate the known
truth conditions of each
sentence to those aspects
(words) that recur in other
sentences and can be
assigned identical roles..
how we use it
Donald Davidson
formal theories of meaning

reveal very little about the
conditions under which
individual sentences are true
goal: to relate the known
truth conditions of each
sentence to those aspects
(words) that recur in other
sentences and can be
assigned identical roles..
Panizza, 2011
language presents regularities
implicit knowledge that speakers

have of their native language
(speaker competence, speaker intuitions
call it Grammar)
Panizza, 2011
there is some kid who is eating a sandwich

conceptual mapping between
content words
how can we capture the meaning of there is, some, who and a?
mapping between natural language and formal language: syntax + formal semantics
some
D. Panizza
Journal of Neurolinguistics
introduces
the/ existence
of entities xxx (2010) 129
uated; b) sets of situations where sentence (10a) is true, after negation

(4) dxdy [kid(x) ^ sandwich(y) ^ eat (x,y)]
logical formFormula
= explicit(4)
representation
of conditions
under
which
a sentence
is true
states that (3)
is true if and
only
if there
is an individual
x wh
where
entities
arey.made
formallyas
explicit
individual
y logical
that isrelationships
a sandwichbetween
such that
x eats
Therefore,
soon as o
ere
Panizza, 2011
D. Panizza
/ Journal of Neurolinguistics xxx (2010) 129
there is some kid who is eating
a sandwich
(4) dxdy [kid(x) ^ sandwich(y) ^ eat (x,y)]
Formula (4) states that (3) is true if and only if there is an individual x who is a kid, an
individual y that is a sandwich such that x eats y. Therefore, as soon as our model pr
it isstanding
true if and
only
if there
is an individual
x
in an
eating
relation
with a sandwich,
(3) is true.
who isToa kid
there
is an
y that
get and
some
grasp
forindividual
the power
of this method, let us consider slightly more comp
is aincluding
sandwichphrasal
such asconnectives.
x eats y
(5)
a Some boy is eating a sandwich or an orange.

b dxdydz [kid(x) ^ sandwich(y) ^ orange(x) ^ (eat(x,y) n eat(x,z))]
(6)
a Some boy is eating a sandwich and an orange.

TRUE!
b dxdydz [kid(x) ^ sandwich(y) ^ orange(z) ^ (eat(x,y) ^ eat(x,z))]
First, to know under which conditions sentences such as (5) and (6) are true, we n
implement(10a)
the meaning
of the connectives
or and
in our formal
language.
What does(10
thi
sentence
is evaluated;
b) sets
ofand
situations
where
sentence
semantics is the truth functions, which takes truth values as input, and yields a truth value
may rely on rst-order predicate logic and assign to and and or the truth tables of logical co
Formula (4) states that (3) is true if and only if there is an individual x who is a kid, and there is an
individual
individual yy that
that isis aa sandwich
sandwich such
such that
thatxxeats
eatsy.y.Therefore,
Therefore,as
assoon
soonasasour
ourmodel
modelprovides
providesa aboy
boy
standing
in
eating
Formal
standing
in an
anNeurosemantics
eatingrelation
relationwith
withaasandwich,
sandwich,(3)
(3)isistrue.
true.
Panizza,more
2011
To get some grasp for the power of this method, let us consider slightly
complex sentences
To get some grasp for the power of this method, let us consider slightly more complex sentences
including phrasal connectives.
Some boy is eating a sandwich or an orange
(5) a Some boy is eating a sandwich or an orange.

(5)
Some boy is eating a sandwich or an orange.
a) ba dxdydz
[kid(x) ^ sandwich(y) ^ orange(x) ^ (eat(x,y) n eat(x,z))]
(6) a Some
boy
eating
a sandwich
and an
orange.
boyisisis
eating
a sandwich
an orange
(6) a Some
Some boy
eating
a sandwich
and and
an orange.
[kid(x) ^ sandwich(y) ^ orange(z) ^ (eat(x,y) ^ eat(x,z))]
b) bb dxdydz
dxdydz [kid(x) ^ sandwich(y) ^ orange(z) ^ (eat(x,y) ^ eat(x,z))]
logical
entailment
First, to know under which conditions sentences such as (5) and (6) are true, we need a way to
First, tothe
know
underofwhich
conditions
sentences
such
as (5)language.
and (6) What
are true,
we
need
aformal
way to
implement
meaning
the
connectives
or
and
and
in
our
formal
does
this
job
in
we need
ameaning
way to of
implement
the meaning
of
implement
the
the
connectives
or
and
and
in our
formaland
language.
this
in formal
semantics is the truth functions, which takes truth values
as input,
yields aWhat
truthdoes
value
as job
output.
We
or andisand
ourfunctions,
formal language
semantics
the in
truth
takes
truth
asor
input,
and yields
value
as output.
We
may
rely on rst-order
predicate which
logic and
assign
tovalues
and and
the truth
tables aoftruth
logical
conjunction
and
may rely on respectively,
rst-order predicate
andbeing
assign
to and
and
the
truth tables
logical
conjunction
and
disjunction,
with thelogic
former
true
when
allor
the
conjuncts
are of
true
and the
latter being
disjunction,
with
the former
being
true
all thetruth
conjuncts
are rst
true part
and of
thetheir
latter
being
truth
functions:
truth
values
true
when
at respectively,
least
one of the
disjuncts
is true.
Now
we when
can compare
(5) to values
(6). The
logical
true when
at least
one is
ofat
the
disjuncts
Now ywe
can
(6). The
rst
their
logical
form
is identical.
There
least
a kid x,isa true.
sandwich
and
ancompare
orange z.(5)
Thetosecond
part
ofpart
(5b)of
says
that
the
form
is identical.
There isand
at least
a kid x, a sandwich
andsays
an orange
Thealso
second
partone
of (5b)
says that
the
kid
eats
both the orange
the sandwich,
whereasy(6b)
that hez.can
eat just
of them.
Thus,
first
order
eats
both
orange
andfrom
the sandwich,
whereas
(6b) says
thatand
he
can
also
one
of them.
Thus,
eating
athe
sandwich
eating
an
orange
or(5b)
and
akid
straight
intuition
coming
the
comparison
between
(6b)
is eat
thatjust
the
former
is more
logicformer is more
a straight than
intuition
coming
from
the comparison
(5b)
and (6b) in
is which
that the
restrictive
the latter.
More
precisely,
(5b) is true between
in a subset
of situations
(6b) is true, hence
T(5b) is true
T eating
Tboth a sandwich
T the latter.
restrictive
than
MoreRoughly
precisely,
in aissubset
of situations
in which (6b)
is true,
hence
the
former is
logically
stronger.
speaking,
if there
a boy
and an
orange,
it
the former
is logically
stronger.
Roughly
speaking,
ifthem.
there This
is aTboy
eating
both a sandwich
and
an orange,
it
plainly
follows
that
there
is
also
a
boy
eating
one
of
meaning
relation
that
holds
between
(6a)
F
F
T
plainly
follows
that
there
is
also
a
boy
eating
one ofanother
them. This
meaningBrelation
thatofholds
between- i.e.
(6a)
and (5a) is called entailment. A proposition A entails
proposition
if the truth
A implicates
TbetweenF(5a)
and (5a)entails
is called
entailment.
AMost
proposition
B if
the(6a)
truth
of Aany
implicates
- i.e.
T - the
FA entails
ensures,
truth of B.
importantly,
thisanother
relationproposition
and
- like
other types
ensures,
- the truth
of B. sentences
Most importantly,
this relation
between
(5a)in
and
(6a) -they
like are
anyevaluated.
other types
of
formalentails
relationships
among
- holds regardless
of
the context
which
F
F
F
F- holds regardless of the context in which they are evaluated.
of formal
relationships
among between
sentences(5a)
Namely,
the
entailment relation
and (6a) is due to the (logical) meaning of the connectives or
Namely,
entailment
(5a)inand
(6a)this
is due
to the (logical)
meaning
of the connectives or
and
and, the
rather
than on relation
the kindbetween
of scenario
which
sentence
is uttered
and veried.
Formula (4) states that (3) is true if and only if there is an individual x who is a kid, and there is an
individual
individual yy that
that isis aa sandwich
sandwich such
such that
thatxxeats
eatsy.y.Therefore,
Therefore,as
assoon
soonasasour
ourmodel
modelprovides
providesa aboy
boy
standing
in
eating
Formal
standing
in an
anNeurosemantics
eatingrelation
relationwith
withaasandwich,
sandwich,(3)
(3)isistrue.
true.
Panizza,more
2011
To get some grasp for the power of this method, let us consider slightly
complex sentences
To get some grasp for the power of this method, let us consider slightly more complex sentences
Some boy is eating a sandwich or an orange
(5) a Some boy is eating a sandwich or an orange.

(5)
Some boy is eating a sandwich or an orange.
a) ba dxdydz
[kid(x) ^ sandwich(y) ^ orange(x) ^ (eat(x,y) n eat(x,z))]
(6) a Some
boy
eating
a sandwich
and an
orange.
boyisisis
eating
a sandwich
an orange
(6) a Some
Some boy
eating
a sandwich
and and
an orange.
[kid(x) ^ sandwich(y) ^ orange(z) ^ (eat(x,y) ^ eat(x,z))]
b) bb dxdydz
dxdydz [kid(x) ^ sandwich(y) ^ orange(z) ^ (eat(x,y) ^ eat(x,z))]
logical
entailment
implement the meaning of the connectives or and and in our formal language. What does this job in formal
implement
meaning
of in
theevery
connectives
and and
in which
our
formal
language.
does
this
in formal
sentence
is true
circumstance
in
sentence
(b)aWhat
is true
semantics
isthe
the(a)
truth
functions,
which
takesortruth
values
as input,
and
yields
truth
value
as job
output.
We
semantics
the truth functions,
takes
truth
asor
input,
and yields
value
as output.
We
may
rely onisrst-order
predicate which
logic and
assign
tovalues
and and
the truth
tables aoftruth
logical
conjunction
and
does
not hold:
(a)and
can
betrue
true
with
(b)
being
false
maythe
relyconverse
on respectively,
rst-order
predicate
assign
to and
and
the
truth
tables
logical
conjunction
and
disjunction,
with thelogic
former
being
when
allor
the
conjuncts
are of
true
and the
latter being
disjunction,
with
the former
being
true
all the conjuncts
are rst
true part
and of
thetheir
latter
being
true
when at respectively,
least one of the
disjuncts
is true.
Now
we when
can compare
(5) to (6). The
logical
logical entailment
true when
at least
one is
ofat
the
disjuncts
Now ywe
can
(6). The
rst
their
logical
form
is identical.
There
least
a kid x,isa true.
sandwich
and
ancompare
orange z.(5)
Thetosecond
part
ofpart
(5b)of
says
that
the
form
is identical.
There isand
at least
a kid x, a sandwich
andsays
an orange
Thealso
second
partone
of (5b)
says that
the
kid
eats
both the orange
the sandwich,
whereasy(6b)
that hez.can
eat just
of them.
Thus,
eats
both
orange
andfrom
the sandwich,
whereas
(6b) says
thatand
he
can
also
of them.
Thus,
eating
athe
sandwich
eating
an
orange
or(5b)
and
akid
straight
intuition
coming
the
comparison
between
(6b)
is eat
thatjust
theone
former
is more
a straight than
intuition
coming
from
the comparison
(5b)
and (6b) in
is which
that the
more
restrictive
the latter.
More
precisely,
(5b) is true between
in a subset
of situations
(6b)former
is true,is
hence
T(5b) is true
T eating
Tboth a sandwich
T the latter.
restrictive
than
MoreRoughly
precisely,
in aissubset
of situations
in which (6b)
is true,
hence
the
former is
logically
stronger.
speaking,
if there
a boy
and an
orange,
it
the former
is logically
stronger.
Roughly
speaking,
ifthem.
there This
is aTboy
eating
both a sandwich
and
an orange,
it
plainly
follows
that
there
is
also
a
boy
eating
one
of
meaning
relation
that
holds
between
(6a)
F
F
T
plainly
follows
that
there
is
also
a
boy
eating
one ofanother
them. This
meaningBrelation
thatofholds
between- i.e.
(6a)
and (5a) is called entailment. A proposition A entails
proposition
if the truth
A implicates
TbetweenF(5a)
and (5a)entails
is called
entailment.
AMost
proposition
B if
the(6a)
truth
of Aany
implicates
- i.e.
T - the
FA entails
ensures,
truth of B.
importantly,
thisanother
relationproposition
and
- like
other types
ensures,
- the truth
of B. sentences
Most importantly,
this relation
between
(5a)in
and
(6a) -they
like are
anyevaluated.
other types
of
formalentails
relationships
among
- holds regardless
of
the context
which
F
F
F
F- holds regardless of the context in which they are evaluated.
of formal
relationships
among between
sentences(5a)
Namely,
the
entailment relation
and (6a) is due to the (logical) meaning of the connectives or
Namely,
entailment
(5a)inand
(6a)this
is due
to the (logical)
meaning
of the connectives or
and
and, the
rather
than on relation
the kindbetween
of scenario
which
sentence
is uttered
and veried.
Panizza, 2011
formal semantics
syntactic analysis of the sentence
set of conversion rules that translate the linguistic input into a symbolic
structure (Logical Form)
where the formal relations (e.g. dependency, scope relations, argument
structure, agreement etc.) between the expressions are made explicit
semantic interpretation process

takes Logical Form as input
computes the truth conditions of the proposition by composing the elements
contained in this structure (individuals, variables, predicates etc.)
following a finite set of conversion rules (e.g. truth functions)
compositional fashion: the meanings of sentences are derived from the meanings
of their composite parts
formal language
(e.g. our toy
language)
vocabulary
basic expressions
syntax
set of rules of conversion
formal language
(e.g. our toy
language)
Panizza, 2011
vocabulary
basic expressions
syntax
set of rules of conversion
Montagues fragment of English
Montagues
contention
problems
although natural languages

contain vagueness and
idiosyncrasies
discrepancy between
natural languages and
formal ones
no fundamental theoretical difference

between the syntax-semantics
relation in English and a formal
language like first-order predicate
logic
modern semantics: formal (artificial)
languages are model of the grammar of
natural languages
realized in cognitive systems that are distinct from directly

observable human linguistic behavior they help to explain
additional ingredients: more powerful logic (modal logic), economy principles,
parsing strategies, pragmatic rules, interfaces at various levels
the question is:
Panizza, 2011
is formal semantic a good description of how language

comprehension works, as invented by crazy linguists?
or is it a psychologically plausible model in cognitive science?
better ask Marr!
3 levels of computation
information processing systems must be understood at 3 levels of analysis

(one level is not enough!)
1) COMPUTATIONAL LEVEL
what the system does and why
2) ALGORITHMIC LEVEL
how it performs its computation
3) IMPLEMENTATION LEVEL
how it is realized in the brain

(what neural structures and activities)
Panizza, 2011
formal semantics provides a mapping between syntactic structures and conceptual

representations, by composing the lexical meaning of single words into propositions in
a recursive fashion
formal model: simplification, coded into a symbolic format, of a real word context or situation
abstract, symbolic and grammar-driven representations
logical operators (,,,), variables (x, y, z), set-theoretic operations (,,,)
predicates (boy(x)), higher-order logic systems ( , )

back in the 80s
modules:
Panizza, 2011
modularity provided a strong

computational and implementational
framework for linguistic modularity
informationally encapsulated
one module did not access the
representations of another module
implemented in different brain areas

Wernickes, Brocas, other cortical areas
phonology
processing:
morpho-syntax
semantics
pragmatics
strictly serial
the output of one module was passed as input to the next module
no jumps, no feedback, no parallel processing (modules had to wait!)
in the 2000s
Panizza, 2011
crisis of modularity in cognitive science
no clear localisation of linguistic functions

where is semantics?
multimodal
areas
rise of distributed networks
732 M. Garagnani and F. Pulvermuller
demise of strict seriality
B
PM L
parallel and highly interactive systems
x = 70 to 20
4.7
M1 L
A1
PF L
AB
z=0
PB
y=0
3.1
x = 70 to 20
Brocas and Wernickes areas are directly linked with sensorial (auditory), associative (prefrontal) and motor (dorsal 10.1
PFi
and ventral pre-motor cortex and primary motor cortex) through
adjacent and long-distance cortical connections
PM i
M1i
similar networks found in monkeys (who does not speak!)
AT
TO
V1
z=0
Garagnani & Plvermuller,y(2016)

=0
conceptual grounding
General lexicosemantic
activation of motor areas in comprehension tasks

kick
lick
activates motor cortex of leg/tongue
face/mouth word
(lick)
arm/hand word
(pick)
leg/foot word
(kick)
3.1
Panizza, 2011
Rauschecker
in the 2000s
mirror neurons
brocas homologue in monkeys

(evolution of language areas)
Processing pathways in language evolution
support for embodied

view
ofstimuli,
language
kinds
of sequential
even beyond the auditory domain.
in the processing and categorization of self-produced sounds will,

therefore, have to be tested by other means (Remedios et al.,
2009).
The involvement of the dorsal auditory pathway, including
premotor and inferior parietal regions, in the encoding and representation of temporally extended sounds (or sound sequences)
became especially evident, when imagery of musical melodies
was investigated (Leaver et al., 2009). During the learning of
such sequences, the basal ganglia were actively engaged, whereas
after these sequences became highly familiar, the same sequences
activated more and more prefrontal areas. It appears, therefore,
that the basal ganglia are responsible for the concatenation of
sequential auditory information or formation of chunks, which
represent information about conditional probabilities for one
sound being followed by another. Once the chunks have been
formed, they are once again stored in prefrontal regions. A similar chunking process occurs with cued sequences of learned finger
movements (Koechlin and Jubault, 2006). This process involves
prefrontal cortex near Brocas area and has, therefore, been
compared with models of language (Hagoort, 2005), redefining
Brocas area in terms of chunking (unification) of semantic,
syntactic, and phonological information.
Thus, the role of the dorsal stream can be conceptualized into
one of sensorimotor integration and control and applies to all
Specifically for speech, the dorsal processing stream plays a role

in speech production as well as categorization of phonemes during on-line processing of speech (Rauschecker and Scott, 2009;
Rauschecker, 2011; Figure 1). The former role conforms to the
classical idea of an efference copy or feed-forward model and
allows for fast and efficient on-line control of speech production.
By contrast, the latter function can be formalized as an inverse
model during real-time speech processing, creating the affordances of the speech signal in a Gibsonian sense (Gibson, 1966;
Rauschecker, 2005). Both functions require a (direct or indirect)
connection between sensory and motor cortical structures of the
brain, whereby subcortical structures (e.g., the basal ganglia) provide an additional link setting up transitional probabilities during
associative learning of sound sequences.
Comparing human and monkey brain connectivity along
the dorsal stream, there may be quantitative differences in the
strengths of these connections, but there does not seem to be a
difference in principle (Frey et al., 2008). Similarly, in the ventral stream, the fine-grain organization of cortical areas and the
fine-tuning of its neuronal elements may be richer in humans
than in monkeys, providing humans with a perceptual network
for the detection of more subtle differences in the acoustic signal.
The decisive distinction between humans and monkeys may,
embodied language
Panizza, 2011
not only meaning is grounded in motor system..
linguistic meaning is processed by the brain in the very same motor representations
involved in perceiving, controlling and executing motor programs
new cognitive primitive mechanism:
can symbolic
NO!
representation perceptual symbols
still exist?
internal simulation
construct novel conceptual representations

those that we already experienced
the abstract symbol view of meaning is one of the

most remarkable misunderstanding in the history of
science (Edelman, 1992, Nobel price for.. Chemistry)
Neither semantics nor grammar is symbolic, in the sense of
theory of formal systems, which consists of rules for
manipulating disembodied meaningless symbols (Gallese &
Lakoff, 2005, mirror neurons expert and former formal linguist)
however..
Panizza, 2011
sparse evidence in humans
crisis of mirror neurons
damage to motor system does not lead

to comprehension deficits
unlike other double dissociations
ideational
apraxia
motor
apraxia
recognise know how to use but do
not manage to perform the right
movements
recognize but dont

know how to use
ideational
apraxia
visual agnosia
dont recognize but know
how to use
recognize but dont

know how to use
aphasia
recognize and use but dont know how to say
anyways..
maybe
Panizza, 2011
perceptual symbols
provide a link
with grammar (syntax)
grounded representations
simulate symbolic/formal
behaviour
abstract power
remember the dutch trains! (Hagoort et al., 2004)

no difference between word knowledge and
word meaning (or lexical/conceptual semantics
and formal/compositional semantics)
= brain networks
= processing timing
the dutch trains are sour = white
Let us go back to the afrmative sentences in (5a) and (6a) to test whether the algorithms
Formal
Neurosemantics
Panizza, 2011
and (8) work for these cases. Sentence (5a) is usually uttered in the circumstances in whi
speaker does not know for certain whether there is a kid eating a sandwich or an orange. Eith
if it is being eaten by the kid, describes the meaning of this sentence. It seems, thus, that the
productive system based on perceptual simulation
Barsalous
perceptual symbols:
rithm in (7) would work perfectly for (5a). In both cases the actual scenario would mat
simulated frame, which alternates the two sub-events where a boy is eating either a sandwich
orange. As for the conjunctive proposition in (6a), here the speaker knows that a boy is eating b
1. construct an internal representation of the proposition via perceptual simulation
orange and a sandwich. Therefore this is the only situation satisfying the meaning of the conju
The 2a.
algorithm
we
sketched
out
in (8) would
work forthe
this
case,under
as conjoining
the two kinds o
simulate
two
separate
sub-events
alternating
entity
disjunction
OR ():
being
eaten
by some
would
exactly match the scenario described by (6a). We have demons
while
holding
theboy
frame
constant
so far, that algorithms such those in (7) and (8) may account for the meaning of logical fu
ANDgrounding
(): 2b. simulate
one frame
containing
both
on simple
operations
applied
to events
perceptual symbols such as alternating or conjo
sub-events.
positive
The
next whether
step of this
argument
is to matches/mismatches
show that the algorithms
in (7) and
(8) run into s
vs.
3. verify
the actual
scenario
the simulated
frame
negation
problems in cases where connectives such as and and or interact with other logical functors, s
negation. Let us consider sentences (9a) and (10a), which are identical to (5a) and (6a) wi
exception of the presence of a negative quantier (no, which means there is no individual) ap
sentence in which disjunction is embedded under negation
to boys.
(reverts the truth condition)
(9)
a No boys are eating a sandwich or an orange.

b :dxdydz [kid(x) ^ sandwich(y) ^ orange(x) ^ (eat (x,y) n eat (x,z))]
Barsalous algorithm predicts that a situation with boys eating both sandwich and
(10) a No boys are eating a sandwich and an orange.
oranges would be true!
b :dxdydz [kid(x) ^ sandwich(y) ^ orange(z) ^ (eat (x,y) ^ eat (x,z))]
Sentence (9a), under its more natural interpretation2, means that there is no boy eating a san
if it is being eaten by the kid, describes the meaning of this sentence. It seems, thus, that t
rithm Neurosemantics
in (7) would work perfectly for (5a). In both cases the actual scenario would ma
Formal
Panizza,
simulated frame, which alternates the two sub-events where
a boy2011
is eating either a sandwi
orange. As for the conjunctive proposition in (6a), here the speaker knows that a boy is eating
orange and a sandwich. Therefore this is the only situation satisfying the meaning of the conj
First-order
logic
eating a sandwich
eating an orange
or
not or
The algorithm we sketched out in (8) would work for this case, as conjoining the two kinds
being eaten by some boy would exactly match the scenario described by (6a). We have demon
T
T
T
F
so far, that algorithms such those in (7) and (8) may account for the meaning of logical f
F
T symbols such asT alternating
F
grounding on simple operations
applied to perceptual
or con
sub-events.
T
F
T
F
The next step of this argument is to show that the algorithms in (7) and (8) run into
F
problems in cases where connectives
such as and andFor interact with Fother logical
functors,
T
negation. Let us consider sentences (9a) and (10a), which are identical to (5a) and (6a) w
exception of the presence of a negative quantier (no, which means there is no individual) a
sentence in which disjunction is embedded under negation
to boys.
(9)

First-order logic predicts the right meaning: no boys are eating any of the two kinds of food
:dxdydzalgorithm:
[kid(x) ^ sandwich(y)
orange(z) logic
^ (eattruth
(x,y)function
^ eat (x,z))]
easy fix forbBarsalous
make OR like^First-order
2
OR (): Sentence
, means
that
there is no
boy eating a sa
(9a),separate
under itssub-events
more natural
interpretation
simulate two
alternating
the entity
under
disjunction
while
andholding
there isthe
no frame
boy eating
an orange
either. The
lastsub-event
part of formula
(9b),both
once applied to the
constant
and consider
a third
including
existential
(:dx),
accounts
for such meaning. That is, the disjunction under neg
entities.predication
Alternate the
three
sub-events
true just in case both disjuncts are false.
exception of the presence of a negative quantier (no, which means there is no individual) ap
to boys.
(9)
Panizza, 2011

sentence in which conjunction is embedded under negation


Sentence (9a), under its more natural interpretation2, means that there is no boy eating a san
and there is no boy eating an orange either. The last part of formula (9b), once applied to the n
existential predication (:dx), accounts for such meaning. That is, the disjunction under nega
First-order
logic
eating
a sandwich
and not and
true just in case
both disjuncts
are false. eating an orange
If we apply the Barsalous algorithm in (7) to interpret the disjunction in (9a) we will obtain the
T species that there
T is a Fmismatch betwe
T
meaning. Recall that, according
to Barsalou, negation
simulated and the perceived
situation. Thus, we rst
(9a)
F
F
T apply (7) to sentence
T and we get two
native sub-events in which some boy is eating a sandwich (the rst sub-event) and some boy is ea
F scenario
orange (the second sub-event).
Then we are to check
T
F whether the actual
T displays a di
situation. If no boy is eating sandwiches or oranges the meaning of (9a) turns out to be true and (7)
F
F
T both an oran
up working ne. But what if we are attending to a situation where someFboy is eating
a sandwich? Negation applied to the result of (7) would be satised in this case, as the actual sc
does not mach either any one of the alternative disjunctive frames, where some boy is eatin
Barsalous
one kind of food.
this is notone
what
(9a) containing
means, in that
such
a situation is clearly ruled o
ANDHowever,
(): 2b. simulate
frame
both
events
algorithm Barsalous algorithm for disjunctive meaning fails thereby to predict the right meaning of o
embedded under negation. The reason underlying its failure is that (7) implements just the mea
the exclusive interpretation of or. When disjunction is embedded under negation, people tend to
both algorithms
leave open the possibility that there is a boy only eating a sandwich or an orange
or an inclusive interpretation (cf. Chierchia, 2004). The author nevertheless admitted that his pr
was aimed
to offer
a viable
implement
one ofelse?
the possible meanings of or and negation. Fo
so far so good
but what
if there
is away
boyto
eating
something

Panizza, 2011
D. Panizza / Journal of Neurolinguistics xxx (2010) 129
a No boys are eating a sandwich and an orange.

ntence (9a), under its more natural interpretation2, means that there is no boy eating a sandwich
here is no boy eating an orange either. The last part of formula (9b), once applied to the negated
ential predication (:dx), accounts for such meaning. That is, the disjunction under negation is
ust in case both disjuncts are false.
we apply the Barsalous algorithm in (7) to interpret the disjunction in (9a) we will obtain the wrong
according
to Barsalous
algorithm,
ning. Recall that,
according
to Barsalou,
negation species that there is a mismatch between the
ated and theanything
perceived
situation.
Thus,awe
apply (7) to sentence (9a) and we get two alterthat
mismatches
boyrst
eating
e sub-eventsboth
in which
some boy
eatingisa TRUE!
sandwich (the rst sub-event) and some boy is eating an
sandwich
andis
orange
ge (the second sub-event). Then we are to check whether the actual scenario displays a different
tion. If no boy is eating sandwiches or oranges the meaning of (9a) turns out to be true and (7) winds
orking ne. Butitwhat
if we
are attending
to a situation
where some boy is eating both an orange and
would
include
the boy eating
an
dwich? Negation
applied
to the result
(7) would be satised in this case, as the actual scenario
orange,
a sandwich
and of
a banana.
not mach either
any one of the alternative disjunctive frames, where some boy is eating only
WRONG!
ind of food. However, this
notto
what
(9a) means, in that such a situation is clearly ruled out.
nois
way
fix this.
arsalous algorithm for disjunctive meaning fails thereby to predict the right meaning of or when
dded under negation. The reason underlying its failure is that (7) implements just the meaning of
xclusive interpretation of or. When disjunction is embedded under negation, people tend to assign
inclusive interpretation (cf. Chierchia, 2004). The author nevertheless admitted that his proposal
negation
rules
out any situation
imed tological
offer a viable
way
to implement
one of the possible meanings of or and negation. For, both
in which a boy is eating both a sandwich

and an orange (regardless whether he is
other
stuff)
rn (1989),eating
Levinson
(2000)
and Chierchia (2004) maintain that the exclusive interpretation of or is due to a pragmatic
nce (Scalar Implicature) that is generally not computed (or suspended) under negation and conditionals, as empirically
Panizza, 2011
formal semantics provides a mapping between syntactic structures and conceptual

representations, by composing the lexical meaning of single words into propositions in
a recursive fashion
it accounts for logicality of language (and there is much more)
formal model: simplification, coded into a symbolic format, of a real word context or situation
abstract, symbolic and grammar-driven representations
logical operators (,,,), variables (x, y, z), set-theoretic operations (,,,)
predicates (boy(x)), higher-order logic systems ( , )

can we find neuroscientific evidence?
Panizza, 2011
Pylkkanen, Olivieri and Smart (2009)

Hagoort et al. study does not distinguish between lexical and compositional semantics
both conceptual/lexical semantics violations
the dutch trains are sour = white
they are both composable

thats why elicit the same neuropsyc. eects
operation by which a verb predicate receives a reversative meaning
verbal un-prefixation
unbutton a shirt =
semantically constrained
undo the result of buttoning a shirt
requires an accomplishment verb

event structure involving a change of state
the wine was being uncorked

the thirst was being unchilled
the thirst was being uncorked
semantic/compositional violation: unchilling is not impossible to

conceive but semantically banned from our grammar
world knowledge violation
Panizza, 2011
SEMANTICS VS. WORLD KNOWLEDGE IN PFC
both violation generate an

effect at LIPC
only semantic/compositional
violations elicit an effect at the
Ventromedial Prefrontal
Cortex (VmPFC)
earlier than the effect at LIPC
11
Panizza, 2011
compositional operations involve distinct brain networks

with respect to conceptual violations
compositional operations may precede integration of meaning and knowledge
new locus for semantic composition: VmPFC
also higher activation for increased compositional complexity

(coercion: begin the book, Pylkkanen and McElree, 2007))
time
that participants
have
available
for comprehending
com
r
view
the
picture
can
they
start
the
verification
processing even after athe
certai
ue nor false. Only when
Formal
Neurosemantics
lated d
th
This
allowed
us to manipulate
amount
of in both
Panizza, 2011
the the
comprehender
hey start the verification
participants
available
for
comprehending
lated
the depicted stateinofthe
affa
nipulate
the have
amount
of
Ldtke et al. (2008)
1. Sample Materialsin the unprimed conditions.
able for Table
comprehending
how do speakers process negation?
in front of the tower there is no ghost
Sentence
Picture
In front of the tower there is a ghost.

Sample
Materials
process negation and
compare negated content to
compositional content
real scenario
Picture
Picture
Relation
f the tower there is a ghost.
st.
simulate the negated content
Mentio
Relation
Simulations
Mentioned
In front of the tower

there is a ghost.
Mentioned
Not
m
True
In front of the tower there is no ghost.
Not m
f the tower there is a ghost.
st.
Rel
Not mentioned
Not mentioned
Fals
Panizza, 2011
Ldtke et al. (2008)
how do speakers process negation?
in front of the tower there is no ghost
immediate integration model

delay

real scenario
no delay
two stages model

real scenario
sentence processing (see above). In principle, these

contrast to the ERP studies reported above, verification
priming effects should be observed at both delay conis temporally distinct from sentence comprehension in
ditions because (a) mentioning should facilitate picture
the present paradigm. Participants first read the senprocessing even afterPanizza,
a certain 2011
delay, and (b) because
tence that in itself is neither true nor false. Only when
the comprehender in both delay conditions has simuthey later view the picture can they start the verification
lated the depictedverification
state of affairstask
in the primed but not
process. This allowed us to manipulate the amount
of sentence-picture
task:
Ldtke
et al.
(2008) have available for comprehending
in the unprimed conditions. However, priming effects
time that
participants
2 delays:
exp. design
300 ms vs. 1500 ms
Table 1. Sample Materials

Sentence
Picture
Relation
Simulations
Condition
Priming
Mentioned
True affirmative
Primed
Not mentioned
False affirmative
Not primed
Not mentioned
True negative
Not primed
Mentioned
False negative
Primed
predictions:
priming/mentioning
(True a. & False neg.)
mismatch
(False)
facilitates
Lu
dtke et al.
surprisal eect if
negation is processed
1357
Panizza, 2011
Ldtke et al. (2008)
results
Vor dem Turm ist ein/kein Geist

in front of the tower there is a/no ghost
frontal negativity
at 300ms
on the noun after
negation vs.
affirmative
early linguistic
processing
of negation
Figure 1. Grand mean averages related to ein/kein (a/no) and the following subject noun onset for selected electrode leads.
marked.
short delay
(300 ms)
Ldtke et al. (2008)
results
Panizza, 2011
N400 for not primed/mentioned

conditions (False pos., True neg.)
Lu
dtke et a
truth/negation interaction
long delay (1500 ms)
N400 for not primed/mentioned

conditions (False pos., True neg.)
truth/negation
interaction
1363
main effect of negation
main effect of truth
Lu
dtke et al.
at long delays the negation was integrated

in the interpretation of the sentence
Panizza, 2011
Ldtke et al. (2008)
frontal negativity
at 300ms
on the noun after
negation vs.
affirmative
support for two-stages model

truth/negation interaction
in all conditions
delay
2
main effect of negation
main effect of truth
only with long delay

real scenario
Panizza, 2011
the puzzle from NPIs
Panizza & Romoli (2013)
what is the meaning of any?
A:
do you have potatoes?
B:
no, I dont [have any/a potato]
A:
not even small/old/sweet ones?
B:
no, I dont have any potato/*a potato
any = a
emphatic use of any

it excludes any more
specific kind of potato
another puzzle
from NPIs:
their distribution
neg
non-neg
Panizza, 2011
NPI licensing: NPIs like negative contexts...

...but also non-negative ones.
a. No man has ever set foot on the moon.

b. I doubt that a man has ever set foot on the moon.
c. If a man has ever set foot on the moon, he would be a fan of the Police.
d. Every man who has ever set foot on the moon is famous.
e. Before a man had ever set foot on the moon,
we thought it was an impossible venture.
f. The moon is the farthest place on which an astronaut has ever walked.
g. Have you ever set foot on the moon?
what do these contexts have in common?

they license entailment from sets to proper subsets,
they are DOWNWARD ENTAILING
(Ladusaws generalization)
Panizza, 2011
DOWNWARD ENTAILING
No man has ever set foot on the moon.
No man taller than 190 cm has ever set foot on the moon.
Every man who has ever set foot on the moon is famous
Every man taller than 190 cm who has ever set foot on the moon is famous
man-taller-than-190cm
men
UPWARD ENTAILING
A man taller than 190 cm has ever set foot on the moon.
A man has ever set foot on the moon.
they license entailment from set to proper subsets,

Panizza, 2011
they license entailment from set to proper subsets,

linguistic question: why do NPIs behave this way?
accidental property shared by every language in the world
or..
it is the consequence of something deeper
NPIs distribution is determined by their meaning
(Kadmon & Landman, 1993; Krifka, 1995; Lahiri, 1998;
Chierchia, 2004, 2006, 2013)
we need a way to implement the semantics of NPIs in a formal system
Panizza, 2011
focus sensitive operators: only, even, contrastive focus
activate Domain alternatives
speaker A: did any of your friends vote for Berlusconi?

speaker B: only John and Mary did.
context provides the Domain: friends = {John, Mary, Anna, Paul}
only activates alternatives
conventional implicature: they must be entailed by the assertion, otherwise FALSE
J &M & A & P voted for B
J & A & M voted for B
J & P & A voted for B
M & A & P voted for B
J & A voted for B
J & P voted for B
M & P voted for B
M & A voted for B
J voted for B
M voted for B
P voted for B
A voted for B
ever
Panizza, 2011
existential quantifier over interval of times

activates domain alternatives
speaker A: have you voted for Berlusconi last elections?

speaker B: a) No I havent voted for Berlusconi.
b) No I havent ever voted for Berlusconi.
Panizza, 2011

1996
2001
2006
2008
2013 (last elections)
Panizza, 2011
ever

considers a contrast set (like focus elements)

1996
2001
2006
2008
Panizza, 2011
ever

activates sub-domain alternatives
OK!

1996
2001
2006
2008
downward
entailment
Panizza, 2011
ever


speaker B: a) Yes I have voted for Berlusconi.
b) Yes I have ever voted for Berlusconi.
1996
2001
2006
2008
Panizza, 2011
ever


1996
2001
2006
2008
Panizza, 2011
ever


1996
2001
2006
2008
upward
entailment
Panizza, 2011
ever
there is a t in D in which I voted for Berlusconi and for every sub-domain

alternative there is not a t in which I voted for Berlusconi
ANALYTICAL CONTRADICTION
1996
2001
2006
2008
upward
entailment
goals
language-related
mechanisms
ERPs
conflicts
Panizza, 2011
cognitive processes
morpho-syntactic conflicts
Kaan and Swab
(2003)
# the child throw the toy
(LAN) / P600
the children throw the toy
conceptual/lexical conflicts
Kutas and Hylliard
(1980)
she spread the warm bread with butter

she spread the warm bread with socks
N400
Panizza, 2011
NPI: ever, any, jemals, mai

Saddy et al.
(2004)
*a man with a beard was ever happy

no man with a beard was ever happy
violation of a
linguistic rule
P600
unexpected word,
lexical conflict
last resort,
unconvincing
why?
N400
N400: (Saddy, Drenhaus and Frisch, 2004; Pablos,

Shirley, Erdocia, Laka, Williams and Saddy, 2011)
N400 + P600 (Drenhaus, Baszczak and Schtte, 2007;
Steinhauer, Drury, Portner, Walenski and Ullman, 2010)
task-related effect
(conflict monitoring)
semantic/logical
violation
Panizza, 2011
post-verbal position (after the verb or the auxiliary)

mai is ever
= meaning
= distribution
a. Un uomo non ha mai camminato sulla Luna.

A man has not ever walked on the Moon.
b. *Un uomo ha mai camminato sulla Luna.
*A man has ever walked on the Moon.
Ques%ons
Hai mai camminato sulla luna?
Have you ever walked on the moon?
pre-verbal position (before the verb or the auxiliary)

mai means never (ever + NEG)
c. Un uomo mai ha camminato sulla Luna.
A man has never walked on the moon.
Panizza, 2011
pre-verbal position (before the verb or the auxiliary)

mai means never (ever + NEG)
but only in non-DE contexts!
it means ever otherwise.
d. Se io mai dovessi andare sulla Luna mi porterei la crema solare.
If I was ever to go on the Moon I would bring the sun protection with me.
e. Ogni uomo che mai sia stato sulla Luna diventato famoso.
Every man who has ever been on the Moon became famous.
research question
how does the parser know when mai is to be
interpreted as (n)ever?
on which kind of information does it rely?
when is this info accessed?
Panizza, 2011
design
postverbal
control
a. Il mister credeva che i ragazzi non avrebbero mai giocato sotto la pioggia.
'The coach believed that the boys would not ever play under the rain.'
NPI
violation
b. *Il mister credeva che i ragazzi avrebbero mai giocato sotto la pioggia.
'The coach believed that the boys would ever play under the rain.'
preverbal
preverbal
mai
c. Il mister credeva che i ragazzi mai avrebbero giocato sotto la pioggia.
preverbal
violation
d. *Il mister credeva che i ragazzi non mai avrebbero giocato sotto la pioggia.
'The coach believed that the boys would never play under the rain.'
'The coach believed that the boys would not never play under the rain.'
Panizza, 2011
design
preverbal
mai

perfect system
(predicting monotonicity)
imperfect system
the parser knows whether mai is

in a DE environment before
interpreting it (from semantic or
syntactic cues)
the parser does not immediately

access the information about the
monotonicity of the context
performs the right interpretation

(negative if not-DE; positive if DE)
performs the default interpretation

of mai (NPI) and adds a negation if
the interpretation yields an error
(non-DE context)
Panizza, 2011
predictions
perfect system
(predicting monotonicity)
NPI
- control
violation
cost of dealing with semantic
clash, fixing the structure &
meaning
NPI
- control
violation
clash, fixing the structure &
meaning
different
processes
preverbal mai
control
no clash, no fixing, adding a

negation
imperfect system
(repair strategy)
preverbal mai
control
same
processes
clash, adding a negation
Panizza, 2011
(explorative)
preverbal
violation
fixing negation easily
the parser fixes the negation, which

is indeed present in the sentence,
and solves the problem before
interpreting mai.
NPI violation
the parser interprets mai in the

wrong context as if it is a NPI
violation.
Panizza, 2011
frontal electrode
200
200
400
results
600
800
postV gram
preV gram
postV ungram
preV ungram
Fz
Pz
FP600
4
N400
200
400
600
800
1000
1200
200
control
NPI
violation
Cz
'The
coach believed that the boys would not ever play under the rain.'
200
200
preverbal
violation

preverbal
mai
400
600
800
Panizza, 2011
200
200

400
posterior electrode
results
600
800
1000
1200
200
postV gram
preV gram
postV ungram
preV ungram
200
400
600
800
postV gram
preV gram
postV ungram
preV ungram
Pz
Pz
N400
0
200
400
600
800
1000
1200
200
P600
control
NPI
violation
200
200
preverbal
mai
preverbal
violation
400
600
800
Panizza, 2011
200
200
central electrode

400
600
800
1000
1200
200
results
200
400
800
postV gram
preV gram
postV ungram
preV ungram
FP600
Cz
600
Pz
N400
200
400
600
800
postV gram
preV gram
postV ungram
preV ungram
1000
1200
200
P600
Pz
2
200
200
preverbal
violation
N400 + FP600 + P600
preverbal mai
larger N400 + FP600
NPI
violation
400
600
800
Panizza, 2011
discussion
grammatical control
Panizza, 2011
discussion
preverbal mai
bigger N400 + FP600

Panizza, 2011
discussion
NPI
violation
N400 + FP600 + P600
Panizza, 2011
discussion
preverbal
violation
N400 + FP600 + P600
Panizza, 2011
preverbal mai
bigger N400 + FP600
>
NPI
violation
preverbal
violation
N400 + FP600 + P600
why
an N400
for NPI
violations?
unexpected word,
lexical conflict
task-related effect
wrong predictions!
Panizza, 2011
preverbal mai
bigger N400 + FP600

NPI
violation
preverbal
violation
N400 + FP600 + P600
unexpected word,
lexical conflict
why?
wrong predictions!
task-related effect
semantic/logical
violation
provides a good
account
preverbal mai
Panizza, 2011
bigger N400 + FP600

adding the negation = repair strategy due to imperfection of the system

cf. Reinhart, 2006
implications for ERP psych
morpho-syntactic conflicts
(LAN) / P600
semantic/logical conflicts
N400 / (FP600)
conceptual/lexical conflicts
N400

Part Six: Formal Neurosemantics: Neurolinguistics

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Part Six: Formal Neurosemantics: Neurolinguistics

Uploaded by

Copyright:

Available Formats

NEUROLINGUISTICS

more complex structures

agents of communication share

extra-linguistic concepts about the world

is something missing in this picture?

more complex structures

extra-linguistic concepts about the world

more complex structures

extra-linguistic concepts about the world

To know the meaning of a sentence is to know its truth conditions

the snow is white is true if and only if the snow is white

formal theories of meaning

formal theories of meaning

language presents regularities

implicit knowledge that speakers

there is some kid who is eating a sandwich

uated; b) sets of situations where sentence (10a) is true, after negation

(4) dxdy [kid(x) ^ sandwich(y) ^ eat (x,y)]

a Some boy is eating a sandwich or an orange.

a Some boy is eating a sandwich and an orange.

Some boy is eating a sandwich or an orange

(5) a Some boy is eating a sandwich or an orange.

Some boy is eating a sandwich or an orange

(5) a Some boy is eating a sandwich or an orange.

semantic interpretation process

Montagues fragment of English

although natural languages

no fundamental theoretical difference

realized in cognitive systems that are distinct from directly

is formal semantic a good description of how language

better ask Marr!

information processing systems must be understood at 3 levels of analysis

what the system does and why

how it performs its computation

how it is realized in the brain

formal semantics provides a mapping between syntactic structures and conceptual

how it performs its computation

how it is realized in the brain

modularity provided a strong

implemented in different brain areas

crisis of modularity in cognitive science

no clear localisation of linguistic functions

demise of strict seriality

parallel and highly interactive systems

similar networks found in monkeys (who does not speak!)

Garagnani & Plvermuller,y(2016)

activation of motor areas in comprehension tasks

activates motor cortex of leg/tongue

brocas homologue in monkeys

Processing pathways in language evolution

support for embodied

in the processing and categorization of self-produced sounds will,

Specifically for speech, the dorsal processing stream plays a role

not only meaning is grounded in motor system..

construct novel conceptual representations

the abstract symbol view of meaning is one of the

sparse evidence in humans

crisis of mirror neurons

damage to motor system does not lead

recognize but dont

recognize but dont

remember the dutch trains! (Hagoort et al., 2004)