International Symposium on Performance Science © Richard Parncutt 2013

ISBN tbc All rights reserved

Piano touch, timbre, ecological psychology,

and cross-modal interference

Richard Parncutt
Centre for Systematic Musicology, University of Graz, Austria

The piano has a wide timbral range, and performance quality is often
judged in timbral terms. Yet despite decades of research, there are still
fundamental disagreements about the nature and origin of piano touch.
Scientists (acousticians) maintain that the timbre of a single tone cannot
be varied independently of its loudness. Performers, humanities scholars
and concert audiences take the opposite for granted: timbre and loudness
can be independently varied by gestural means. Both sides are right, but
their implicit definitions of timbre differ, and both fail to clearly
distinguish between physical measures and descriptions of subjective
experience. Scientists assume that timbre depends only on physical
sound parameters; but experiential parameters generally depend on
concurrent input from other senses, the listener’s relevant knowledge
and expectations, and immediately preceding and following events. The
paradox of timbre disappears if we accept, based on empirical evidence,
that timbre generally depends on input from more than one sensory
modality (weak synesthesia). Embodied corporality and conceptual
metaphors are the norm - not the exception. Gestural and ecological
approaches to timbre perception pose existential challenges to
disembodied cognitive orientations.

Keywords: timbre; piano; touch; ecology; synesthesia

The piano has a remarkable timbral range. Piano timbre depends strongly on
pitch (higher is brighter) and loudness (louder is brighter) in ways that are
unique to the instrument. Piano timbre is also affected by complex physical
interactions (among strings, soundboard, internal resonances) and
perceptual interactions (among sensations and emotions), which in turn
depend on timing, dynamics and pedaling.
The way a pianist strikes a key (“touch”) seems to influence timbre, even
when loudness (key velocity) is constant. In fact, the hammer hits the string
in free flight, so any such effect must be tiny; empirical studies suggest that if
such an effect exists, it is inaudible. We can sometimes hear fingertips hitting
002 WWW.PERFORMANCESCIENCE.ORG

keys (“touch precursor” or “early noise” in “staccato touch”; Goebl et al.,

2004) but that is a small and probably negligible aspect of “touch”.

WEAK SYNESTHESIA

To account for the richness of piano timbre and demystify piano touch, we
need an ecological, multimodal concept that acknowledges the role of vision,
proprioception, the somatic sense, and gesture. Weak synesthesia or cross-
modal interference occurs when perceptual input in one modality (seeing,
hearing tasting and so on) influences perceptual judgments in another. Weak
synesthesia can occur in all perceptual modalities (Martino & Marks, 2001)
and is probably innate (Walker et al., 2010).
Although we physically pick up information via different sensory
modalities (hearing, vision, touch), and within each modality there are
separable sensations (e.g., pitch, timbre), we cannot completely separate
modalities or sensations. The reason is ecological and evolutionary: sounds
are only interesting (and consciously perceived) if they carry information
about environmental interaction that could affect survival or reproduction.
We tend to perceive environmental objects holistically, focusing on their
affordances - what we can do with them (Gibson, 1979).
Both sport and music performance can benefit when attention is directed
to the effects of movements (external focus of attention; distal stimulus)
rather than the movements themselves (internal focus of attention; proximal
stimulus; Wulf & Prinz, 2001). Golf players make faster progress when their
attention is directed to the ball and its goal rather than body movements.
Pianists can be more successful if they concentrate on sound rather than
technique. The sophisticated motor control mechanisms that regulate our
movements are largely unconscious, which allows us to focus on external
goals. But the perception of a motoric goal cannot be separated from
proprioception (kinesthesia) - perception of the relative position of body parts
and corresponding muscular effort. Similarly, pianists’ perception of timbre
cannot be separated from their perception of the gestures used to achieve it.
Research on audiovisual mirror neurons (Kohler et al., 2002) and auditory-
motor interactions (Zatorre et al., 2007) further implies that listeners at a
piano recital share the performer’s proprioception.
Cognition is embodied when it is “deeply dependent upon features of the
physical body of an agent” (Wilson & Foglia, 2011) – a central issue in music
psychology (Leman, 2008). Understanding sound via the body is an example
of conceptual metaphor: ideas in one domain are understood in terms of
ideas in another (Lakoff & Johnson, 1980).
INTERNATIONAL SYMPOSIUM ON PERFORMANCE SCIENCE 003

WEAK SYNESTHESIA IN PERCEPTION AND PERFORMANCE

Pianos and rooms are generally interdependent: anyone who has ever travelled
with a piano knows that the same Steinway or Bösendorfer not only sounds
different in different halls, but also seems to react differently in its mechanism.
Indeed, the resistance of the key, over and above the measurable mechanical
aspect, is a psychological factor. (Brendel; 1976; www.oocities.org)

The physical resistance or bounciness of the piano mechanism may change

slightly from one hall to another depending on temperature and humidity,
but that is not Brendel’s point. He is talking about a “psychological factor”.
Pianists are highly sensitive to the touch-sound relationship; that is a major
aspect of their art. For a pianist, the sense of touch – the sense of the keys
under the fingers – is generally inseparable from the produced sound.
There are many examples of weak synesthesia in music. For example,
musical pitch is understood in spatial terms: it rises and falls. The timbre of a
jazz voice is compared with familiar environmental objects (e.g. “round”) or
the body or the singer (e.g. “relaxed”; Prem & Parncutt, 2008).
Unlike pianists, wind and string players have a high degree of
independent control over the exact pitch and timbre of individual tones. But
even the best performers do not clearly separate intonation from timbre (Ely,
1992; Platt & Racine, 1985); musicians who play with good timbre are judged
by experts and amateurs to have good intonation and vice-versa. Another
example: In the best performances of Renaissance choral music, intonation is
close to 12-tone equal temperament (Devaney et al, 2011) – perhaps because
12ET offers an optimal compromise between the clarity of Pythagorean
tuning (in which scale steps are clear and stable) and just tuning (in which
roughness and beats are minimized). The special feel of just intonation as
idealized by Renaissance music aficionados (Duffin, 2007) may be a timbral
illusion – another example of synesthesia.
Ecological psychology is also relevant for musicology and aesthetics. An
example: Acousmatic music is abstract, electronically synthesized sound
heard from loudspeakers. Listeners constantly guess and imagine sources or
causes of musical sounds – just as we do in everyday life when we hear a
sound that could be important. Electronically generated sounds sound less
strange and more “musical” when we notice their similarity to familiar sounds
and imagine their sources. “[T]he acousmatic curtain does not merely serve to
obscure the sources of sounds. Indeed, it can be seen to intensify our search
for intelligible sources, for likely causal events” (Windsor, p. 31).
004 WWW.PERFORMANCESCIENCE.ORG

Modern approaches to music theory and the psychology of musical

structure have been disembodied by cognitive epistemologies, ignoring
environmental interaction. An ecological approach might start instead with
an empirical study of the relationship between physics and experience, as we
perceive complex tones in real speech and music (Terhardt, 1984).

REDEFINING TIMBRE

These diverse examples suggest that weak synesthesia is the rule rather
than the exception. Anything that we experience in any sensory modality can
be influenced by any other modality. If that is true, we need a new, explicitly
ecological definition of timbre. Timbre depends generally on input from
other senses (weak synesthesia) - not to mention the listener’s relevant
knowledge and expectations, and immediately preceding and following
events. These dependencies are not errors – they are intrinsic to timbre.
Research papers on musical timbre often begin by apologizing about
current definitions. According to the American Standards Association (1960),
timbre is “that attribute of sensation in terms of which a listener can judge
that two sounds having the same loudness and pitch are dissimilar”, and
"timbre depends primarily upon the spectrum of the stimulus, but it also
depends upon the waveform, the sound pressure, the frequency location of
the spectrum, and the temporal characteristics of the stimulus” (cited after
Wikipedia). All of that is true, but it is not the whole story. A more
appropriate definition of timbre might include the following:
1. Like pitch, loudness, and (in vision) color, timbre is purely experiential. It
has no physical existence, but corresponds to physical states and events.
2. Timbre is a holistic property of a sound source or auditory image that can
depend on concurrent input from all relevant senses: hearing, vision,
touch, gesture perception. Our ability to consciously separate sensory
inputs is limited. Timbre often depends on feelings in the body while
performing, or an audience’s projections of those feelings. Timbre can
also be affected by acoustic or other aspects of a listening space,
emotional reactions, and associations with other music or events.
3. A complete description of timbre includes quantitative and qualitative
elements. Both are indispensable, and both are intrinsically vague and
intangible. From a quantitative viewpoint, timbre is multidimensional;
the axis labels are part of timbre’s qualitative description. More generally,
timbre descriptions refer to the physical environment and the human
body, including speech (Traube, 2004).
INTERNATIONAL SYMPOSIUM ON PERFORMANCE SCIENCE 005

4. Like loudness, timbre is a mixture of sound quality (proximal perception)

and sound source quality (distal). Psychoacousticians traditionally study
proximal loudness and timbre in experiments with artificial sounds heard
on headphones, and then consider neural foundations. But in everyday
life and music, loudness and timbre usually refer to sound sources - not
sound as sensation. Timbre generally depends on imagined visual and
tactile properties of sound sources, and the listener’s past experience of
those sources. An example: The temporal and spectral characteristics of
clarinet sound vary enormously from one register to another, but we still
recognize the sound as belonging to one category called “clarinet”. That in
turn suggests that our experience of timbre is also generally influenced by
spontaneous, learned categorizations.
The long-standing failure of scientists and musicians to agree about touch in
piano music may be part of a broader failure to come up with a realistic
operational definition of timbre. This failure is inhibiting interdisciplinary
interaction. One solution might be to inform both sides about basics of
experiential psychophysics and ecological psychology. Another might be to
agree on a new definition of timbre. Scientists may be the most resistant to
change, given the current dominance of the philosophical worldview known
as materialism, according to which the only things that exist are matter and
energy as defined by physicists. But with that worldview it is impossible to
study artistic experience. You cannot study something that does not exist.
The impression that piano timbre depends on gesture, arm weight, and
touch is valid if we accept that experiences exist in their own right, and are
generally multimodal. But we must also agree that in the physical world, the
spectral and temporal envelopes of an isolated piano tone cannot be changed
independently of physical intensity. A rational discussion of the relationship
between physics and experience will become possible when both sides agree
that the previous sentences are complementary and not contradictory. A new
interdisciplinary platform will enable more effective and realistic
investigations of musical interpretation in practice, on the basis of subjective
interactions among performers’ proximal sensations (tactile, auditory, visual,
proprioceptive) and distal perception and cognition (performance space;
communication with the audience; cultural context).

Address for correspondence

Richard Parncutt, Centre for Systematic Musicology, University of Graz, Merangasse 70,
Graz, Austria; Email: parncutt@uni-graz.at
006 WWW.PERFORMANCESCIENCE.ORG

References

American Standards Association (1960). American standard acoustical terminology.

New York: American Standards Association.
Brendel, A. (1976). Musical thoughts & afterthoughts. London: Robson.
Devaney, J., Mandel, M. I., Ellis, D.P.W., & Fujinaga, I. (2011). Automatically extracting
performance data from recordings of trained singers. Psychomusicol, 21, 108–36.
Duffin, R. W. (2007). How equal temperament ruined harmony (and why you should
care). New York: Norton.
Ely, M. C. (1992). Effects of timbre on college woodwind players’ intonation
performance and perception. Journal of Research in Music Education, 40, 158-167.
Gibson, J. J. (1979). The ecological approach to visual perception. Houghton Mifflin.
Goebl, W., Bresin, R., & Galembo, A. (2004). Once again: The perception of piano touch
and tone. Procs International Symposium on Musical Acoustics, Nara, Japan.
Kohler, E., et al. (2002). Hearing sounds, understanding actions: Action representation
in mirror neurons. Science, 297 (5582), 846-848.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: U Chicago Press.
Leman, M. (2008). Embodied music cognition and mediation technology. MIT Press.
Martino, G., & Marks, L. E. (2001). Synesthesia: Strong and weak. Current Directions in
Pyschological Science, 10, 61-65.
Platt, J. R., & Racine, R. J. (1985). Effect of frequency, timbre, experience, and feedback
on musical tuning skills. Perception & Psychophysics, 38, 543-553.
Prem, D., & Parncutt, R. (2008). Corporality in the timbre vocabulary of professional
female jazz vocalists. In M. Marin et al. (Eds.), Procs of SysMus.
Terhardt, E. (1984). The concept of musical consonance: A link between music and
psychoacoustics. Music Perception, 1, 276-295.
Traube, C. (2004). An interdisciplinary study of the timbre of the classical guitar. PhD
thesis, McGill University, Montreal.
Walker, P., et al. (2010). Preverbal infants’ sensitivity to synaesthetic cross-modality
correspondences. Psychological Science, 21, 21-25.
Wilson, R. A. and Foglia, L. (2011). Embodied cognition. In E. N. Zalta (Ed.), Stanford
Encyclopedia of Philosophy. plato.stanford.edu.
Windsor, L. (2000). Beyond the acousmatic: The interpretation of electronic sounds. In
S. Emmerson (Ed.), Music, electronic media and culture (pp. 7-35). Ashgate.
Wulf, G., & Prinz, W. (2001). Directing attention to movement enhances learning: A
review. Psychonomic Bulletin and Review, 8, 648-660.
Zatorre, R. J., Chen, J. L., & Penhune, V. B. (2007). When the brain plays music:
Auditory–motor interactions in music perception and production. Nature Reviews
Neuroscience 8, 547-558.