Our Senses, True Jacks of All Trades

by Nastasia Griffioen

Late 1970s. Another day at the office for Sue Thomas.
A couple of weeks ago she had been hired by the FBI
to analyse finger-prints, a job she had quickly come to
find terribly tedious. She is not a regular employee
though Sue is deaf. The FBI had reasoned that a
deaf person would have an easier time staying focus-
ed during the notoriously meticulous task. However,
today Sue was summoned to her boss office for what
she expected would be the end of her career at the
FBI something she wouldnt have minded at the
time; she had almost walked out on herself anyway.
Instead, Sue was asked to interpret a silent
video recording of a conversation, which she did
effortlessly. Next thing she knew, she was
redistributed to the lip-reading department of the
Bureau, becoming the first deaf lip-reading expert in
the history of the FBI.
Like many other deaf people, Sue had started
learning to read lips since the day she lost her hearing,
leaving her with vision as one of the
few means by which she could
understand others speech.
However, people without any
auditory impairment whatsoever
also rely on their eyes for help in
understanding others
for a great deal.

Do You See What Im Saying?

Apparently, we do not only use our
ears to hear what others are saying. Instead, we
derive a huge amount of speech information from the
movement of lips, a purely visual source. This
codependence of senses can easily be detected in
everyday life: for one, eye-tracking studies have
demonstrated that people pay a great deal of attention
to the lips of the speaker when conversing with
someone. Looking at the movements of the mouth
helps us distinguish between sounds and syllables that
might otherwise be easily confused. This proves to be
especially handy when we encounter someone
speaking a language we are not sufficiently proficient
in or even when we meet someone with an unfamiliar
or profound accent.



















Considering this visual-auditory link its hardly
surprising that the visual cortex of the brain of
blind people what with being out of a job and all
partly takes on the processing of auditory stimuli.
The brain proves to be a highly plastic organ when
things dont go the way Mother Nature intended.
However, not only our vision and hearing team up
to face the commonplace challenge of dealing with
external stimuli.

A Pinch of Colour

In fact, blindfolding people for as little as 90 minutes
appears to prime the visual cortex for responding to
touch. The purpose of this is to heighten sensitivity
to touch and this is actually achieved by involving
the visual cortex. The implications of sensory
collaborations like this are even relevant for a
significant amount of people in their everyday lives:
nearsightedness often enhances auditory and
spatial skills, even if glasses are worn (which leave a
good part of the visual periphery blurry).

Guitar Licks

Another notable link is the one between
taste and hearing. Researchers have
managed to bias the results of taste tests
by simply tweaking what people hear
when they are eating. This finding is
especially suitable for commercial
implementation: I imagine many
companies have already hired
professionals specialised in sensory
ideals what people like to feel and hear
when interacting with a given product. Not only do
sounds interact with taste, sights also have the
capacity of changing flavours: orange flavoured
drink will taste of cherry if it is tinted red and vice
versa.

Promiscuous Perception

So obviously, the last decade of research has shown
us that the idea of the brain as a Swiss Army Knife
in which many distinct regions are solely dedicated
to different senses proves to be faulty. Rather, our
brain derives meaning from the world in as many
ways as possible by blending different forms of
sensory perception, like a true Jack of all trades.
Further Reading

The Role of Auditory Cues in Modulating the Perceived Crispness
and Staleness of Potato Chips. Massimiliano Zampini and Charles
Spence in Journal of Sensory Studies, Vol. 19, No. 5, pp. 347-363;
October 2004.

Speech Perception as a Multimodal Phenomenon. Lawrence D.
Rosenbaum in Current Directions in Psychological Science, Vol. 17,
No. 6, pp. 405-409; December 2008.