Professional Documents
Culture Documents
ZKM SA Supper 2017-08-17
ZKM SA Supper 2017-08-17
Sonification and
the Relation
between Science
and Art
During an event related to the David Bowie is exhibition in April 2013, the musicians
Alexis Kirke and Martyn Ware took the stage with an eccentric set of compositions
at the Victoria and Albert Museum in London. Their pieces, which they referred
to as a “career sonification” of David Bowie, translated statistical analyses of
Bowie’s work over the years into a series of sometimes quite dissonant-sounding
electronic pieces. One piece used an analysis of the emotional content of Bowie’s
lyrics to drive a “hyperspeed piano” tune, while another mapped his album sales
onto pitch – resulting in both painfully high tones and low rumbling sounds, to mark
the huge fluctuation in commercial success over the years.(FIG. 1)1
Alexandra Supper 11
S
inging Stars and Banging
Brains: Musical Precedents
for Sonification
12
4
to sonification, declaring Lucier’s piece and practices can make important See Douglas Kahn, “Alvin
similar compositions as artistic forerunners of contributions to sonification Lucier, Edmond Dewand
und Music for Solo Per-
sonification.7 research.9 At the same time, former,” in Klangmaschinen
zwischen Experiment und
there is widespread agreement Medientechnik, ed. Daniel
(even among artists and com Gethmann (Bielefeld:
transcript Verlag, 2010),
Near, but out: Demarcating posers) within the community 211–29; Andi Schoon and
Florian Dombois, “Soni-
Sonification from Music that sonification should not fication in Music,” in Pro-
be equated with music, and ceedings of the 15th Inter-
national Conference on
The musical examples discussed in the previous there have been some pleas to Auditory Display, May 18–
22, 2009, Copenhagen,
section have predated the coining of the term limit the definition of the term Denmark, http://hdl.handle.
“sonification,” and have emerged independently “sonification” to cases where net/1853/51415; Volker
Straebel, “The Sonification
of the dedicated sonification research community. it is used systematically as a Metaphor in Instrumental
Music and Sonification’s
This community has developed over the last three scientific method, effectively Romantic Implications,”
decades and has found its institutional embodi excluding artistic approaches.10 in Proceedings of the 16th
International Conference
ment in the International Community for Auditory The research commu on Auditory Display, June
9–15, 2010, Washington,
Display (ICAD), founded in 1992. It convenes at nity dedicated to sonification DC, http://hdl.handle.net/
annual conferences, in which an interdisciplinary thus engages in a delicate bal 1853/50066; Alexandra
Supper, “Sublime Frequen-
group of researchers – ranging from psychologists ancing act to position itself in cies: The Construction of
Sublime Listening Experi-
to physicists, and from computer scientists to relation to artistic approaches. ences in the Sonification
composers – come together to present their latest They seek contact with sound of Scientific Data,” Social
Studies of Science 44,
research about the use of non-speech sound to artists and musicians, but are no. 1 (2014): 34–58.
5
display data and convey information. careful to present sonification Richard Taruskin, The
Much of the work of the community as something other than music Oxford History of Western
Music. Volume 5: The Late
concerns the systematic technical development or sound art. Quite characteristi Twentieth Century (New
York: Oxford University
and evaluation of techniques for transforming cally for this delicate balancing Press, 2005), 55. Also see
data into sound, especially ones that can be used act, the sonification researchers Ian Willcock, “Composing
Without Composers?
by scientific specialists to complement existing, Gerold Baier and Thomas Creation, Control, and Indi-
viduality in Computer-based
usually visual, data analysis techniques. Many Hermann have welcomed their Algorithmic Composition,”
scientists outside of this research community, how audience at a presentation in in Electronics in New Music,
ed. Claus-Steffen Mahnkopf,
ever, react with scepticism to the idea of listening the context of a festival for con Frank Cox, and Wolfram
Schurig (Hofheim: Wolke
to data: the notion that only seeing can generate temporary music by expressing Verlag, 2006), 221–35.
objective insights, while hearing creates sub their gratitude for the audience’s 6
Indeed, in a personal inter-
jective impressions, is widespread. Consequently, interest “even though this has view I conducted with
Lucier (Vienna, November 1,
an important activity of the sonification com nothing to do with music.”11 The 2008), he mentioned that
munity consists in “lobbying for the ear”: seek- fact that they regard their work his desire to explore such
phenomena through his
ing to establish sonification as a scientific and on the sonification of brain work was a point of dis-
agreement between him and
objective method by emphasizing the capabilities waves as having nothing to do John Cage, who accused
of the human hearing system in systematically with music was apparently him of being too invested in
cause-and-effect relation-
processing information.8 In order to understand no reason for them to turn down ships rather than sound on
its own terms.
their strategies for doing so, the way in which an invitation to present their 7
they draw the boundaries between science and art work at a music festival; quite See Volker Straebel and
Wilm Thoben, “Alvin Lucier’s
is revealing. on the contrary, they very enthu Music for Solo Performer:
Experimental Music Beyond
From the very beginnings of this com siastically took this chance to Sonification,” Organised
munity, composers and musicians have been introduce an artistically informed Sound 19, no. 1 (2014): 17–29.
8
prominently involved in sonification research; audience to the principles of See Supper, “The Search
for the ‘Killer Application’”
and to this day, a broad consensus exists that sonification while also caution (see note 2, above); Alexandra
the involvement of artists and composers is bene ing them that it is not the same Supper, “Lobbying for the
Ear: The Public Fascination
ficial for the success of sonification: not just as music. with and Academic Legiti
macy of the Sonification of
because they can help to create public interest, Similarly, many of Scientific Data” (PhD diss.,
but also because their skills related to technical the annual sonification confer Maastricht University, 2012).
9
implementation, aesthetic design, and listening ences organized by the ICAD Ibid.
13
community feature an evening programme with
concerts, often with sonification-based music.
The inclusion of these concerts stimulate cross-
pollination between artistic and scientific
applications of sonification; but they also convey
the message that such artistic applications are
best suited to the evening programme rather than
the scientific sessions during the day. Art and
music are thus kept near, but out.12
Alexandra Supper 14
and musicians to do more than that. Unlike the
musical examples mentioned earlier, these artistic
sonification projects have been inspired and influ
enced by scientific terminology and approaches
for translating data into sound; yet, as I want to
show, they also open new possibilities for thinking
about scientific phenomena. With a few examples
of sonifications of astronomical data, I show
how they can open up a space for epistemological
reflection. Subsequently, I discuss how sonifica
tions of geophysical data have been used to raise
environmental awareness.
The British artist duo Semiconductor
have used the sonification (and visualization) of
solar astronomical data as the basis of their
multimedia installations Brilliant Noise and Black
Rain.(FIGS. 2a-c, 3a-d),14 As the artists explain in a video
interview for the Creators Project, they are inter
ested in exploring “how man experiences the
nature of the physical world around him and how
the tools of science come into that.”15 An import
ant aspect of this exploration is that both of the
aforementioned pieces work with raw data from
satellite transmissions, rather than cleaned-up
and processed datasets. Aesthetically, the grainy
texture of the resulting sounds and images pro
vides a stark counterpoint to the frequently quite
polished and glossy representations of outer
space in the popular scientific imagination. As art
historian Inge Hinterwaldner points out, “Semi
conductor embraces what is normally filtered
out”:16 the noises and distortions that are usually
painstakingly removed from scientific datasets
become an essential part of the work of art. By
leaving in and drawing attention to artefacts and
glitches, the artists also remind us “of the pres
ence of the human observer who endeavours to
extend our perceptions and knowledge through
technological innovation,”17 and thus of the many
human interventions that make scientific repre
sentation possible in the first place.
The piece Bonner Durchmusterung [Bonn
Patternization] by electronic composer Marcus
Schmickler (in collaboration with composer/
sonification researcher Alberto de Campo, visual
artist Carsten Goertz, and astronomer Michael
Geffert) similarly uses sonification as a space
for epistemological reflection.(FIG. 4) Unlike in the
films of Semiconductor, the sounds of Bonner
FIGS. 3a–d Semiconductor, Black Rain, 2009. Filmic installa Durchmusterung are not driven by its images.
tion, 03:00 min. / 17:00 min. loop, single channel and installation.
A Semiconductor work by Ruth Jarman and Joe Gerhardt. If anything, the piece reverses the usual hierarchy
15
FIG. 4 Marcus Schmickler
et al., Bonner Durchmus-
terung , 2009. Composition
based on sonifications
of astronomical models and
data with projections, 10.2
channel audio, projections,
33 mins. Commissioned by
IYA 2009 (International Year
of Astronomy) and Deutscher
Musikrat.
between sound and vision: considering the prom models and visualizations in astronomical
inent role of visualization in the scientific enter research, and call into question “the relationship
prise, one might expect that the visual projection between data and reality of the observed objects.”19
aims for a more objective representation of How, for instance, does one come from a com
scientific facts, while the sonification serves as plex series of numbers to an understanding of the
an illustrative soundtrack. In Bonner Durchmuste objects or even to a consistent phenomenology
rung, however, much more artistic license was of the cosmos? The essay concludes by encourag
taken with the visualization than with the sonifi ing artists and scientists to work together on a
cation. While the sounds are quite straightfor mutually enriching “poetic interpretation, depiction
wardly driven by scientific data and contextualization of scientific knowledge.”
14 sets and sound fairly abstract, Indeed, in my interview with the composer, he
See http://
semiconductorfilms.com/ the visualization “treats the stressed that he was interested in sonification as a
art/brilliant-noise/ and subjects more freely or asso means of triggering interdisciplinary reflexivity.20
http://semiconductorfilms.
com/art/black-rain/. ciatively,”18 evoking concrete,
15
See video file on http:// urban spaces.
thecreatorsproject.
vice.com/blog/meet-
Sonification in Sound Art: Sonic
semiconductor-the-mad- The programme notes for the Ecologies
scientists-of-the-visual-
art-realm. piece not only briefly explain
16 the ten astronomical phenom Not only astrophysical data have inspired sound
Inge Hinterwaldner, “Semi-
conductor’s Landscapes ena that have been sonified artists and musicians in their work, however. In
as Sound-Sculptured
Time-Based Visualizations,” (including solar eruptions and this section, I want to discuss several examples of
Technoetic Arts: A Journal gamma ray bursts), but also artistic sonifications based on data from another
of Speculative Research 12,
no. 1 (2014): 15–38, here p. 30. promise “an epistemological domain, that of geophysics. With their Aftershock
17
See http:// exchange between music and sound installation,(FIGS. 5a-c) which debuted at the
semiconductorfilms.com/ astronomy.” Starting out from Norwegian art gallery ROM in 2011, the sound
art/black-rain/.
18 the statement that even “raw artist Natasha Barrett and geoscientist Karen Mair
Interview by the author
with Marcus Schmickler, data are deeply dependent on a promise listeners the ability to “experience the
Cologne, April 15, 2010. theoretical model of the world intricacies of the inherently 3D process of rock
19
See http://patternization. on which the measuring pro deformation from the inside.” Aftershock is in fact
org/ for the quoted
program notes and some cedure is based,” Schmickler not a single sound installation, but rather a set of
sound examples. and his collaborators reflect four interrelated works, which are placed in spa
20
Interview with Marcus on the role of mathematical tially separate but adjacent zones of the exhibition
Schmickler.
Alexandra Supper 16
FIGS. 5a–c Natasha Barrett and
Karen Mair, Aftershock , 2011.
Interactive sound art installation.
17
21 space, thus allowing the sound yet he also insists that it is “a work of art” rather
See Natasha Barrett and
Karen Mair, “Aftershock: of the four installations to than a “scientific demonstration of natural pheno
A scienceart collaboration
through sonification,” intermingle. The installation is mena.” What’s more, Adams points out, the piece
Organised Sound 19, no. 1 meant to encourage an active does not aspire to simulate the natural world,
(2014): 4–16. Also see www.
natashabarrett.org/crush3. exploration of geophysical but rather “is a heightened form of experience
html for more information
and a short video. phenomena; in fact, the instal itself.” Unlike Aftershock, the piece continuously
22 lation remains silent unless an emits sound (and light) whether a listener is pres
Kyle Gann, “Fairbanks:
A Long Ride in a Slow Ma- “exploring listener” is present ent or not, but nonetheless, the listener’s experi
chine,” NewMusicBox (2006),
www.newmusicbox.org/ and triggers the sounds.21 ence of a sense of place stands central according
articles/fairbanks-a-long- Aftershock is not the to the composer: “The essence of this work is
ride-in-a-slow-machine/.
23 only recent example of a sound the sounding of natural forces interacting with the
John Luther Adams, The
Place Where You Go to Listen:installation based on geophys consciousness of the listener.”23
In Search of an Ecology of ical data. The sound-and-light In a recent article about the piece, musi
Music (Middletown: Wesleyan
installation The Place Where
University Press, 2009), 5, 8. cologist Tyler Kinnear suggests that Adams’s
24
Tyler Kinnear, “Voicing You Go to Listen by composer sounding of the inaudible voices of nature con
Nature in John Luther John Luther Adams, installed tains a “gesture of environmental advocacy.”24
Adams’s The Place Where
You Go to Listen,” Organ- at the University of Alaska’s Adams, argues Kinnear, “is not interested in
ised Sound 17, no. 3 (2012):
230–39, here p. 237. Museum of the North in Fair representing an environment; rather, he seeks to
25 banks, is even more ambitious transform it.”25 Indeed, Adams deliberately tried
Ibid., 231.
26 in its translation of geoscientific to avoid “conventional modes of musical expres
Adams, The Place Where You
Go to Listen, 139. Also see data into sound.(FIG. 6) Rather sion and picturesque evocations of nature,”26 and
David Rothenberg, “Go There than sonifying a single dataset, thus used sine tones and filtered noise, rather than
to Listen: How Music Based
on Nature Might Not Need Adams has opted to tackle sounds that mimic natural processes or musical
Natural Sounds,” in The
Farthest Place: The Music several at once, combining seis instruments. Rather than offering a simulation
of John Luther Adams, mological, geomagnetic, and of natural phenomena, Adams tries to evoke an
ed. Bernd Herzogenrath
(Boston: Northeastern Uni- meteorological datasets – all of experience of space and the environment which
versity Press, 2012), 107–15.
27
which are translated into sound is fundamentally driven by ecological concerns
Adams, The Place Where You in real time. The result is a about “overpopulation, overconsumption, pollu
Go to Listen, 9.
28 complex sonic tapestry based tion, deforestation and widespread extinction.”27
Ibid.
29
on a number of scientific Climate change and environmental destruction are
Andrea Polli, “Soundscape, parameters (from the current recurrent themes of the composer’s diary written
Sonification, and Sound
Activism,” AI & Society 27, position of the sun and moon during the making of the installation.28
no. 2 (2012): 257–68.
30
to seismic and geomagnetic John Luther Adams is not the only
See www.andreapolli.com/ activity), described as follows composer or sound artist who has drawn upon
centralpark.
by music critic Kyle Gann: sonification to express environmental and eco
“You walk in, separate yourself logical concerns: such issues form a red thread
from the world directly outside, sit on the bench, running through the works of sound artist Andrea
and slip into the red-and-violet, or blue-and- Polli – in a recent journal article, Polli muses
yellow, moods of the five glass panels in front about the potential of sonification to raise envi
of you. A continual hum greets you, and after a ronmental awareness.29 For instance, her 2009
moment you begin to sort out the strands of album Sonic Antarctica(FIG. 7) combines meteo
the complex tapestry that the hum turns out to rological sonifications, field recordings, and
be. There are sustained chords, an intermittent snippets of interviews with climate researchers
rattle of deep bells overhead, and an irregular who talk not only about their scientific findings,
boom of extremely low frequencies that you have but also about their environmental motivations;
to focus on to remain aware of.”22 another project, called Heat and the Heartbeat of
In a book companion to the installation, the City,(FIGS. 8a, b) makes concerns about climate
Adams emphasizes that the piece transforms change audible by translating predicted tempera
“inaudible forces of nature into audible sound” ture changes in New York’s Central Park into
and that these forces therefore drive some of sound.30 Both of these projects engage in a dialogue
the decisions traditionally made by the composer; with the scientists studying the phenomena that
Alexandra Supper 18
FIG. 7 Andrea Polli, Sonic Antarctica , 2007/2008. Project including a radio broadcast, live performance
as well as a sound and visual installation.
19
Polli translates into sound, and in both cases, The latter, in particular, is often neglected in
the voices of those scientists have become part of discussions about the value of interdisciplinary
the works themselves. If Polli uses her work to and art-science research.
raise environmental awareness, then, that does The distinction between different logics
not just mean that she draws attention to those of interdisciplinarity can illuminate how the
issues; her work also becomes a place for inter relationship between science and art is negotiated
disciplinary dialogue between artists and scientists within the scientific community dedicated to
on issues related to the environment. sonification research. As I have discussed, this
community values the contributions of musicians
Conclusions and composers for increasing the public appeal
and accessibility (logic of accountability), as well
Contrary to the musical examples in the first as the technical and aesthetic design of sonifi
section of this essay, all of the artistic projects cation (logic of innovation). But at the same time,
described in the previous two sections were the boundary between scientific and artistic
to some extent informed by scientific termin- uses of sonification, while interpreted flexibly,
ology and approaches for sonifying data – and is not altogether dissolved: the community is
indeed, in several cases, the artistic projects have engaged in a careful balancing act between science
fed back into scientific research. For instance, and art. This balancing act is constitutive of
the Aftershock collaboration, which started the (inter)disciplinary identity of the sonification
out as an artistic collaboration between sound community, and fuels many of the debates going
artist Barrett and geophysicist Mair, has led to on within the community about what sonification
a scientific poster presentation at the 2014 general is and should be. In other words, encounters
assembly of the European Geosciences Union,31 with sound art and music help the sonification
while sound artist Andrea Polli has presented community to rethink the nature of their work
one of her projects to an audience of sonification and its relation with its objects and publics – the
specialists at the 2004 ICAD conference.32 logic of ontology is at play within the sonification
Clearly, then, sonification is a domain community, too.
that opens up possibilities for exchange and However, the relationship between science
interactions between science and art. These and art is not one-sided: the case of sonification
exchanges and interactions can take a myriad of shows that not only science can benefit from
forms. In an article about the emerging practice the engagement with artistic practices, but that
of art-science (though not primarily in relation the reverse is also true. Indeed, the three different
to sound), Georgina Born and logics of interdisciplinarity also run in the oppo
31
See http://meetingorganizer. Andrew Barry have identified site direction: for artists – especially those work
copernicus.org/EGU2014/ three different “logics” accord ing in traditions that lack mainstream exposure –
EGU2014-4489.pdf. Original-
ly, Barrett and Mair had also ing to which art-science initia the engagement with scientific topics can also
proposed a special session
dedicated to sonification, tives can operate: the logic help to attract new audiences (logic of account
but the session was eventu- of accountability, in which art ability), while encounters with scientists working
ally withdrawn (see http://
meetingorganizer.coper- helps to make science account on sonification can lead to new tools that can
nicus.org/EGU2014/ses-
sion/15515). able and accessible to a broad also be used in artistic contexts (logic of innova
32 public; the logic of innovation, tion). Perhaps most intriguingly, however, the
Andrea Polli, “Atmospherics/
Weather Works: A Multi- in which interdisciplinary engagement of composers and sound artists with
Channel Storm Sonification
Project,” in Proceedings research fuels industrial inno scientific data can also take the form of a logic
of the 10th International vation and economic growth; of ontology. Indeed, as many of the examples in
Conference on Auditory Dis-
play, July 6–9, 2004, Sydney, and the logic of ontology, this essay have shown, their ability (or at least
Australia, www.icad.org/
websiteV2.0/Conferences/ directed at altering “ways of intention) to stimulate epistemological and onto
ICAD2004/papers/polli.pdf. thinking about the nature logical reflection, and to second-guess taken-for-
33
Georgina Born and Andrew of art and science” and “trans granted scientific assumptions and conventions,
Barry, “Art-Science: From
Public Understanding to forming the relations between is a particularly fascinating aspect of many artistic
Public Experiment,” Journal artists and scientists and sonification projects.
of Cultural Economy 3, no. 1
(2010): 103–19, here p. 105. their objects and publics.”33
Alexandra Supper 20