See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/3230382

Discourse & Technology: Multimodal Discourse Analysis

Article in IEEE Transactions on Professional Communication · October 2005

DOI: 10.1109/TPC.2005.853943 · Source: IEEE Xplore

CITATIONS READS
11 3,439

1 author:

Heloisa Moura
Illinois Institute of Technology
15 PUBLICATIONS 105 CITATIONS

SEE PROFILE

All content following this page was uploaded by Heloisa Moura on 12 April 2015.

The user has requested enhancement of the downloaded file.

IEEE TRANSACTIONS ON PROFESSIONAL COMMUNICATION, VOL. 48, NO. 3, SEPTEMBER 2005
Philip LeVine and Ron Scollon, Editors
Discourse & Technology: Multimodal Discourse Analysis
Book Review
—Reviewed by
HELOISA MOURA
Index Terms—Context, discourse analysis, media, mode, technology.
T he personal computer revolution, together with
advancements in telecommunication, especially
the growth of the world wide web, has increasingly
allowed the integration of new technologies into
people’s daily lives, influencing social interactions,
work organization, and discourse. Within the
academic world, new communication technologies
have impacted the way research is conducted and
the way data is collected and stored. Accordingly,
technological innovations have brought changes to
the field of discourse analysis.
This book presents a collection of papers selected by
peer review for the 53rd Georgetown University Round
Table on Languages and Linguistics, GURT 2002, on
the theme “Discourse and Technology: Multimodal
Discourse Analysis.” The focus of the discussion
is on the twofold impact of new communication
technologies, first on the way discourse analysts
and linguists collect, transcribe, and analyze
discourse data, and second on social interactions and
discourses themselves.
The essays discuss five main sub-themes that were
central in the conference: (1) the effect of technology
on discourse and multimodal discourse analysis,
(2) the role of the web in discourse analysis, (3)
multimodal discourse analysis in studies of social
actions and interactions, (4) multimodal discourse
analysis in educational social interactions, and (5)
the use of multimodal discourse analysis in doing
analyses in workplaces. Together, they emphasize
that all discourse or language in use is multimodal,
that is, “is always and inevitably constructed across
multiple modes of communication” [p. 2]. The essays,
however, are not strictly organized according to these
sub-themes; as the editors state, several papers fall
under more than one category, and one essay is not
Manuscript received May 9, 2005; revised May 16, 2005.
The reviewer is with the Institute of Design,
Illinois Institute of Technology, Chicago, IL 60616 USA
(email: moura@id.iit.edu).
IEEE DOI 10.1109/TPC.2005.853943
Book Publisher: Washington, DC: Georgetown University Press,
2004, 229 pp., no index.
0361-1434/$20.00 © 2005 IEEE
329
associated with any of the five sub-themes. For the
purpose of this review, each essay will be discussed
as a representative of a single theme, with emphasis
given to the most representative papers within each
group.
In the first chapter, “Multimodal discourse analysis
as the confluence of discourse and technology,” the
editors Ron Scollon (professor of linguistics and senior
faculty research fellow at Georgetown University) and
Philip LeVine (a Pulitzer Prize-winning poet who has
taught English and creative writing for over 30 years
at California State University in Fresno and for several
years has been a visiting professor in creative writing
at New York University) open the discussion. The
authors provide an overview of the book, organizing
the papers in the five areas described above, and
stressing the multimodality of social action.
The first area, “Why should we study discourse and
technology and multimodal discourse analysis?”
argues that discourse is inherently multimodal.
Essays relating to this theme include chapters 2
(“Ten reasons why linguists should pay attention
to visual communication,” by Theo Van Leeuwen,
Cardiff University), 3 (“The problem of context in
computer-mediated communication,” by Rodney
Jones, City University of Hong Kong), and 13 (“The
moral spectator: Distant suffering in live footage
of September 11, 2001,” by Lilie Chouliaraki,
University of Copenhagen). Van Leeuwen emphasizes
that face-to-face communication is a multilayered
communicative act “whose illocutionary force comes
about through the fusion of all the component
semiotic modalities: dress, grooming, facial
expression, gaze, gesture” [p. 7]. In his analysis,
Van Leeuwen uses a famous poster to illustrate the
blending of different modes, image and text, which,
together, form a single multimodal communicative
act, rather than separate speech and image acts.
Jones explores perspectives on context, and
affirms that, although context is accorded
importance in the field of linguistics, it is often
conveniently not addressed in studies of computer
mediated communication. Jones argues that new
communication technologies are forcing linguists
330 IEEE TRANSACTIONS ON PROFESSIONAL COMMUNICATION, VOL. 48, NO. 3, SEPTEMBER 2005
to examine the professional constraints imposed
by technology, such as the tendency to privilege
written and spoken texts over other phenomena,
and to consider objects, actions, and people as
the only elements making up the environment.
The author points out the need for reviewing the
approaches to context, which should be understood
as a function of interaction and negotiation. He
also claims the need to challenge the dichotomies
upon which basic assumptions about computer
mediated communication rest, which separate, for
example, virtual from real and sender from receiver.
Multimodal discourse is also analyzed in chapter
13, where Chouliaraki discusses extracts from live
television footage on the events of September 11,
2001. The author explores the role of the spectator
as a moral subject and points out how pity becomes
eloquent through the multimodal and multifunctional
semiotics of television.
Chapters 4, 5, and 6 relate to the second sub-theme,
“The role of the web in discourse analysis.” Essays
in this group discuss the new forms of discourse
and discourse analysis enabled by the web. Chapter
4, “‘The way to write a phone call’: Multimodality in
novices’ use and perceptions of interactive written
discourse (IWD),” was written by Angela Goddard,
Manchester Metropolitan University. It reports a study
on the nature of IWD, as produced and experienced
by users, focusing on the multimodality, constraints,
and affordances of different communication systems.
The fifth chapter, “Trying on voices: Using questions
to establish authority, identity, and recipient design
in electronic discourse,” by Boyd Davis and Peyton
Mason, University of North Carolina, looks at choices
and adaptation of conversational practices in the
electronic medium, examining the communicative and
ritual functions of rhetorical and leading questions
in asynchronous electronic discourse, and exploring
gender differences. The analysis suggests the use
of different strategies for presenting oneself during
online conferences, such as suitable professional
voice. The sixth essay, “Mock Taiwanese-accented
Mandarin in the internet community in Taiwan: The
interaction between technology, linguistic practice,
and language ideologies,” by His-Yao Su, University
of Texas at Austin, looks at the web as space that
offers opportunities for playing with social categories,
such as gender, socioeconomic status, and age. The
author focuses on a linguistic practice that probably
originated in Taiwan, where “Taiwanese internet
users draw on the stereotypical linkage between
Taiwanese-accented Mandarin in speech and the
multiple social meanings associated with such an
accent to create a new form of language play” [p. 59].
The essays related to “Multimodal discourse analysis
in studies of social actions and interactions” look
at social interactions, captured mainly through
video recording, highlighting the importance of
multiple modes during data analysis. In chapter 8,
“The multimodal negotiation of service encounters,”
Lurent Fillettaz, University of Geneva, discusses the
complex articulation of speech, gesture, action, and
material setting, focusing on the impact of nonverbal
behavior on the construction of service encounters.
In chapter 9, “Multimodal discourse analysis: A
conceptual framework,” Sigrid Norris, Georgetown
University, introduces a multimodal framework for
discourse analysis, moving away from the focus on
spoken language and looking at the interplay between
language and context. This conceptual framework
“allows for the integration of all heuristically
identifiable communicative modes and the analysis
of concurrent higher-level actions that a social actor
performs” [p. 113]; it “permits the incorporation of
all identifiable communicative modes, embodied
or disembodied, that social actors orchestrate in
face-to-face interactions” [p. 101]. The framework
grew from the realization that analyzing one mode
of communication without the others leaves out a
large part of what is being communicated, and that
communication is both achieved through different
modes separately and together at the same time.
Within this framework, the author discusses modal
density and the foreground–background continuum.
Chapter 10, “Files, forms, and fonts: Mediational
means and identity negotiation in immigration
interviews,” by Alexandra Johnston, Georgetown
University, analyzes an adversarial interview between
a U.S. Immigration and Naturalization Service
officer and applicants for permanent visa, aiming
to understand how gatekeepers make conclusions
about the veracity of the information presented by
applicants that can lead to approval or denial of a visa
request. In chapter 11, “Modalities of turn-taking in
blind/sighted interaction: Better to be seen and not
heard?” Elisa Everts discusses the perpetuation of
the oppression and marginalization of various groups
of people through the use of language. The author
explores the interaction between a blind participant
and several sighted participants, focusing on the
ways sighted people’s language use discriminates
against blind people by privileging certain modalities
of discourse over others. In chapter 14, “Ethnography
of language in the age of video: ‘Voices’ as multimodal
constructions in some contexts of religious and
clinical authority,” Joel Kuipers argues that, while
speaking, people process, manage, and control
information relying on a wide variety of information
sources, including acoustic, visual, gustatory, and
linguistic ones. Rather than debating the primacy
of one over another, the author explores “a form of
ethnographic holism by placing spoken discourse in
a multimodal context” [p. 167]. For that purpose, he
looks at participants in religious rituals and clinical
encounters.
IEEE TRANSACTIONS ON PROFESSIONAL COMMUNICATION, VOL. 48, NO. 3, SEPTEMBER 2005
The fourth sub-theme, “Multimodal discourse
analysis in educational social interactions,”
includes papers that use data involving students
in educational institutions. Chapter 12, “‘Informed
consent’ and other ethical conundrums in videotaping
interactions,” by Elaine Yakura, Michigan State
University, explores ethical issues concerning
videotaping and common understandings of informed
consent. The author discusses some possible solutions
for these problems, suggesting collaboration among
researchers and participants rather than a one-way
flow of information. In chapter 15, “Multimodality
and new communication technologies,” Carey Jewit,
University of London, discusses the impact of new
communication technologies on social interaction
and discourse and its relation to multimodality.
The author explores the different affordances
of technologies of dissemination (media) and
technologies of representation (mode), as resources
for meaning-making, focusing on the analysis of three
computer-programming applications. Chapter 16,
“Origins: A brief intellectual and technological history
of the emergence of multimodal discourse analysis,”
by Frederick Erickson, UCLA, presents a selective
history of the study of talk in social interaction,
emphasizing the role of information technology and
pointing at the ways in which discourse is being
transformed.
The fifth and last area, “The use of multimodal
discourse analysis in doing our analyses in
workplaces,” focuses on sites where participants
are engaged in tasks with real world material
outcomes and illustrates how new technologies of
video recording are opening new areas of research. In
View publication stats
331
chapter 7, “Materiality in discourse: The influence
of space and layout in making meaning,” Ingrid de
Saint-Georges, Georgetown University, empirically
examines the interrelations between material and
different semiotic processes, such as gestures,
language, actions, physical layout, space, time, and
images, emphasizing the multimodal turn in discourse
analysis. Chapter 17, “Studying workscapes,” by
Marilyn Whalen, Jack Whalen, and colleagues, Palo
Alto Research Center, looks at interactions within
workplaces, exploring the role of naturalistic research
in the human-centered design of technology.
The volume successfully supports the argument that
all discourse is multimodal, and brushes, to a greater
or lesser extent, the five sub-themes that organize
the articles. One shortcoming, though, is that the
book does not build a unified framework for analysis
of multimodal interaction, but, perhaps, that was
never its intent. Another problem is the absence of a
glossary identifying key concepts, or an index of main
terms.
In summary, this book includes a rich collection of
essays covering varied areas relevant to the impact
of technology on discourse and discourse research.
The 17 chapters give a broad view of the field and
the changes that are taking place within it, exploring
multiple threads of the discourse. In addition to
linguists, researchers in human computer interaction,
communication, cognitive psychology and design will
also benefit from this book. Overall, it makes for
insightful reading that will motivate both who are new
to the discussion on multimodal discourse analysis
and those who are experts.