Texto2 - Pozzer - and - Roth - 2019 Texto 1

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 40

Cultural Studies of Science Education

https://doi.org/10.1007/s11422-019-09910-5

ORIGINAL PAPER

A cultural‑historical perspective on the multimodal


development of concepts in science lectures

Lilian Pozzer1   · Wolff‑Michael Roth2

Received: 9 May 2018 / Accepted: 3 January 2019


© Springer Nature B.V. 2019

Abstract
As part of a series of investigations in which we explore the integration of verbal and non-
verbal aspects of communication into a dialectical, sense-constitutive unit during science
lectures, this study adapts the notions of catchments (i.e., repetitions of essential features of
the gesture-speech dialectic) and growth points (i.e., moments in which ideas in the form of
a gesture-speech dialectic are born) to analyze, from a multimodal communicative perspec-
tive, the articulation and development of scientific concepts in the course of several con-
secutive lessons dealing with the circulatory system. The presence of catchments and the
identification of growth points within and across lessons allow us to understand how sci-
entific concepts are instantiated (i.e., taught) in and during science lectures. The results of
our analysis make evident the dialectical relations between the various semiotic resources
that are integrated into the communicative unit, helping us elucidate teaching of scientific
concepts as a process, a drama in several acts, that unfolds in time and across consecutive
lectures.

Keywords  Multimodality · Gestures · Dialectics · Cultural-historical activity theory ·


Speech activity

Executive summary

Como parte de uma série de estudos que tem como objetivo investigar de que forma os
aspectos verbais e não verbais da comunicação oral são integrados dentro de uma uni-
dade de significação dialética durante aulas de ciências primariamente expositivas,  este
estudo  adapta noções de catchments (repetições de elementos essenciais do par dialético

Lead Editor: M. J. Reiss.

* Lilian Pozzer
Lilian.Pozzer@umanitoba.ca
Wolff‑Michael Roth
mroth@uvic.ca
1
Department of Curriculum, Teaching and Learning, Faculty of Education, University of Manitoba,
Winnipeg, MB R3T 2N2, Canada
2
MacLaurin Building, A567, University of Victoria, Victoria, BC V8P 5C2, Canada

13
Vol.:(0123456789)
L. Pozzer, W.-M. Roth

gesto-discurso) e growth points (momentos nos quais ideias na forma de um par dialético
gesto-discurso emergem), para analisar, a partir de uma perspectiva comunicativa multi-
modal, a articulação e o desenvolvimento de conceitos científicos ao longo de várias lições
consecutivas abordando o tópico do Sistema Circulatório. A presença de catchments e
a identificação de growth points na mesma lição e ao longo de várias lições consecuti-
vas permite-nos entender como conceitos científicos são realizados (ou seja, ensinados)
durante aulas de ciência expositivas. Os resultados de nossa análise evidenciam a relação
dialética entre os vários signos semióticos que são integrados na unidade de significação
comunicativa, auxiliando-nos a compreender o ensino de conceitos científicos como um
processo, um drama em vários atos, que se desenvolve ao longo do tempo e ao longo de
várias lições de ciências.

Thought always is something whole, in extension and range something much


larger than the individual word. A speaker often develops one and the same
thought over the course of several minutes. In his thinking this thought is rep-
resented as a whole and does not slowly develop in sections in which his speech
develops. What exists in thought simultaneously, successively unfolds in speech.
(Vygotskij 2005, p. 1012)
Concepts are the currency of science. Science concepts, at the secondary and univer-
sity levels, are often taught by means of lectures, which make available to students several
multimodal resources, such as, for example, speech, gestures, body orientations, facial
expressions, prosody, videos, three-dimensional models, drawings, diagrams, graphs, and
photographs. Although much maligned, lectures have an integral place in the reproduction
of science and its historical continuity (Roth and Friesen 2014). During lectures, there-
fore, the communication of scientific concepts occurs along trajectories driven by the dia-
lectical relation among the various semiotic (that is, meaning-making) resources lecturers
use, which together constitute a communicative unit (Pozzer-Ardenghi and Roth 2007).
In this sense, concepts are performed during lectures; that is, concepts come to exist (and
thus can be reified from the lecturer’s and the audience’s points of view) in and through
various material and public means, each contributing different aspects of/to the concept.
Moreover, lectures unfold in time, and so does the performance of concepts during lec-
tures. As Lev Vygotsky (We use the English spelling “Vygotsky” in the text itself but the
Romanized spelling of the name on the Russian book covers, Vygotskij, in citations and
the reference section.) suggests in the introductory quotation, a speaker might have some
idea in general but undeveloped form, which unfolds and articulates itself in speaking. In
non-scripted and non-memorized speech, therefore, speakers do not know their thoughts
in advance but find these in their words (Merleau-Ponty 1945). Thus, “thought is not
expressed in the word but concludes/accomplishes itself in it” (Vygotskij 2005, p. 1013).
That is, prior to speaking, a thought is incomplete, yet-to-be concretized and finalized,
which allows for the phenomenon Maurice Merleau-Ponty describes. There is therefore a
functional rather than an ontogenetic development: “the movement of the thought process
from thought to word and back is development” (p. 962). Thinking and speaking are two
mutually constitutive events, each following its own development, but united in word-sig-
nification. Word-signification “constitutes the irreducible unit of the two processes, about
which we cannot say whether it is a phenomenon of speech or a phenomenon of thinking”
(Vygotskij 2005, p. 954). That is, the words a lecturer says cannot be directly attributed to
thought—thinking and speaking are manifestation of a higher unit of sense (Schütz 1932).

13
A cultural-historical perspective on the multimodal development…

The two therefore may be thought of as sense-constitutive, contributing to sense as much


as being constituted by it.
Any sign is part of a number of sense-constitutive contextures, one of which is significa-
tion, where the sign is used to refer to something else, experience, phenomenon, or thing
(Schütz 1932). Signification is a process on both cultural-historical and individual-develop-
mental planes. Signification is rather stable, though not expressible like a thing but requir-
ing the unfolding sequence of words and statements. The sense of a statement or theme is
variable, depending not only on linguistic forms—words, syntax, and intonations—but also
the non-verbal aspects of the situation (Vološinov 1973). The theme therefore continuously
unfolds together with the ideas that are articulated in speech: it is subordinate to the total
experience (pereživanie) in and of the situation (Vygotskij 1984). The theme, subordinated
to experience, constitutes a whole and has a dynamic of its own, because of the dialogi-
cal relations that bind together words, non-verbal expressions, and the material and social
situation. Dialectical here means that two entities, which appear to be in opposition, really
are one-sided manifestations of an integrative, higher-order unit: speech and gesture/bodily
movement are but one-sided expressions of a higher order sense-constitutive unit. Thus, the
theme is an irreducible communicative unit subordinate to and reflecting the totality of the
experience of the situation. In a strong sense, “the theme is in fact indivisible” (Vološinov
1973, p. 101) and therefore cannot be reduced to smaller elements of which it would be
composed. The theme of a statement manifests the experience in and of the situation. It is
irreducible because it is the never-constant dimension of the statement. But the “significa-
tion of the utterance, in contrast, may be analyzed as a sequence of significations attached
to the linguistic elements from which it is made” (p. 101). Following the significations of
communicative performance, therefore, is precisely what is at the heart of our account of
lectures provided here.
Most importantly, each individual word specifically and statements (from spoken sen-
tences to poems and novels) more generally can be understood only as a characteristic of
speaker and audience. When we listen to a science lecture, then, the words are addressing
us, spoken for the purpose of helping us understand. This, in a sense, constrains lecturers,
who cannot just say what they want to say—if this were possible at all—but, in the saying,
address the need of the audience both in terms of the intelligibility of the individual phrase
and in the overall structure of the narrative, its genre. This means that the lecture is not
so much an expression of the speaker as it is an expression of anticipated needs of the lis-
tener in a given situation/context. From the perspective of the listener, therefore, a lecture
unfolds a topic, concept, or idea even though aspects of the lecture might exist in the dim
forms of undeveloped thoughts at the beginning of the speech. As new semiotic resources
are integrated into the unfolding communicative unit, the latter is stabilized or develops
and changes. Insofar as a particular scientific concept may be taught during several con-
secutive lessons, as is the case in the lessons that constitute our database, it is expected
that some of these resources in the communicative unit will be repeated not only within
a lesson but also across different lessons. However, when teachers lecture on a scientific
concept over the course of several lessons, they do not in fact repeat the same thing over
and over again—all repetition constitutes difference rather than the same (Deleuze 1968).
Rather, the scientific concept, concretely available in the teacher’s bodily performance,
develops within the unit constituted by the various consecutive lessons.
The transformation and development of a scientific concept becomes available for anal-
ysis through the identification of features of gestures and speech emsembles that allow us
to follow the development of the narrative in time; that is, it allows us to identify both
themes and significations as they unfold and change during lectures. In cultural-historical

13
L. Pozzer, W.-M. Roth

approaches, societal activity is the smallest unit that includes every, specifically human,
characteristic (A. N. Leont’ev 1978). An integral and constitutive part of activity is speech
activity (A. A. Leont’ev 1969/1971). Consistent with this conceptualization, the notion of
language action has been proposed as the smallest unit of analysis that includes (at least)
speech and gestures (McNeill 1985). Through the analysis of this gesture | speech ensem-
ble, he contends, we can access “a global and undivided picture of the conceptual con-
tent, while concurrently the content is segmented into words and arranged across time in
the speech channel” (p. 262). The analytical concepts of catchments and growth points are
those gesture | speech features that afford the possibility of tracking the performance of
meanings within specific themes (that is, signification of utterances) during lectures that
are already subsumed to higher order social, cultural and historical structures. We elaborate
on these two features of nonverbal communication (catchments and growth points) in the
sections below. By identifying the catchments and growth points in the narrative as the
teacher attempts to communicate scientific concepts to his audience (students), we are able
to follow the development of concepts in situ and through time (thus, historically, albeit in
a microscopic dimension), as these are articulated and made available through a combina-
tion of multimodal semiotics resources. In our approach, we emphasize temporal aspects of
communication in science: it develops on a moment-to-moment, individual developmen-
tal (ontogenetic), and cultural-historical (phylogenetic) scale (Roth 2014). None of these
scales can be understood independently but mutually presuppose each other because of
their dialectical relation.
The purpose of this study is to articulate a cultural-historical approach to the devel-
opment (elaboration, explication) of concepts during science lectures from a multimodal
communicative point of view. In this, we simultaneously provide a framework for future
studies in this area that elaborates on the dialectic of verbal and non-verbal aspects of com-
munication from a sociocultural, micro-analytical perspective. Understanding the concrete
details of communication during lectures is important because this is the very material stu-
dents use to evolve their own understanding, insofar as all that students have are the con-
crete details that a teacher (bodily, verbally) articulates in the lectures.

Communication as a situation‑encompassing unit

In this paper, we focus on multimodalities in consecutive science lectures. We seek to


understand how each and every one of the modalities and resources made available dur-
ing multimodal science lectures are integrated into a unit, here operationalized as a theme
(Vološinov 1973), and how they contribute to teach a concept. Within the various resources
available during lectures, the most prominent are speech and gestures. In this study, there-
fore, we follow the line of work begun by the Russian social psychologist Lev Vygotsky
and taken up by David McNeill: to study and theorize communication in terms of an over-
arching unit that manifests itself in different expressions and forms ([deictic, iconic] ges-
tures, prosody, body position, body movement, body orientation, and words) (Roth and
Pozzer-Ardenghi 2006). In the dialectical approach proposed, speech, gesture, and other
things that the teacher made available to students are different ways in which an idea
expresses itself. But the two or more expressive forms cannot be reduced to each other
because they are of different kinds. The overarching unit, in Hegelian language, “sublates”
(i.e., integrates and overcomes) the different expressions (speech and gesture); however,
in doing so, it also harbours an inner contradiction: gesture, speech, and other expressive

13
A cultural-historical perspective on the multimodal development…

forms are different yet are (one-sided, partial) manifestations of the same higher order idea.
Inner contradictions are expressions of movement and development in systems of activity
that are never self-identical (Roth 2014). Elsewhere we argue that speech and gesture con-
stitute different signifiers for the same signified as part of the same sign (Pozzer-Ardenghi
and Roth 2008). The inner contradiction is a manifestation of communication that propels
forward until the speaker has a sense of having achieved some stopping order that derives
from a sense of semantic and grammatical completion (McNeill 2002). In the following
subsections, we articulate the ideas of a dialectical communicative unit, its moment of
emergence (i.e., growth point), and its repetitions (catchment) and stabilization, as means
to understand the teaching of scientific concepts during lectures as a communicative event
unfolding in time.
According to McNeill, the concepts of growth point and catchment have also cognitive
implications beside their immediately communicative aspects. However, we purposefully
chose not to address the cognitive dimensions of these concepts in this article, as we are
not interested particularly in the cognitive aspects of the teacher’s discourse, but rather, on
how the concepts are made available to the audience in a concrete manner, and how, com-
municatively, the various ideas are associated to one another within and across the lessons.
That is, we are interested in how concepts are accessible as real, objective things in the
lecture performances that others can point to, talk about, and render accounts of.

Meaning‑making resources in multimodal analysis: the communicative meaning


unit

In multimodal discourse, as in the case presented here of a teacher lecturing about the
human circulatory system, different modes are used not only simultaneously but also, and
most importantly, as mutually constitutive parts of the same whole, which stand in dialecti-
cal relationship and presuppose each other. Thus, even when we refer to the gestures the
teacher performs, it is presupposed that, in analyzing these gestures, we do so from within
the meaning unit that also contains the speech, here constituted of words and how they are
articulated (that is, prosodic aspects of speech) and any other resource the teacher may be
using at the moment. In the same way in which speech can be analyzed through its lexical
and grammatical constituents, nonverbal elements of the discourse, such as gestures, can be
decomposed into their parts for analytical purposes, and yet, their role as meaning-making
resources in the particular sociocultural and historical context in which they were origi-
nally deployed can only be grasped through the analysis of the entire multimodal meaning
unit. Gunther Kress and Theo Van Leeuwen (1996/2006) refer to this as the grammar of
multimodal texts, in which the relations among various modalities make meaning as much
as and beyond any meaning each modality may potentially make in isolation. Moreover,
these authors also argue that meanings are culturally and historically specific, so that even
a microanalysis of meaning-making resources used during a multimodal lecture will be,
at the very least, socio-historically and culturally situated. In our case, the sociohistorical
activity is the high school science lecture, with its goals, rules, division of labor and power
relations that constrain and afford particular ways in which communication takes place.
That is, not only the genre lecture is socioculturally and historically identified as an accept-
able way of teaching in high school, but also the subject-specific rules of written and/or
spoken texts come to bear in the analysis of a science lecture (Unsworth 2001).
The dialectical communicative unit that we use as the smallest unit of analysis here
includes more than speech and gestures; it also includes other semiotic resources that are

13
L. Pozzer, W.-M. Roth

made available in the setting. The theme, therefore, is constituted by all the verbal and non-
verbal resources available in a setting: they are dialectically related, that is, both express the
idea without being reducible to each other. Both Vygotsky (2005) and Valentin Vološinov
(1973) quote the same paragraph from A Writer’s Diary of the novelist Fyodor Dostoevsky
in which six drunken workers each pronounces the same (curse) word. Although this word
has a dictionary signification, the sense is different each time it is articulated, the difference
deriving from intonation. That is, to understand what each drunkard has said, we must not
focus on the word but on the overall communicative unit. The entire conversation, part of
the totality of experience of the subject-in-setting, is led by means of intonations. “These
value judgments, as well as the corresponding intonations were wholly determined by the
immediate social situation of the talk and therefore did not require any referential support”
(Vološinov 1973, p. 104). Intonation does not reflect intellectual appreciation but is an
expression of affect (emotion), which constitutes a very different way by means of which
experience is reflected in the person: It complements and interpenetrates with the practical
and intellectual aspects of the situated experience (Vygotskij 1984). This is why others in
the cultural-historical tradition focus on speech activity, which is entirely subordinated to
and constitutive of productive activity (A. A. Leont’ev 1969/1971). This productive activ-
ity is a unit that encompasses everything that we might find in a given field of interaction,
tools (e.g., chalkboard), (material and ideal) objects, rules of engagement, and division of
labor.
The fact that we understand a lecture performance as a whole may lead to potential
problems: if the ideas presented in different modalities are different, apparent logical con-
tradictions may make it difficult for the audience to understand. The “decalage” of words
and gestures present a logical contradiction that makes it difficult even for graduate stu-
dents and professors to understand the talk (Roth and Bowen 1999). Vygotsky’s analysis
of the independent development of thought and speech actually provides an answer to
such contradictions, which arise from the fact that the two processes have their independ-
ent development. Gestures, being imagistic (visual) and holistic (three-dimensional), relate
differently to both thinking and the setting than the linearly unfolding verbal speech. As a
visual mode, gestures realize meanings differently from other modes, such as words (Kress
and Van Leuween 1996/2006).
When we attempt to understand how scientific concepts are communicated during lec-
tures, we take into consideration the theme, the communicative unit as a whole, outside of
which we do not really know what has been said and whether the saying is in jest, a joke, a
serious command, and so on. Consistent with the cultural-historical literature, we suppose
that neither speech nor gesture accurately and entirely represents this communicative unit,
but only partially and obliquely refers us to it, or serves entirely to support the expres-
sion of something else (e.g., intonation) while independent of the signification. Each single
resource is but a one-sided manifestation of the whole idea; they are identical in the sense
that through them the theme as signification comes into existence; they inherently are non-
identical because they constitute radically different forms. In this sense, all the semiotic
resources that contribute to signification constitute a unity/identity of non-identical things.

Growth points and catchments: repetition and difference

In this study, we use the term “gesture” to refer to spontaneous hand/arm movements that
occur during conversation, which have also been called “representational gestures” and
“gesticulations.” This type of gesture is non-conventional, that is, it is associated with

13
A cultural-historical perspective on the multimodal development…

Fig. 1  Still frames representing three parts of a gesture. a The preparation phase. b The stroke of the ges-
ture. c The retraction phase of the gesture, back to rest position

signification that derives from the current conversation, based on the context in which the
gesture has been used (Roth 2001). Thus, these gestures are idiosyncratic and their signi-
fication is global (i.e., any part of the gesture makes sense only when the entire gesture
is taken into consideration). Even though gesture and speech pertain to different modali-
ties (i.e., imagery and words), gesture-speech pairs cannot be separated. They are produced
together, synchronously and spontaneously, during conversations, which themselves are
subordinated to the ongoing activity (A. A. Leont’ev 1969/1971).
When analyzing the form of gestures, we can distinguish movements that constitute
different parts of the gesture. For example, consider the gesture represented in Fig.  1.
Before any gesture is performed, there is a preparation phase, which corresponds to the
moment the speaker abandons the previous position (i.e., rest position) and moves the
hands/arms in preparation for the gesture about to be performed. In Fig. 1, the gesture
starts with the palms of the hands facing down, and with both hands open and placed
side-by-side at the level of the teacher’s chest. To get to this initial position, the hands
are lifted and placed in this particular manner. These movements that antecede the

13
L. Pozzer, W.-M. Roth

gesture constitute the preparation phase of the gesture and are represented in the still
frames in Fig. 1a. The teacher shifts body orientation, turning to face the students, while
lifting the hands and arms from the rest position (first still frame in Fig. 1a) to the level
of the chest (last frame in Fig. 1a), when ready to perform the gesture.
Figure  1b represents the stroke of the gesture: The teacher moves the hands apart
from each other, sideways. This phase of the gesture usually coincides with the utter-
ance of the word or words that realize the meaning of the gesture | speech ensemble. In
Fig. 1, for instance, the teacher performs the stroke of the gesture at the same time that
he utters, “branching,” when talking about blood vessels in the circulatory system. Ana-
lyzing the movements that constitute the stroke of gestures and the word(s) associated
with this gesture phase allows us to more precisely articulate the meaning of a gesture
| word ensemble. Finally, after the stroke of the gesture has been performed, the hands
are brought back to the rest position, with movements that signaled the end of the ges-
ture, that is, the retraction phase (this latter phase is represented in Fig. 1c).
Through the analysis of gestures | speech ensembles, researchers are also able to
identify the instance when a new idea is introduced in the discourse. In these instances,
we find new holistic units, growth points; and these mark the “specific starting point for
a unitary thought” (McNeill 2005, p. 106). They constitute a minimal unit or idea that
integrates imagery (gesture) and words (speech), but which includes, in the approach
proposed here, other semiotic resources such as visual representations, salient aspects of
the setting, prosody, and orientation, all as part of the same theme. Whatever is commu-
nicated emerges at a certain point in the conversation, and although it relates to the pre-
vious context, it does not exist previously to the very moment when it is articulated. For
Vygotsky, this growth point constitutes the undeveloped general, which concretely real-
izes and develops itself in different ways (i.e., particularizes itself) with each repetition.
Alternatively, during a conversation there are many other instances when partici-
pants try to articulate and stabilize an idea; usually, in the search for stabilization, an
idea is recurrently addressed. These instances of recurrence are identified through the
repetition of certain gesture features, which are called catchments (McNeill 2002). The
existence of catchments points to a common idea. By analyzing catchments, we can
also identify ideas that the speaker considers to be related to each other, thus revealing
larger discourse units. This is particularly useful to understand how scientific concepts
are articulated in a developing and unfolding fashion within and across lecture situa-
tions, when new aspects of a certain concept are introduced while others are repeated
and reinforced. In this sense, the repetitions are re-iterations of meanings, which, once
stabilized, can then be contrasted with the new significations emerging in the narra-
tive and that moves the narrative along (Vološinov 1973). Moreover, the identification
of catchments and growth points seems ideal for the investigation of how concepts are
taught during lectures, given that lectures are inherently sequential at the content level,
and in these situations, new, more complex scientific concepts are built on top of other,
previously presented concepts, in a process that unfolds over more than one single les-
son, often involving various lessons (consecutive or not). The identification of recurrent
ideas in these situations become important for understanding how new ideas are con-
nected to others already presented, and also to establish how different scientific concepts
are presented and related to each other. In this, we follow the line of work developed
by Gunther Kress, Carey Jewitt, Jon Ogborn, and Charalampos Tsatsarelis (2001) and
Wolf-Michael Roth (2000) in science education, but expanding it to focus on a sequence
of lessons dealing with the the same scientific conceptual unit (circulatory system) and
using micro-analysis of multimodalities used (e.g., gesture, prosody).

13
A cultural-historical perspective on the multimodal development…

Research context

Our data collection took place in a Canadian grade 12 Biology course. Over a period of
3 months, we videotaped every lesson in this course. The main topic of these lessons was
human anatomy and physiology. Twenty-six lessons were recorded, 70  min long each,
totaling over 30 hours of recorded material. We used two digital cameras to videotape the
teacher, moving around the room, following him as he walked around the perimeter of
the classroom. This allowed us to record the teacher’s body movements, gestures, facial
expressions, position in the room, and also all the visual and material resources he used,
such as, for example, the chalkboard and what was written and drawn on it, the projector
screen, various three-dimensional models and props, the TV, and maps and diagrams hang-
ing over the blackboard.
The teacher was very experienced and had taught biology at this school for many years.
In this class, he taught to an audience of fifteen students, seven males and eight females.
The teacher taught the subject primarily by means of lectures, involving a variety of visual
resources beyond the use of the blackboard, including videos, three-dimensional models,
overhead projections, demonstrations, and students’ presentations.
In this study we are concerned with the communicative development of ideas and
concepts in lectures; we are particularly interested in how a teacher communicates (i.e.,
teaches) a concept throughout several consecutive lessons, and how different ideas are
associated to other ideas, previously presented, through the multimodal resources made
available to the audience while performing the concepts. To follow the development of
the concepts during the same and across different lessons, we selected an entire curricular
unit, the circulatory system, representative of the other units taught during the time of data
gathering, which was structured as a sequence of lessons that addressed curricular top-
ics—identifiable as terms, processes and its mutual relationships—that were taught primar-
ily during multimodal lectures. We proceeded to analyze all lessons within the curricular
unit identified (by the teacher) as pertaining to the circulatory system. During seven con-
secutive lessons (over 9  h of recorded material), the teacher introduces students to con-
cepts related to the circulatory system. By analyzing each one of these lessons, we came
to understand how the concepts develop within and across the lessons, from a microper-
spective that focuses on growth points and catchments as means to identify the temporally
sequencing of meanings through discourse. This microperspective, focusing primarily on
multimodal communication, does not lose sight of the macro (social, cultural, historical)
aspects of communication; indeed, we contextualize our analysis of the communicative
development of concepts over these seven lectures within the social and cultural approach
that considers the resources used in communication (speech, gesture, diagrams, etc.) as
tools that are already implicated in specific social purposes (in this case, teaching). As an
illustrative case upon which to elaborate this understanding, we choose here to explore the
significations of a particular gesture | word ensemble and its catchments that happened dur-
ing the first three lessons on the circulatory system.

13
L. Pozzer, W.-M. Roth

Repetition and difference: development of ideas and concepts


in and as unfolding narrative

In lectures, a concept—though it may exist as an undeveloped, indefinite, and yet-to-be


realized whole in the mind of the speaker (Vygotskij 2005)—is not and cannot be articu-
lated for students in one single instant. Rather, lecturers develop a concept in and through
the material production of talk until some point when they have a sense to have completely
articulated the concept. That is, the communication of scientific concepts occurs through
the temporal deployment of a variety of different semiotic resources—indeed bridging dif-
ferent media (Friesen 2011)—that together constitute a communicative unit: the theme.
Thus, a concept is not taught as a single thing but each concept is the outcome of an event,
the cumulative effect of the production of coordinated material signs. At any given time,
the lecturer’s speech (i.e., the words uttered), his body orientations, the gestures performed,
and drawings on the chalkboard might all be part of the communication, contributing dif-
ferent aspects of a scientific concept that is unfolding in front of the students’ eyes and ears.
Each new aspect of a concept or each new related idea that is brought into the speech at
some point in time becomes salient against everything else already present in the setting.
The unitary category of experience captures the practical dimensions of the situation (set-
ting, physical performance) together with the intellectual and affective dimensions (Vygot-
skij 2001). That which appears as new emerges at an instant and is called a growth point
(McNeill 2005); it becomes salient and thereby allows us to follow the process by which
scientific concepts emerge, to be developed and thereby communicated in the classroom.
In the following subsections, we describe this process first through a microanalysis of
the unfolding and developing theme in the opening lesson dealing with the circulatory
system, when the teacher first introduces concepts related to blood circulation. We articu-
late in our analysis an initial occurrence of growth points, when new ideas are introduced
into the communicative unit. These ideas will later reoccur as the teaching of the concepts
progresses in time and complexity, until they are stabilized. That is, the recurrences mark
something like a search for stabilization of the concepts being taught and coincide with the
catchments the teacher enacts. Therefore, in this first section we focus on the occurrence of
catchments and growth points within the same lesson, following the sequential introduction
of new aspects of the concept of blood circulation. In the second section, we focus on the
occurrence of catchments to one particular gesture across three consecutive lessons at the
beginning of the seven lessons concerning the circulatory system, further explicating the
process through which concepts are performed in science lessons.

“The receiving chambers of the heart”: introducing the concept of circulation


of blood

At some point in a lecture, teachers introduce a new concept. But because it takes some
time for an idea to be developed, the concept is not all available when the teacher begins
to introduce it. Yet when we begin the analysis once an idea or concept is fully developed,
we can then trace it backward until some starting point (i.e., the growth point). In this
and the next subsections, we articulate how new ideas and concepts come to be introduced
and developed, borrowing on the notions of growth points and catchments (McNeill 2005).
During the opening lesson dealing with circulatory system, the teacher presents for the first

13
A cultural-historical perspective on the multimodal development…

Fig. 2  The teacher’s chalkboard


diagram of the heart

Fig. 3  Gesture performed simultaneously with the underlined words in Episode 1

time in this class concepts related to blood circulation and anatomy and physiology of the
heart, such as the four chambers of the heart, and the flux of blood between these chambers
and other parts of the body. The four chambers of the heart are named “right ventricle”
and “left ventricle” for the top right and top left chambers, respectively, and “right atrium”
and “left atrium” for the bottom right and bottom left chambers, respectively. The teacher
starts by drawing a square diagram (Fig. 2) on the chalkboard, which he then identifies as
a diagram of the heart. He then writes “R” and “L” on each side of this diagram, identify-
ing the right and left sides of it, respectively. There is no similarity in shape between this
square diagram and a human heart; nonetheless, the iconicity is established explicitly when
the teacher associates this diagram to his speech in Episode 1 below. The square diagram
has four chambers and so does the human heart. What these chambers look like in a human
heart is not salient at this moment; rather, the existence of four chambers is emphasized in
this diagram. Thus, at this juncture already, meaning is created in the dialectical relation
between diagram and words uttered.
When Episode 1 begins, the teacher identifies for the first time (in this class) the cham-
bers of the heart. Synchronously with line 02 in episode 1, the teacher performs a gesture
(Fig. 3).
Episode 1 (Lesson #1—0:47:56 (Time (h:min:sec) represents how far along from the
beginning of each lesson the particular episode occurred.))

01 The heart has four chambers (1.25 s) the top


02 ones are the receiving chambers (0.60 s)

This particular gesture, as other gestures more generally, has a preparation phase, a
stroke, and a retraction phase. The preparation phase for this gesture begins when the
teacher brings his right hand away from the chalkboard (where he just finished draw-
ing the square diagram of the heart) and starts to take his left hand out of his pocket.
Throughout the preparation phase of this gesture, the teacher shifts his body orienta-
tion from the chalkboard to the students. By turning towards the students, the teacher
articulates what should be the focus of attention at that particular moment. It is the

13
L. Pozzer, W.-M. Roth

Fig. 4  Diagram of the heart with


the addition of two red arrows

articulated focus because communication generally and words and gestures specifically
accent the visible (El’konin 1994). That is, through his body orientation, the teacher
shifts the accented visible from the diagram of the heart (chalkboard) to himself, and
consequently, to the gestures, speech, and other actions he performs at that time.
Once the teacher directly faces the students, the stroke of the gesture begins. It coin-
cides with the uttering of the word “receiving,” and the first syllable of the word “cham-
bers.” The second part of the word “chambers” is pronounced while the teacher returns
to the rest position (the retraction phase of the gesture), already turning back to the
chalkboard. This particular gesture is performed towards the teacher’s own body (that
is, he moves his hands up at the height of his neck and out to the space in front of him,
and then bring them back closer to his torso), thereby assuming a referential position
that implies his embodiment in the situation. Even though this gesture co-occurs with
the utterance, “the top chambers of the heart,” these receiving chambers are articulated
in terms of their location in a human body, in this case, his own. The teacher’s hands,
moving towards his torso, embody the imagery of “bringing something into his heart.”
To use the teacher’s words, his heart, or more precisely, the top chambers of his heart
are “receiving” something. As soon as this gesture is finished, the teacher retracts his
hands, and turns towards the chalkboard again.
The shift in body orientation communicates the shift in attention: the diagram of the
heart once more comes into focus. In fact, to be properly understood, the teacher’s ges-
ture in Episode 1 needs to be contextualized within the theme that includes the diagram
as well, as much as the diagram needs to be coupled with the speech and gesture to repre-
sent a human heart and its four chambers. This theme includes the diagram on the board
(which represents the heart, including its top or receiving chambers), the gesture, the words
being uttered (“the top ones are the receiving chambers”), and the teacher’s own body, as
the speaker and gesturer’s body. But it also includes a referential position for the gesture
per se, and as a pointer through shifts of body orientation. Thus, the gesture is performed
towards the teacher’s body, which was brought to the foreground through the shifting ori-
entation from the chalkboard to the students. What becomes salient here is the idea of the
heart having four chambers and the function of the top two chambers (“receiving” blood);
indeed, the top chambers are not available in the gesture or in the teacher’s body, but in the
teacher’s words and in the diagram on the chalkboard, where they are represented as the
top two compartments in the diagram of the heart. Whereas the location of the heart cham-
bers is available through speech and diagram, their “receiving” quality, which is associated
to the process of blood circulation, is available simultaneously through the speech and the
gesture. Therefore, through the body orientation, a link comes to be established between
the diagram and the words with the gesture and the body itself; thereby, the entire theme
makes available information that goes beyond what each one of the resources that consti-
tute this unit could communicate when considered in isolation. The idea comes to be real-
ized differently in each of these modalities that cannot be reduced to any single, isolated
modality.

13
A cultural-historical perspective on the multimodal development…

In the sequence, the teacher draws two red arrows on the diagram of the heart (Fig. 4),
while simultaneously uttering the following (Episode 2):
Episode 2 (Lesson #1—0:48:04)

03 so blood is going to flow [(1.19 s) in (1.09 s)] and blood is going


to
[draws arrow on left side of diagram]
flow [(1.75 s) in]
04      [draws arrow on right side of diagram]

Here, the teacher continues his explanations, elaborating on what it is that the top cham-
bers are receiving—blood; he starts his sentence with “so,” which refers back to the previ-
ous utterance and connects it with the sentence that follows. Previously, the teacher had
identified the top two chambers of the heart as the receiving chambers; now, he identifies
them as receiving blood that is flowing in. The idea was not complete until this last bit
of information was provided. In fact, the use of the adverbial conjunction (“so”), which
constitutes a logical connection, literally implies that there is an implicative association
between the idea of receiving and what is being received: The top two chambers of the
heart (represented in the square diagram of the heart in the chalkboard) receive (an action
articulated in words and in the teacher’s gesture) blood (represented with two red arrows
drawn on top of the diagram). Notice also that the color red the teacher chooses to draw the
arrows is iconically related to the color of human blood. Shifting attention from the dia-
gram to the teacher and then back to the diagram would enable the audience to associate all
the visual and verbal information made available in concrete material form; and even this
shift of attention is guided by the teacher’s shifts of body orientation.

The sense of the Latin word “atrium”: introducing scientific terminology

The identification of growth points in the course of the lesson allows us to understand how
each new idea that ultimately constitutes a particular scientific concept is linked to previ-
ous and subsequent ideas, and is communicated during the lecture. In this section, as the
teacher continues lecturing, he turns to the audience again and he talks about the origin of
the Latin word “atrium.”
Episode 3 (Lesson #1—0:48:16)

05 (4.41 s) in uh Latin I think it is Latin there is the a name for the


06 front room in the house (0.51 s) the place where people enter and
arrive
07 and it is the (0.58 s) atrium (1.16 s) in churches we called it
foyer
08 (0.38 s) uh the front entrance in our school (0.99 s)

Similar to Episode 2, Episode 3 also introduces a new aspect of the idea of the top
chambers of the heart. This however is not immediately apparent in the words uttered in
Episode 3, which seem disconnected from the theme articulated so far in Episodes 1 and
2. The teacher no longer provides information related to the chambers of the heart, but
rather, he introduces the notion of atrium as a “place where people enter and arrive.” This

13
L. Pozzer, W.-M. Roth

Fig. 5  Gesture that occurred simultaneously with the underlined words in Episode 3

introduction of a new term is marked by a long pause, (4.41  s), and the occurrence of a
gesture (Fig. 5).
The gesture the teacher performs coincides with the uttering of the words “where people
enter and arrive” (line 06). The stroke of this gesture is synchronized with the utterance
“enter.” This gesture is similar in form to the gesture in Episode 1; however, when per-
forming this gesture, the teacher stands in a different referential position than when he was
performing the gesture in Episode 1; now he physically and metaphorically creates a space
that lies right in front of him– the space “where people enter and arrive,” without, however,
including his own body into this space, as he did in Episode 1. The synchronized gesture
and speech constitute a growth point, that is, the exact moment when the concept of atrium
is introduced in this lesson. Before this moment, the teacher talked about the chambers of
the heart and about how blood flows into the top two chambers. To the initiated, atrium
and the top chambers of the heart as the receiving chambers together form one and the
same idea; atrium is simply the scientific term used to refer to the top chamber of the heart.
However, this signification is not available just yet to the uninitiated; at this point, the simi-
larity between the ideas conveyed in Episodes 1 and 2 and what is being communicated in
Episode 3 resides only in the unspecified idea of something entering somewhere; blood
entering the top chambers of the heart, which are the receiving chambers (Episodes 2 and
1, respectively), and the etymology of the word atrium, meaning a physical space where
people “enter and arrive.”

“The atria are the receiving chambers”: connecting ideas and moving


the development of the concept onwards

To understand that different material productions (different words, drawings, gestures)


across different modalities (speech, gesture, body, chalkboard drawings) belong to and
constitute the same idea or concept, lecturers specifically and speakers generally have
to provide resources that allow the audience to connect the material and thus perceptual
resources. More so, if it takes different ideas to build a concept, these, too, need to be con-
nected. How do lecturers achieve connections or achieve producing resources for the audi-
ence to connect different material productions that have occurred across the continuous
flux of time?
The new idea the teacher just presented (Episode 3) has not yet been articulated in terms
of the other aspects of the concept of blood circulation, which the teacher has already pre-
sented in Episodes 1 and 2. As the teacher continues, however, the connection between the
new term (“atrium”) and the concept as articulated thus far is established. In the sequence,
still talking about the atrium, the teacher performs a catchment for the gesture in Episode
1, that is, these two gestures (in Episodes 1 and 4) present similar features, which enable us
to identify the gesture in episode 4 (Fig. 6) as being a repetition of the gesture in Episode 1
(Fig. 3).

13
A cultural-historical perspective on the multimodal development…

Fig. 6  Gestures performed simultaneously with the underlined words in Episode 4

Episode 4 (Lesson #1—0:48:32)

09 the word atrium [(0.39 s) is (2.85 s)]


[writes ‘atrium’ on the board]
10 just means a receiving area or entranceway

With this catchment and the associated words (lines 09–10), the teacher establishes the
connection between what was communicated in Episodes 1 and 2, that is, the concept of
blood flowing into the top chambers of the heart, and the idea of atrium introduced in Epi-
sode 3. Here, the teacher clarifies that the word atrium “just means a receiving area” (line
10). In Episode 1, the teacher had talked about the top chambers of the heart as being the
receiving chambers; thus, the recurrence occurs both in the word “receiving” and in the
catchment identified in his gestures. The novelty introduced in Episode 3 is the Latin word
“atrium.” Once the word has been introduced, however, it is then associated with the previ-
ous ideas by means of a repetition of word and gesture that denote that the teacher is still
elaborating and stabilizing the idea he has introduced in Episodes 1 and 2 that is related to
blood circulation.
In Episode 4, however, two repetitions occur. The teacher actually repeats the gesture
twice: first, synchronously with “receiving,” and then simultaneously with the utterance of
“area.” Being repetitions against the background of their earlier occurrence changes their
significance—they make themselves salient as repetition, thus being the same and differ-
ent simultaneously. Each catchment performed here points to the existence of a recurrent
idea being articulated; the first one makes salient the idea of “receiving,” whereas the sec-
ond is linked to the idea of a physical location (“area”), which was articulated in Episode
3. Thus, through catchments and repetition of words, Episode 4 links the ideas presented
in Episodes 1–3 as belonging to the same theme, which, nonetheless, has moved forward
and changed through this very articulation of meanings. This movement in signification is
equivalent to the concept being developed as it is performed.
Finally, in Episode 5, all the resources that are part of the same theme (i.e., gesture, the
word atrium, the idea of receiving chambers in the heart, and the diagram of the heart on
the chalkboard) are associated. The teacher turns towards the chalkboard and writes “RA”
and “LA” inside the top chambers of the diagram of the heart, while verbally identifying
them as the “right atrium” and the “left atrium” (line 11), respectively.
Episode 5 (Lesson #1—0:48:40)

11 and so this then becomes [the right (0.78 s) atrium and left
atrium]
[writes ‘RA’ and ‘LA’ on diagram]

Here, similarly to Episode 2, the teacher uses the particle “so” to connect the previ-
ous sentences to his following sentence. The introduction of the concept of atrium and its

13
L. Pozzer, W.-M. Roth

Fig. 7  a Gesture performed simultaneously with the utterance of “empty spaces” in line 12 in Episode 6. b
Diagram of the heart with capital letters designating each one of the atria and the ventricles

identification as a receiving area are associated with the diagram of the heart, and conse-
quently, with the concepts of blood circulation within the heart. Different aspects of the
concept of atria have been connected as this idea achieves stabilization: The top chambers
of the heart are the ones that receive blood, and they are called atria.

“The ventricles in the brain and the ventricles in the heart:” building an analogy

Another way in which new ideas come to be introduced to the unfolding development of
the concept is by means of analogies. However, analogies do not just exist. Students often
do not see two situations as analogous, which has science educators questioning whether
analogies constitute friends or foes and characterizing them as capricious (cf. Aubusson,
Harrison and Ritchie 2006). They have to be performed in such a way that the audience
can perceive two different aspects of the unfolding communication to refer to something
that the aspects have in common. By comparing two entities, both the similarities and the
differences between these two entities become evident, and an association between the two
is thus made possible. Moreover, “through this process, the socio-cultural and socio-histor-
ical meanings of the everyday were brought into the science classroom as another mean-
ing-making resource” (Kress et al. 2001, p. 66). As the teacher continues lecturing on the
circulation of blood within the heart, he introduces new information and moves the lecture
forward with the presentation of a new theme (Episode 6).
Episode 6 (Lesson #1—0:48:48)

12 (3.14 s) there = re [empty spaces] in the brain (0.29 s)


(0.39 s) called
[gesture in Fig. 7a]
13 ventricles (0.47 s) there’re empty spaces in the [heart (0.26 s)]
called
[touches diagram]
14 ventricles (0.61 s) and [this is the right ventricle (1.31 s)] and
the
[writes ‘RV’ and ‘LV’ on diagram]
15 left ventricle (2.21 s)

Following a relatively long pause (3.14 s), which frequently signals that the lecture turns
to a new topic or a new activity, the teacher starts talking about the ventricles. He compares
the “empty spaces in the brain” with the “empty spaces in the heart” and associates both
by using their scientific name, ventricles. While uttering the first part (line 12), the teacher
performs the gesture in Fig. 7a, synchronized with the words “empty spaces,” thus calling

13
A cultural-historical perspective on the multimodal development…

Fig. 8  Gesture performed simultaneously with the underlined words in Episode 7

attention to this particular characteristic of the entities named ventricles. That is, the anal-
ogy between the ventricles in the brain and the ventricles in the heart works to emphasize
the communalities of both entities: they are empty spaces, and they are called ventricles.
The teacher has already taught the unit dealing with the nervous system in this class.
The term “ventricle” has already been introduced to students, but as a structure in the
brain and not in the heart. Comparing both structures of the different organs (i.e., brain and
heart) makes both similarities and differences between the brain ventricles and the heart
ventricles salient: the most obvious difference between these is that they belong to different
organs; the similarity then is attributed to the fact that they are both “empty spaces,” and
they are both called “ventricles,” which the teacher emphasizes through the repetition of
these terms (lines 12–15).
In the beginning of Episode 6, the teacher is turned towards the students. Simultane-
ously with the utterance of “heart” (line 13), however, the teacher turns towards the chalk-
board and approaches the diagram of the heart, touching the bottom-left chamber in the
diagram with his left middle finger and the bottom-right chamber with his left thumb. The
teacher’s body orientation functions once again as an index—the focus of attention has to
be shifted to the diagram and back to the concepts related to the anatomy of the heart. The
teacher then turns again to the audience and utters “ventricles” (line 13). This functions as
emphasis on the word “ventricle,” which is presented now as a new term associated with
this concept. He turns back to the chalkboard once more to write “RV” and “LV” on the
two lower parts of the diagram of the heart (Fig.  7b), thus identifying them as the right
ventricle and the left ventricle respectively.
Here again, changing orientation signals a shift in the focus of attention and connects
speech, gestures and diagram into the same unfolding theme. The heart ventricles, which
are introduced through a comparison with brain structures also called ventricles, are related
to the bottom chambers in the diagram of the heart, first by means of a pointing gesture,
then through an abbreviation written directly on the diagram itself.
In Episode 7, the teacher continues to talk about the ventricles, introducing their func-
tion by means of another analogy, which had been previously introduced in this class. At
the same time, the teacher also connects the concepts of atria and ventricles, by contrasting
their functions.
Episode 7 (Lesson #1—0:49:03)

16 the vent- if the atria are (0.48 s) receiving areas the ventricles
17 are the big pumps (0.65 s) these are the pumping chambers (2.0 s)

In line 16, when the teacher says, “receiving,” he performs another catchment (Fig. 8)
for the gestures produced in Episode 1, which once more makes salient the (recurrent) idea
of the atria being the receiving chambers of the heart. This time, however, the notion is
contrasted with the function of the ventricles. This introduces a new idea (the ventricles
as the “big pumps,” line 17) that coincides with (and thereby is identifiable through) the

13
L. Pozzer, W.-M. Roth

occurrence of a growth point. Here, the teacher performs a gesture the main feature of
which is the short hand movement that is performed repeatedly (the squeezing movement).
The imagery is powerful, insofar as it resembles the contracting movements of muscles,
especially cardiac muscle. Thus, when the teacher utters “big pumps” simultaneously with
the gesture in Episode 7 (Fig.  8), he is making salient in the theme an important aspect
associated with the contraction of cardiac muscle, namely the fact that these contractions
control the flowing of the blood in our circulatory system. In the context of the concept
of blood circulation, these repetitions are iconic to the regular contractions of the cardiac
muscle (“squeezing”), which controls the flowing of the blood.
The stroke phase of the gesture in Episode 7 and the associated word “pump” constitute
a growth point, which propels the development of the concept of blood circulation forward,
introducing a new aspect of this concept. Previously, the atria were defined as areas that
receive blood that was flowing into them; how the blood get there was not articulated at
that stage. Now this aspect is made available with the introduction of the concepts of ven-
tricle and contraction. The word contraction, however, has not yet been used. Rather, it is
the imagery (the gesture) and the analogy (“big pumps”) that bring about the idea of con-
traction. The analogy of “pumps” is in fact recurrent, since the teacher has introduced the
unit concerning circulatory system with an analogy between the fluids circuit in a car and
the blood circulation in a human being, where the water pump was analogous to the heart.
Later in the same lesson, as in subsequent lessons, the teacher did use the word contraction
or variations of it (e.g., contracting, contracts) while at the same time repeatedly perform-
ing the gesture in Episode 7, that is, producing various catchments. In the next section, we
focus on the occurrence of these catchments, from the first time the gesture appeared in
Episode 7 to the end of the same lesson and throughout the following two lessons in the
unit concerning the circulatory system. We focus on how new aspects of the concepts of
contraction and circulation of blood are introduced to the class and associated with the
ideas represented in these catchments, paying close attention to the integration of various
resources within the unfolding theme that are used in the production of signification.

The development of a scientific concept across consecutive lessons

Scientific concepts and ideas are complex, unfamiliar for most students, and abstract in
most cases. Therefore, teaching (lecturing) them may take not only more than an instant, as
we showed in the previous section, but also more than a single lesson (lecture). They come
to life in the drama of the performance. This raises several questions, including “How are
concepts and ideas communicatively developed across consecutive lessons?” and “How
is the coherence between material sign productions belonging to the same concepts and
ideas achieved when they occur in different lessons?” In fact, the Vygotskian idea of the
drama allows us to think of a concept or idea as a play performed in several acts, here
constituting a drama that is playing out across days. As a play, the performance is a cul-
tural phenomenon through and through and cannot be reduced to the individual subjectiv-
ity of the teacher. The temporal spread comes with its own requirements, for there is a
demand on ensuring narrative coherence to be experienced on the part of the listener. Thus,
the lecture not only presents new ideas but also provides the resources for understanding
and retaining the narrative thread. In this section, we present answers to these questions
by discussing the occurrence of various catchments for the gesture that has been associ-
ated with the idea of contraction (Episode 7) as these occurred within and across lessons.

13
A cultural-historical perspective on the multimodal development…

Fig. 9  Graphic representations of pitch (dotted line) and intonation (continuos line) of utterances and still
frames of the synchronous gesture in Episode 7. a Preparation phase. b Stroke. The teacher performs the
same movement twice within the same gesture. c Retraction phase

Repetitions (i.e., catchments) of this gesture occurred several times in this same lesson and
also in the subsequent lessons while the teacher talked about circulation of blood. We pre-
sent here a microanalysis of gestures, speech, pitch, intonation, and material resources used
by the teacher, which enables us to simultaneously focus on the integration of multimodal
resources in the theme and on the progressive development of the concept of circulation of
blood, until it reaches its stabilization.

Episode 7 (Lesson #1—0:49:09)


“The ventricles are the big pumps.”

13
L. Pozzer, W.-M. Roth

Fig. 10  Graphic representation of the variation in pitch (dotted line) and intonation (continuous line) of the
utterance that is synchronous with the stroke of the gesture (represented in the still frames) that occurs in
Episode 8. The teacher repeats the same movement twice within the same gesture

The gesture performed in Episode 7 first appeared concurrently with the uttering of the
words “the ventricles are the big pumps,” which, as we have seen earlier, refers to an anal-
ogy the teacher had previously introduced in class. Through microanalysis, we synchro-
nized gestures and words, pitch and intonation, which are represented in the graph beside
the photographs above. We notice that the short, repeated hand movements that constitute
the gesture in Fig. 9 are performed simultaneously with the words “are the big pum[ps].”
The subject of this sentence (the ventricles) is uttered during the preparation phase of
this gesture (Fig. 9a), when the teacher moves from the chalkboard, where he had written
the word “ventricles” below the word “atrium.” In this latter orientation, the teacher then
performs the stroke of the gesture (Fig. 9b), with higher pitch corresponding to the utter-
ance of “[th]e” and “pum[ps]” and higher intonation at “are” and “pum[ps]” again. The
body orientation allows for the alignment of the gesture space (i.e., the area in front of the
teacher) with the line of vision of the students. That is, by turning around and facing the
students, the teacher positions himself so as to maximize the chances of the audience see-
ing his gesture. This also functions to shift the focus of attention from the chalkboard to
the teacher’s gesture. The emphasis placed on “are,” “the,” and “pumps” directly connects
the gesture with these words, which in turn denote the action (verb “are”), and the fact that
the ventricles are the parts of the heart that function as pumps (as opposed to, for example,
the ventricles being a pump). Thus, the signification in this instance is constituted by the
words (the ventricles are the big pumps), the gesture, the emphasis placed on certain words
(are, the, pumps), the written word “ventricle” in the chalkboard, and the body orientation
(towards the board, towards the students, and back to the board again during the retraction
phase of the gesture [Fig. 9c]).
Analyzing this signification, we notice that the gesture is representative of the con-
tractions of the ventricles, sending blood outside of the heart (either to the lungs or to
the rest of the body); that is, this gesture is associated with the function of the ventricles,
the particular contracting movement. Even though it has been used simultaneously with
an analogy that refers to the ventricles as an entity, there is compelling evidence that this
gesture is indeed associated with the contraction movement per se, not only because of
the iconic representation of the movement (“squeezing” movement with quick, consecu-
tive repetitions), but also because the stroke of the gesture occurs synchronously with
“are the big pum[ps]”and the emphasis is placed on “are the pump[ps].” Another clue as
to the reference of this gesture is given by the fact that a similar gesture is performed in
association with a different entity (i.e., atria instead of ventricles) in Episode 8 (Fig. 10).
The gesture in Episode 8 presents the same main features of the gesture performed in

13
A cultural-historical perspective on the multimodal development…

Episode 7; that is, it is recognized as a catchment. This time, however, the teacher is not
talking about the ventricles, but rather, about the atria.

Episode 8 (Lesson #1—0:49:15)


“The atria have a fairly easy job to do they contract and pump the blood”

The stroke of this gesture occurs simultaneously with the words “they contract.” The
hands movement is very similar to the one in Episode 7. Here, however, not only is the
subject of the sentence “the atria” (a different entity than the ventricles) but also the
verb associated with the gesture and which is emphasized through both pitch and into-
nation is “contract.” This corroborates the notion that this particular gesture, which was
first performed in Episode 7 and is repeated here (and also various other times through
this and the next lessons), makes reference to the contraction movement of the heart.
In Episode 7, the emphasis was placed on the analogy of the ventricles as pumps; here,
although the word pump is uttered, it is related to the contraction movement of the cardiac
muscle and how blood flows into other chambers of the heart. In both situations, however,
the gesture presents the same referent. There is more evidence corroborating this interpre-
tation of the reference of this gesture as we proceed to analyze the occurrence of the catch-
ments throughout the other lessons dealing with the circulatory system. During this lesson
and throughout the following ones, the same gesture reappears, always in association with
the contracting movement, although with slight variations in form and content, which are
directly related to the development of the concept of circulation of blood.

From contraction to systole, pulse, and pressure: the association


of recurrent ideas across several lessons and the stabilization
of signification of the concept

During the remaining six lessons that constitute the unit concerning the circulatory system,
the teacher continues to talk about the contraction of the cardiac muscle, and the circula-
tion of blood. These concepts still need stabilization, as they are further developed with
the introduction of new terms and ideas that need to be associated with the previous ideas
developed in class. The association of words and prosodic resources with other visual
resources, such as gestures and drawings, provides coherence and the sequential charac-
teristic of a multitude of different concepts that are intimately associated to each other in
the broader curricular unit concerning the circulatory system. That is, whereas one aspect
of the communicative unit realizes meanings recurrently, other aspects introduce new and
more complex meanings, which are associated with the previous ideas through the connec-
tion of all the resources that constitute this theme.
Throughout the next lessons dealing with the circulatory system, the teacher performs
several catchments for the gesture in Episode 7. Although the main features of these ges-
tures are very similar, they present variations that accompany the development in complex-
ity of the concepts. Within the unfolding theme, the integration of gestures, words, and
other resources (prosodic aspects of speech, drawings on the board) makes possible the
association of different and new information with previous information within the same
conceptual unit. For instance, in Episode 9, the teacher repeats certain information while
also adding new information to the theme.

13
L. Pozzer, W.-M. Roth

Fig. 11  Graphic representation of the variations in pitch (dotted line) and intonation (continuous line) of the
utterance that is synchronous with the stroke of the gesture (represented in the still frames) that occurs in
Episode 9

Fig. 12  a Square diagram of the heart as it last appeared in the first lesson dealing with circulatory system
(compare with the one in Fig. 4). b Diagram of the heart in the second lesson dealing with circulatory sys-
tem (left image: the diagram at the beginning of the lesson; right image: the same diagram later on during
the lesson)

Episode 9 (Lesson #2—0:33:36)


“The heart muscle contract”

The gesture in Episode 9 (Fig.  11) is also associated with the word “contracts,” as in
Episode 8. Here, emphasis is placed on both the words “muscle” and “contract,” which
also coincides with the stroke of this gesture. In this situation, both the gesture and words
represent repetitions of the previously enacted gesture | word combinations. However, the
diagram of the heart in the chalkboard has become significantly more complex (Fig. 12).
The teacher now further develops the idea of circulation per se, an idea that had been
briefly introduced in the previous lesson by means of a very schematic diagram of the
blood circuit in the human body (Fig. 4). Previously, the teacher presented this concept by
means of a square diagram of the heart, in which all the chambers have the same size, and
in which vessels (or “tubes”) were represented as parallel lines in the centre of the square.
Dashed arrows leaving from and coming back to the square represented blood circulation
to and from the heart. The square diagram of the heart (Fig. 12a) can be classified as ana-
lytical (i.e., parts that form a whole), inclusive (i.e., only some parts are shown) and with
very low modality (i.e., it lacks naturalistic details that could rend it more realistic) (Kress
and Van Leeuwen 1996/2006). In contrast, the diagram in Fig. 12c is still analytical but it

13
A cultural-historical perspective on the multimodal development…

Fig. 13  Graphic representation of variations in pitch (dotted line) and intonation (continuous line) of the
utterance that is synchronous with the stroke of the gesture (represented in the still frames) that occurs in
Episode 10

attempts to be more naturalistic and exhaustive, in that it provides a shape for the heart that
more closely resembles that of an actual human heart and showcases several structures that
can be seen in the heart (vessels, valves, sutures). Figure 12a did represent vessels in some
way (straight lines connecting the upper and lower chambers of the heart), but those lacked
any realism and, thus, discerning between the different vessels was not foregrounded as
important at that moment. In Fig.  12c, however, the vessels resemble actual vessels and
they can be clearly distinguished (aorta, pulmonary vein, etc.), and they are also named,
making this attribute of the analytical process much more salient. When the teacher draws
and talks | gestures on and about the diagram in Fig. 12c, the significance of being able to
identify the different vessels is foregrounded, that is, it becomes salient information. This
is so because the teacher is now focusing on processes, that is, physiology rather than anat-
omy. Importantly, however, the idea of blood circulation, although evolving and becoming
ever more complex, is present from the beginning, even in the early stages of the square
diagram of the heart (Fig. 4), as discussed earlier.
Thus, the concept of circulation of blood develops through the addition of new infor-
mation that is connected to previous information within the theme by the enactment of
catchments and the repetition of certain words associated with the gestures performed,
as much as from the visuals the teacher draws on the board. In this sense, the theme
provides context for newly introduced forms of text, words and images (in gestures and
drawings), while at the same time evolving as new significations are linked together as
part of the theme. Each new idea contributes to the development of the concept of blood
circulation, and they are connected to each other by means of the multimodal semiotic
resources used, which in turn achieve their signification (that is, become sense-constitu-
tive resources) within the contextual background of the theme (or communicative unit)
they are helping to develop and move forward.
The next time this catchment appears (in Episode 10, Fig. 13) the gesture is synchro-
nous with the words “pumping action” of the heart muscle, which is another way in which
the teacher refers to contraction and the consequent blood flow (in fact, in Episode 8, the
teacher has already used the word “pump” as a verb with the connotation “to send blood
out”). This catchment is a recurrence of both the concept of contraction articulated earlier,
and the analogy of the ventricles as the pump (Episode 7). Through this gesture, the teacher
associates these two ideas into one coherent unit: The contraction of the heart muscle gen-
erates the pumping action, which sends the blood out of the heart through the vessels.

Episode 10 (Lesson #2—0:34:04)


“And that’s the pumping action.”

13
L. Pozzer, W.-M. Roth

Fig. 14  Graphic representations of pitch (dotted line) and intonation (continuous line) of utterances and still
frames of the synchronous gesture in Episode 11. a Pre-stroke hold. b Stroke. c Post-stroke hold

Fig. 15  Graphic representation of variations in pitch (dotted line) and intonation (continuous line) of the
utterance that is synchronous with the stroke of the gesture (represented in the still frames) that occurs in
Episode 12

13
A cultural-historical perspective on the multimodal development…

Fig. 16  Schematic diagram of
the heart. a As it was first drawn.
b In its final appearance after the
teacher added more information
onto it

The emphasis here (as in Episode 7) is again placed on the words “[i]s,” “the” and “[p]
um[ping],” making salient the contraction movement of the ventricles. As the idea of circu-
lation is further explored, the teacher keeps the idea of contraction and blood flow present,
tying the new information to what has been previously taught. Continuing with this lecture,
two more catchments occur in Episodes 11 and 12 (Figs.  14 and 15, respectively), both
synchronized with the word “contracts,” as the idea develops when the teacher articulates
the circulation of blood by expanding on the trajectory of the blood through the different
chambers of the heart and out to other body parts.

Episode 11 (Lesson #2—0:34:10)


“The heart contracts.”

In Episode 11, when the teacher utters “heart,” he holds his hands ready for the gesture
performance (Fig. 14a), while he looks at the diagram of the heart on the chalkboard. The
perfect synchronization of the stroke of the gesture (Fig. 14b) with the word “contracts,”
as well as the emphasis placed on this word (higher pitch in “con” and higher intonation in
“trac”), constitutes the dialectical unit in which imagery and words together and in interac-
tion generate the idea of contraction. The teacher’s gaze orientation connects speech and
gesture with the diagram (Fig. 16), while the pre- and post-stroke holds serve to place even
more emphasis on the idea of contraction.

Episode 12 (Lesson #2—0:34:28)


“They both contract”

In Episode 12, the stroke of the gesture is also simultaneous with the word “contract”
(Fig. 15), but a slightly different configuration of the arms is noticeable. Here, the teacher’s
arms are higher than in the previous gestures. The teacher narrates the contraction of the
atria and how the blood flows from the atria to the ventricles. The atria are located above
the ventricles, and this may account for the performance of the gesture of contraction with
the arms slightly raised, as the contraction movement is actually happening in the upper
chambers of the heart. The diagram of the heart is also different (Fig. 16), lacking some
of the realistic features of the diagram presented in Fig. 12c, but now showing the names
of various vessels and valves and their locations (some with the use of arrows and lines).

13
L. Pozzer, W.-M. Roth

Realism is not emphasized in this diagram anymore; the form of the vessels is no longer
important (that is, it is not made salient); rather, the focus here is on naming the various dif-
ferent vessels and structures of the heart and their approximate locations. In this situation,
a realist representation of the heart would bring forward too many details that may render
the visual too complex if students were to pay attention to all of these details. Moreover,
the teacher is drawing this visual and students are expected to copy it; thus, a highly modal
visual, with several naturalistic features would require an strenuous and time-consuming
effort, which clearly was deemed unnecessary in this case; the teacher opted for making
salient the names and location of the vessels only by eliminating much of the modality of
the attibutes of the heart represented in this diagram.
In the sequence, concurrently with the gesture in Episode 13, a new term is introduced.
The teacher repeats the gesture, but this time it is associated with the word “systole.” The
term systole, cultural-historical in nature as every other term used in the lecture, refers to
the contraction movement of either the atria or the ventricles, whereas the word diastole
refers to the relaxation of the cardiac muscle. When the atria experience systole (i.e., con-
traction) the ventricles experience diastole (i.e., relaxation), which permits the blood to
flow from the atria to the ventricles. At the same time that technicality characteristic of the
language of science is introduced verbally (that is, systole, the scientific terminology for
contraction of the atrium), the recurrence of the main features (catchment) of the gesture
maintains the idea of contraction in the foreground. The gesture is performed with the arms
slightly raised, as in Episode 12, which we may see as an allusion to the fact that the con-
traction the teacher refers to here occurs in the atria, the upper chamber of the heart. The
term “systole” is a novelty, whereas the ideas of contraction of the cardiac muscle which
results in blood flow are recurrent. But both novelty and recurrence are expressed within
the same communicative meaning unit, simultaneously. It is this novelty in recurrence that
allows connections to be made across time and multimodal meaning-making resources.

Episode 13 (Lesson #2—0:34:31)


“Both atria experience systole at the same time.”

Similarly to the pre and post-stroke holds of the gesture in Episode 11, here the holds
(Fig. 17a and 17c) also contribute to emphasize the contraction movement in association
with the word “systole,” as the stroke of the gesture (Fig. 17b) coincides with the utterance
of part of this word, while other words in the sentence are synchronized with the pre and
post-stroke holds.
In Episode 14, the gesture again is slightly different from the previous catchments:
the teacher performs the gesture with both hands closed into fists (Fig. 18). This gesture
coincides with the introduction of the idea of heartbeat into the lesson. Once again, new
information is added simultaneously with recurrent ideas, all integrated into the same unit
that unfolds and develops and, therefore, appears different; whereas the word and general
features of the gesture remain the same, and thus constitute a catchment for the gesture in
Episode 7, some of the features of the gesture change in accordance with the novelty that is
being introduced.
Moreover, with the differentiation in the performance of the gesture, the teacher also
differentiates between the contraction of the atria and the contraction of the ventricles,
which occur consecutively instead of at the same time. The teacher explains that the right
and left atria contract at the same time, while the ventricles are experiencing diastole (i.e.,
relaxation), and that only afterwards do the right and left ventricles contract. This sequence
is important for the proper functioning of the heart and also for the idea of heartbeat, which

13
A cultural-historical perspective on the multimodal development…

Fig. 17  Graphic representations of pitch (dotted line) and intonation (continuous line) of utterances and still
frames of the synchronous gesture in Episode 13. a Pre-stroke hold. b Stroke. c Post-stroke hold

the teacher moves on to explain (Episode 15). In Episode 14, therefore, the teacher is again
talking about the contraction of the ventricles; consequently, the height of the arms when
performing the contraction gesture is again lower than when he performed the same gesture
in reference to the contraction of the atria. The hands in fists constitute a difference in the
imagery of the idea, which is congruent with the difference that is now introduced when
the teacher develops the concept of contraction further, differentiating between systole in
the atria and systole in the ventricles and the sequence of these events in the heart.

Episode 13 (Lesson #2—0:34:31)


“Ventricles contract.”

The next two catchments, in Episodes 15 and 16 respectively, are also performed with
the hands shaped into fists, as the teacher continues lecturing on the topic of heartbeat.

13
L. Pozzer, W.-M. Roth

Fig. 18  Graphic representations of pitch (dotted line) and intonation (continuous line) of utterances and still
frames of the synchronous gesture in Episode 14. a Stroke. b Post-stroke hold

Both gestures (Figs. 19 and 20) are performed while the teacher talks about the contraction
of the ventricles, within the context of circulation of blood within the heart and the heart-
beat. Although the concept is still the contraction of the cardiac muscle that causes the
circulation of blood, now a new idea is brought into the theme, that is, the idea that these
contractions, which have already been associated with the term systole and the “pumping
action” of the heart, are also what generate the heartbeat. Thus, since the first lesson, the
concept has developed sequentially, unfolded like the story in a novel, with the introduc-
tion of more information, but always in association with the previous information provided.
Coherence within the various resources that are dialectically related to each other within
the unfolding theme is imperative to carry this conceptual development forward. Thus, it
is not simply that the teacher adds an idea; the conceptual development requires this intro-
duction and the teacher, like an actor in a drama, is subject to the plot line that some-
one else has authored. The unfolding idea has its own requirements to which the lecturing
teacher is subjected to as much as he is the subject of.

Episode 15 (Lesson #2—0:35:38)


“Inefficient heart beat.”

When performing the gesture in Episode 15, the teacher repeats the hands movement
three times (Fig. 19a), and holds the position for a little over one second (post-stroke hold
[Fig. 19b]). This gesture occurs right after the gesture performed in Episode 14 (Fig. 18),
and the similarities between both gestures (hands in fists) attest to the continuity of the
topic of heartbeat. Before, the teacher was talking about the contraction of the ventri-
cles within the context of the sequence of events within the heart. Now, he considers the

13
A cultural-historical perspective on the multimodal development…

Fig. 19  Graphic representations of pitch (dotted line) and intonation (continuous line) of utterances and still frames of the synchronous gesture in Episode 15. a Stroke. The

13
teacher performs the same movement three times consecutively, within the same gesture. b Post-stroke hold
L. Pozzer, W.-M. Roth

Fig. 20  Graphic representation of the variations in pitch (dotted line) and intonation (continuous line) of the
utterance that is synchronous with the stroke of the gesture (represented in the still frames) that occurs in
Episode 16

Fig. 21  Graphic representations of pitch and intonation of utterances and still frames of the synchronous
gesture in Episode 17. a Stroke. b Post-stroke

possibility of both atria and ventricles contracting at the same time, and concludes that this
would be a very inefficient heartbeat. Continuing with the same idea, the teacher proceeds
to explain that the right and left ventricles also contract simultaneously. He then performs
the gesture in Episode 16 (Fig. 20). Once again, although this catchment presents the main
characteristics found in the other gestures associated with the concept of contraction, the
closed fists are maintained as a special feature that distinguishes the contraction of the ven-
tricles from the contraction of the atria, as this differentiation was introduced in Episode
13.

Episode 16 (Lesson #2—0:36:15)


“They contract at the same time.”

Over the course of the next several lessons, as the teacher continues lecturing on the cir-
culatory system, other catchments occur. In Episode 17, the catchment once more presents
variation in shape (Fig. 21), accompanying the development of the concept. While utter-
ing “contrac[ting],” the teacher gestures with the elbows positioned even higher than in

13
A cultural-historical perspective on the multimodal development…

Fig. 22  Graph representing the blood pressure (B P.) in the different vessels: the letters A (for arteries),
C (for capillaries), and V (for veins) can be seen along the vertical axis of the graph; “D.” is also visible
to the left of the verticle axis of the graph, with a line linking it to the lower picks of the line in the graph
(D stands for diastolic pressure). Also visible is the word “Systole” connected to the higher picks in the
graph line through an arrow. Below the graph, a diagrammatic representation of the heart, divided in four
chambers and with the left ventricle identified as LV; a tick blood vessel leaving the left ventricle is also
represented

Fig. 23  Graphic representation of the variation in pitch (dotted line) and intonation (continuous line) of the
utterance that is synchronous with the stroke of the gesture (represented in the still frames) that occurs in
Episode 18

the previous gestures, and with a greater distance between the hands. The main feature of
this gesture is exaggerated. When we analyze the entire communicative unit, the unfolding
theme, the exaggeration of the arms movements to perform this gesture is congruent with
the idea the teacher is introducing: blood pressure; that is, the contraction of the ventricles
needs to be powerful enough to send the blood out of the heart and into the arteries. The
lecture is about the pressure in the capillaries, veins and arteries, and in this context, the
idea of ventricular contraction is crucial; thus, the recurrent idea of ventricles contracting
is now presented to explain blood pressure in the various vessels.

Episode 17 (Lesson #3—0:13:22)


“Ventricle contracting.”

During this lesson, the teacher draws a graph on the chalkboard (Fig. 22), representing
the blood pressure in the different vessels. A diagram of a heart is also drawn, to which the
teacher adds LV in the space equivalent to the left ventricle and a thick vessel going out of
it. These representations also convey the idea of the contraction of the left ventricle send-
ing blood out of the heart, through arteries, to other parts of our body, which contributes
to arterial pressure, which is represented in the graph. The exaggerated gesture in Episode
17 is therefore associated with the idea of a powerful ventricular contraction, causing the
blood to exit the ventricle with higher pressure in the arteries. Following Kress and Van
Leeuwen’s (1996/2006) visual grammar, the top image refers to the most salient aspect of

13
L. Pozzer, W.-M. Roth

Fig. 24  Graphic representation of the variation in pitch (dotted line) and intonation (continuous line) of the
utterance that is synchronous with the stroke of the gesture (represented in the still frames) that occurs in
episode 19

Fig. 25  Still frames represent-


ing the stroke of the gesture the
teacher performs simultaneously
with a student’s utterance, as part
of Episode 20

the concept—the graph showing the intensity of the contraction in different vessels (along
the horizontal axis, “a” for arteries, “c” for capillaries and “v” for veins). The diagram bel-
low simply reproduces in much less detail the previous information presented: a tick tube
(two parallel lines slightly curving to the left) representing a vessel (an artery) inside which
oxygenated blood exits the left ventricle.
The next two catchments, in Episodes 18 and 19 respectively, also present the same
characteristics of the gesture in Episode 17 (Fig. 21), that is, an exaggerated “squeezing”
movement with the arms. In Episode 18, the gesture is synchronized with the utterance
“surge [of blood]” (Fig.  23), which is a reference to blood pressure caused by the blood
leaving the heart as a result of ventricular systole. The larger gesture provides a visual rep-
resentation for the pressure with which the blood leaves the heart during ventricular systole
(“as the ventricle contracts” in Episode 19, Fig.  24), which is the main idea the teacher
communicates at this moment.

Episode 18 (Lesson #3—0:13:26)


“There’s a surge of blood”
Episode 19 (Lesson #3—0:13:33)
“As the ventricle contracts”

Episode 20 presents the next catchment (Fig.  25), which occurs during another les-
son dealing with the same topic. This catchment is particularly interesting, as the teacher
performs it in silence, synchronized with a student’s utterance. His hands are once again
closed in fists; however, in this particular case, the chalk is held in the right hand and a
piece of paper in his left hand, which may account for the closed fists. He keeps his gaze on

13
A cultural-historical perspective on the multimodal development…

Fig. 26  Graphic representation of the variation in pitch (dotted line) and intonation (continuous line) of the utterance that is synchronous with the stroke of the gesture (repre-
sented in the still frames) that occurs in episode 21. The teacher performs the same movement three times consecutively, within the same gesture

13
L. Pozzer, W.-M. Roth

Fig. 27  Graphic representation of the variation in pitch (dotted line) and intonation (continuous line) of the
utterance that is synchronous with the stroke of the gesture (represented in the still frames) that occurs in
Episode 22. The teacher performs the same movement eleven times consecutively, within the same gesture

the student while gesturing and nods after the student has finished talking, which may be
interpreted as agreement. Nevertheless, the teacher is still lecturing on blood pressure, this
time focusing on the flow of blood within the blood vessels, and this catchment, although
performed in synchrony with someone else’s speech, is also associated with the concept of

13
A cultural-historical perspective on the multimodal development…

contraction movement. It is interesting to note here that the teacher actively orients himself
to the student’s discourse, to the point of even repeating a gesture he has used throughout
several lessons in synchrony with this student’s speech; as Vološinov (1973) suggested,
“meaning is realized only in the process of active, responsive understanding … [it] is the
effect of interaction between speaker and listener” (pp. 102–103, emphasis in the original).

Episode 20 (Lesson #3—0:18:45)

The last two catchments for this gesture occur while the teacher lectures on the regula-
tion of the heartbeat. The teacher repeats the gesture in Episode 21 three times (Fig. 26),
whereas in Episode 22, the movement is repeated eleven times (Fig. 27). These fast repeti-
tions provide imagery for the concept the teacher is explaining: The rhythmically, regular
contractions of the cardiac muscle, which in this particular moment are associated with
the idea of heartbeat. The particular features of these two catchments are the hands move-
ments, as opposed to the arms movements of the previous enactments of the gesture.

Episode 21 (Lesson #3—0:47:18)


“And it will continue to beat.”

Insofar as the teacher performs various repetitions, the synchronization of the stroke of
the gesture is not with a particular word, but rather, the associated idea of heartbeat, and it
is exactly these rhythmic repetitions (both in gestures and in words in Episode 22 [Fig. 27])
that make available this idea.

Episode 22 (Lesson #3—0:47:26)


“That heart will continue to beat a normal rhythm bip bip bip bip bip.”

Thus, from an initial reference to the ventricle as the heart pump, to the idea of the
contractions of the atria and the ventricles, with their sequential rhythmic occurrence, to
the ideas of pulse and heartbeat, the recurrence of gestures provided a thread that con-
nected all the new ideas added to the concept of circulation of blood. Here the differ-
ent resources available (words, pitch, intonation, diagrams, body and gaze orientations,
and gestures) are dialectically related and contribute one-sided, partial expressions of a
whole that is indivisible in its signification.
During the next lessons, the teacher lectures on the lymphatic system, and this par-
ticular gesture no longer occurs. This is further indication that the teacher has achieved
stabilization of the theme of contraction of the cardiac muscle, and is then ready to
start lecturing on a new curricular unit. Throughout his explanations of blood circula-
tion from the heart to other body parts and back to the heart, the teacher continuously
made use of the same gesture as a way to connect different words, diagrams, and ideas
that together formed the concept being taught. Each time this gesture was performed,
the teacher made available the recurrent idea of contraction, which was presented in
association with different information. This association of novelty and recurrence
within the unfolding and developing theme made possible the development of the ideas
in a sequentially manner, not only within the same lesson, but also across consecutive
lessons.

13
L. Pozzer, W.-M. Roth

Towards a view of lectures as dramatic performances

In this article, we present the analysis of the development of a particular scientific concept
within and across lessons through a communicative perspective, focusing specially on the
integration of various different sense-constitutive resources that are dialectically related
within the same theme. We integrate the analysis of talk into a larger, cultural-historical
activity theoretic framework concerned with understanding a continuously changing world
generally and the continuous evolution of thinking, speaking, and their relation (theme).
Adapting cultural-historical concepts of growth points and catchments as a means of keep-
ing track of the development of ideas during teaching, we shed light on how in a lecture
new ideas (meanings) are introduced into the unfolding and developing theme (commu-
nicative unit) and how they are associated with previously presented ideas to (per)form a
concept. For instance, when the teacher first introduced the idea of ventricular contraction,
he connected the imagery of the idea of contraction, realized in the gesture, with the words
“the ventricles are the big pumps,” at the same time making use of an analogy and dif-
ferentiating the function of the ventricles from that of the atria. This complex communica-
tive unit also included the diagram of the heart in the chalkboard and prosodic aspects of
his discourse, which served to emphasize certain words in his utterances. As the teaching
progressed, the teacher repeated the contraction movement in combination with different
utterances and different diagrams, introducing new terms, such as systole, for example, but
maintaining the idea of contraction within the theme by means of the recurrence of the
gesture (i.e., catchments).
The novelty in the recurrence of words, gestures, and diagrams provides coherence
within the theme, emphasizing both the similarities and the differences of each new idea
introduced in relation to the previous ideas presented. Thus, differences in the features of
gestures, for example, reflected differences in the ideas introduced. When the teacher ges-
tured while talking about the contraction of the atria, he performed the gestures with his
arms raised above the position where he performed the same gestures when talking about
the ventricles, in a direct allusion to the fact that the atria are the top chambers of the heart,
which was also illustrated in the diagram of the heart on the chalkboard. When talking
about the ventricular contraction, the gesture features changed, as did the diagram, repre-
senting the ventricles as much larger structures than the atria and with many more natural-
istic details. Therefore, within a multimodal communicative unit, new ideas were added to
recurrent ones, making these evolve, resulting in a progressive, sequential development of
the scientific concept within and across lessons. The integration of multiple resources, each
presenting a part of the whole through a different non-reducible modality, constitutes the
means through which the teacher performs a scientific concept, bringing it to life on the
stage of the classroom. However, to understand such (theatrical) performances, we have
to go beyond an individualistic psychology of the teacher as a cognitive subject. This is so
because the “character of the actor’s stage experiences… comprise a part of the complex
function of the artistic work that has a definite societal, class function historically deter-
mined by the whole state of the mental development of the epoch and class” (Vygotskij
1984, p. 323). As a consequence, affective and intellectual dimensions come to be inter-
woven, leading us to a historical plane of analysis rather than to naturalistic (biological,
individual) psychology. This position, therefore, opens up science lectures to ideological
analysis and societal critique. This is important because it has been shown that lecture per-
formances and the interaction of speech and gestures may assist or hinder student under-
standing (Roth 2001). How do societal, class functions mediate conceptual development

13
A cultural-historical perspective on the multimodal development…

of students attending lectures? The teaching and learning of science, even if it occurs in an
apparently neutral lecture situation, can now be analyzed and understood from the perspec-
tive of a critical psychology that operates on an historical plane of analysis rather than one
of individualistic psychology.
Our account of the development of ideas and concepts has the teacher as its agential
subject in performance. However, “like any concrete mental phenomenon, the actor’s
work represents a part of the social-psychological reality that must be studied and defined
primarily in the context of that whole to which it belongs” (Vygotskij 1984, p. 324, our
emphasis). This emphasizes the role of the teacher and insufficiently thematizes existing
constraints that are associated with the distributed and situated moments of lived-experi-
encing that gather together and integrate the different communicative forms within a given
context. Lecturers cannot just say what they wanted. Instead, to speak in an efficient and
useful way, the speaking has to address the audience, be intelligible, and allow the listener
to understand connections and coherences. As actor, the lecturer creates on the stage what
subsequently become real (affective, intellectual) entities for the whole (theatrical) audi-
ence. Before it becomes “the subject of the actor’s embodiment, these were given a liter-
ary formulation, they were borne in the air, in societal consciousness” (p. 324). Moreover,
because on shorter time scales spontaneous speech provides resources to the speaker per-
taining to what is said, there is a moment of unpredictability regarding just what will be
the next utterance. This makes the lecturer not only the subject of the development of the
ideas and concepts, but he is also subjected to the unfolding communicative process. The
appropriate unit of analysis is the unfolding idea in which the concept is included. But that
idea is connected to and embedded in the context where it has the function of fulfilling a
task (Vygotskij 1984). The teacher has his part in this process; but there are other “forces”
to which the teacher is subject.
Whenever there are student queries or additions, there are new moments entering this
process, changing it, as the lecturer responds. The lecturer may also have a sense from the
non-verbal responses of the audience that something was not understood, at which point
we would expect to observe the repetition and change of a stretch of the lecture. In fact, a
change in expression would be required, for little would be gained if the repetition would
be exactly the same—though the very fact that what is repeated occurs against the original,
which forms a ground, makes the repetition different (Deleuze 1968). Other constraints
derive from the subject matter itself. The lecturer is familiar with the standard presenta-
tions in textbooks. What we observe, as lecture, is some refracted version of the knowledge
of a field and, therefore, a cultural process through and through.
Lectures as a means for teaching have been much maligned (Friesen 2011). Yet they
have their stable place in the repertoire of science education (Friesen and Roth 2014). One
may sometimes hear the suggestion that textbooks or computer-based multimedia also
could present concepts and ideas. However, our account exhibits features that do not exist
in these other media for producing coherence and development within and across lectures.
Whereas these other media constitute fixed representation, the living lecture is a perfor-
mance. Just as theatre audiences appreciate a play generally and one-person-show or stand-
up comedy particularly, we might ask, why should the audience of a lecture not be able to
appreciate it as a performance in the same way that theatre audiences appreciate the per-
formances they observe? Audiences do get the point of a drama or the joke of a stand-up
comedian. Why should the audience of a lecture not understand its main point, an idea or
concept?
One promising avenue is the category of lived-experiencing because it integrates across
practical, intellectual, and affective moments of the human existence (Vygotskij 2001). It is

13
L. Pozzer, W.-M. Roth

in and through experience that further experiences are mediated, so that each multimodal
communicative experience transforms the person, leading to learning and development:
Lived-experiencing is a moving force of human social development (Schütz 1932). The
category is especially promising because it leads us to a cultural-historical appreciation of
individual lectures: “The experiences of the actor, his emotions, appear not as functions
of his personal mental life, but as a phenomenon that has an objective societal sense and
significance, serving as a transitional step from psychology to ideology” (Vygotskij 1984,
p. 328). That is, viewing lectures as performance allows us to make yet another connection
between this research and Vygotsky’s cultural-historical approach. This is so because “all
cultural things are social… They are internalized relations of a social order, transferred to
the individual personality, the basis of the social structure of the personality” (Vygotskij
2005, p. 1023, original emphasis). When we observe human performances, therefore, we
see a “small drama… psychology in terms of drama” (p. 1023), where the original rela-
tions come to be played out again. Because the lecturer had been part of a relation before,
as one of several actors, the small drama we observe in the performance includes the lec-
turer as an actor subject to as much as subject of the drama.

Coda

A clearer understanding of how scientific concepts are communicated (that is, taught) dur-
ing science lectures is crucial for the advancement of studies in science teaching and learn-
ing. We also need to understand how societal and class functions determine the ideas and
their development in the performance and, subsequently, in the reception of these perfor-
mances on the part of the audience. The perspectives that both Vološinov and Vygotsky
bring to the analysis of communication provide the tools for ideological analysis (Roth
2014). They therefore are important tools for grasping the differences in the reception of
science lectures on the part of middle versus under- and working-class students. In this
article, we provide a detailed analysis of the communicative development of scientific con-
cepts during lectures, from the perspective of the teacher’s communicative, multimodal
performance, which resulted in a more complex and thorough understanding of the inte-
gration of multimodal resources in the classroom. How the students perceive and make
use of these various multimodal resources, however, should be the topic of future studies,
which may help us to understand how multimodality in science lectures influences science
learning.
Gestures have been shown to constitute an important aspect of conceptual development
that arises from experience encompassing practical (gestural), affective, and intellectual
moments (Roth 2000). From the perspective of the learner, they are developmentally ear-
lier expressions of change in understanding than the corresponding verbal expressions (e.g.
Iverson and Goldin-Meadow 2005). They may also play an important role in learning from
lectures, given the results of neuroscientific research on mirror neurons, which shows that
we understand movements only when we already have the same movements in our reper-
toire. Future research might be conducted to find out the role of gestures and other non-
verbal communicative means in the understanding of audience members. If one or more
of the non-verbal means were to be suppressed, would this influence the way in which the
audience understands the ideas and concepts performed?

13
A cultural-historical perspective on the multimodal development…

Acknowledgements  The research project from which this article derives was funded by a grant from the
Social Sciences and Humanities Research Council of Canada to Wolff-Michael Roth. We thank the students
for their participation in this project, and we would like to express our gratitude especially to Mr. Burl
Jantzen, for his collaboration and interest in this project. We thank Diego M. Ardenghi for helping us with
the videotaping and subsequent transcribing of the videotapes. We thank reviewers and journal editors for
their insightful comments on earlier versions of this manuscript, which helped us improve the article.

References
Aubusson, P. J., Harrison, A. G., & Ritchie, S. M. (Eds.). (2006). Metaphor and analogy in science educa-
tion. Dordrecht: Springer. https​://doi.org/10.1007/1-4020-3830-5.
Deleuze, G. (1968). Différence et répétition [Difference and repetition]. Paris: Presses Universitaires de
France.
El’konin, B. D. (1994). Vvedenie v psixologiju razvitija: b tradicii kul’turno-istoričeskoj teorii L. S. Vygot-
skogo [Introduction to the psychology of development: in the tradition of the cultural-historical theory
of L. S. Vygotsky]. Moscow: Trivola.
Friesen, N. (2011). The lecture as a transmedial pedagogical form. Educational Researcher, 40(3), 95–102.
https​://doi.org/10.3102/00131​89X11​40460​3.
Friesen, N., & Roth, W.-M. (2014). The anatomy lecture then and now: A Foucauldian analysis. Educa-
tional Philosophy and Theory, 46, 1111–1129. https​://doi.org/10.1080/00131​857.2013.79687​2.
Iverson, J. M., & Goldin-Meadow, S. (2005). Gesture paves the way for language development. Psychologi-
cal Science, 16, 367–371. https​://doi.org/10.1111/j.0956-7976.2005.01542​.x.
Kress, G., Jewitt, C., Ogborn, J., & Tsatsarelis, C. (2001). Multimodal teaching and learning: The rhetorics
of the science classroom. London: Continuum. DOI.
Kress, G. & Van Leeuwen, T. (1996/2006). Reading images: The grammar of visual design. New York:
Routledge.
Leont’ev, A. A. (1969/1971). Jazyk, rec’, recevaja dejatel’nost’. Moscow, Russia: Prosveščenje (We
checked our translation with the German one, Sprache, Sprechen, Sprechtätigkeit, C. Heeschen, Trans.
Stuttgart: Kohlhammer, 1971).
Leont’ev, A. N. (1978). Activity, consciousness and personality. Englewood Cliffs, NJ: Prentice Hall.
McNeill, D. (1985). Language viewed as action. In J. V. Wertsch (Ed.), Culture, communication, and cogni-
tion: Vygotskian perspectives (pp. 259–270). Cambridge: Cambridge University Press.
McNeill, D. (2002). Gesture and language dialectic. Acta Linguistica Hafniensia, 34, 7–37. https​://doi.
org/10.1080/03740​463.2002.10414​607.
McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press.
Merleau-Ponty, M. (1945). Phénoménologie de la perception [Phenomenology of perception]. Paris:
Gallimard.
Pozzer-Ardenghi, L., & Roth, W.-M. (2007). On performing concepts during science lectures. Science Edu-
cation, 91(1), 96–114. https​://doi.org/10.1002/sce.20172​.
Pozzer-Ardenghi, L., & Roth, W.-M. (2008). Catchments, growth points, and the iterability of signs in class-
room communication. Semiotica, 172(1/4), 389–409. https​://doi.org/10.1515/SEMI.2008.105.
Roth, W.-M. (2000). From gesture to scientific language. Journal of Pragmatics, 32, 1683–1714. https​://doi.
org/10.1016/S0378​-2166(99)00115​-0.
Roth, W.-M. (2001). Gestures: Their role in teaching and learning. Review of Educational Research, 71,
365–392. https​://doi.org/10.3102/00346​54307​10033​65.
Roth, W.-M. (2014). Science language Wanted Alive: Through the dialectical/dialogical lens of Vygot-
sky and the Bakhtin circle. Journal of Research in Science Teaching, 51, 1049–1083. https​://doi.
org/10.1002/tea.21158​.
Roth, W.-M., & Bowen, G. M. (1999). Decalages in talk and gesture: Visual and verbal semiotics of ecology
lectures. Linguistics & Education, 10, 335–358. https​://doi.org/10.1016/S0898​-5898(99)00012​-1.
Roth, W.-M., & Friesen, N. (2014). History and the relationship between scientific and pedagogical knowl-
edge: Anatomy lectures then and now. Journal of Curriculum Studies, 46, 180–200. https​://doi.
org/10.1080/00220​272.2013.77359​7.
Roth, W.-M., & Pozzer-Ardenghi, L. (2006). Tracking situated, distributed, and embodied communication
in real time. In M. A. Vanchevsky (Ed.), Focus on cognitive psychology research (pp. 237–261). Haup-
pauge, NY: Nova Science.
Schütz, A. (1932). Der sinnhafte Aufbau der sozialen Welt [The phenomenology of the social world].
Vienna: Julius Springer.

13
L. Pozzer, W.-M. Roth

Unsworth, L. (2001). Teaching multiliteracies across the curriculum. Buckingham: Open University Press.
Vološinov, V. N. (1973). Marxism and the philosophy of language. New York: Seminar Press.
Vygotskij, L. S. (1984). Sobranie sočinenij tom 6 [Collected works vol. 6]. Moscow: Pedagogika.
Vygotskij, L. S. (2001). Lekcii pop pedologgii [Lectures on pedology]. Iževsk: Udmurtskij Universitet.
Vygotskij, L. S. (2005). Psixologija razvitija čeloveka [Psychology of human development]. Moscow:
Eksmo (We checked our translation of the chapter on thinking and speech with the German, Denken
und Sprechen, Munich, German: Beltz, 2002).

Lilian Pozzer  is an Assistant Professor at the Department of Curriculum Teaching and Learning, University
of Manitoba, in Winnipeg, Canada. Her research in science education focuses on classroom interaction and
communication, multimodality, inscriptions and gestures studies. She is the author of Staging and perform-
ing scientific concepts: Lecturing is thinking with hands, eyes, body, and signs (Sense Publishers, 2010;
with W.-M. Roth) and co-author of Critical graphicacy: Understanding visual representation practices in
school science (Springer-Kluwer, 2005; with W.-M. Roth and J.-Y. Han).

Wolff‑Michael Roth  is Lansdowne Professor of Applied Cognitive Science at the University of Victoria.
He investigates knowing and learning across the lifespan in formal educational, leisure, and workplace
settings. His recent book publications include Cognition, Assessment and Debriefing in Aviation (2017),
Understanding Educational Psychology: A Late Vygotskian, Spinozist Approach (with A. Jornet, 2017), and
Concrete Human Psychology (2016).

13

You might also like