The development and temporal integration
of co-speech gesture in narrative speech: A longitudinal study
Júlia Florit-Pons1, Ingrid Vilà-Giménez1, Patrick Louis Rohrer1,2, Pilar Prieto3,1
1
Dept. of Translation and Language Sciences, Universitat Pompeu Fabra, Barcelona, Catalonia
2
Laboratoire de Linguistique de Nantes - UMR 6310, Université de Nantes, Nantes, France
3
Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona, Catalonia
julia.florit04@estudiant.upf.edu, ingrid.vila@upf.edu,
patrick.rohrer@upf.edu, pilar.prieto@upf.edu
Abstract
Children from 5 to 6 years of age have been shown to start
producing non-referential beat gestures in narrative speech.
However, it still remains unclear how the use of these non-
referential gestures along with referential iconic gestures
evolves over time in children’s narrative discourse, and how the
temporal integration between gestures and prosodic
prominence in speech develops. To assess this question, the
current study examines changes in gesture production in terms
of referentiality and temporal alignment properties in narrative
speech using a longitudinal database consisting of 332
narratives performed by 83 children at two different time points
in development (e.g., 5-6 years of age and two years later).
Results revealed that non-referential beat gesture production
increases significantly during these periods of development.
Furthermore, this study provides evidence that referential
iconics and non-referential beat gestures behave similarly in
terms of temporal alignment, showing a high percentage of
gesture integration with prosodic prominence (i.e., between
75% and 85%).
Index Terms: non-referential beat gestures, referential iconic
gestures, temporal alignment, narrative speech
1. Introduction
Previous research has provided evidence that the development
of gesture in children starts in the earliest stages of language
development. By the age of 5-6, children already start
producing non-referential beat gestures in narrative speech (i.e.,
pragmatically meaningful gestures which highlight important
linguistic functions in discourse ([1], [2])), and use them to
mark discursive relationships in speech ([3], [4], [5]). These
studies found that as children grow, the number of beat gestures
produced increases. However, further research is still needed to
examine the relationship between beats and other types of
gestures in narrative discourse. While some authors have found
that referential iconics tend to occur more often than non-
referential gestures ([4], [5]), as far as we are concerned, no
prior study has directly compared the production of these
gestures during development from a longitudinal point of view,
examining the same participants. The current investigation will
contribute to this discussion and will assess the differences in
gesture production in children’s narrative speech comparing the
use of iconic and beat gestures at two points in development (at
the age of 5-6, and two years later).
Gesture and speech have been found to be temporally
integrated with prosodic prominence in adult’s discourse ([2],
[7]). It is well-established that infants start coordinating
pointing gestures and speech before their first words [8],
showing that alignment is found from early stages. However,
less is known about gesture-speech temporal integration as
children begin adding non-referential beat gestures to their
gestural repertoire. To our knowledge, only one study [6] has
examined the alignment between gesture and children’s speech
in 12 6-year-olds. This investigation found that children
produced beat gestures which were linked to lexical words.
However, no clear/significant relation with pitch accents was
found, even though some children had already started to
produce beats which were aligned with pitch accented syllables.
The current study will complement existing research using a
longitudinal database, with more than 80 participants.
This paper aims to assess gesture development and gesture-
speech alignment in children’s narrative discourse. In line with
prior research ([4], [5], [6]), we hypothesize that by the age of
5, children will already produce beat gestures, and some of them
will have already started to develop the ability to align gestural
prominence with prosodic prominence, as previous studies have
shown (e.g., [6]). We think that by the age of 7, the number of
beats used by children will increase significantly, essentially
taking up a larger proportion of the children’s total gestural
repertoire, reflecting a more adult-like performance.
2. Methods
2.1. Participants
Eighty-three children from the Girona area of Catalonia took
part in [9]’s corpus of Catalan narrative speech development.
The children were recorded at two different time points: 5 to 6
years of age (Time 1; M = 5.9; SD = 0.55), and two years later
(Time 2; M = 7.98; SD = 0.60). All participants were typically
developing children without any prior history of language or
communication disorders. Parents filled out a family language
questionnaire [10] and consented to having their children be
video-recorded. All children were Catalan-dominant
Catalan/Spanish bilinguals (88.39%; SD = 9.6).
2.2. Materials and Procedure
At age 5-6, children participated in a narrative retelling task, in
which they were asked to watch two wordless cartoons and
retell them to the experimenter. Two years later, the same
children did the same task with the same cartoons they had
watched at Time 1. For further details on the procedure, see [1]
and [11]. All in all, [9]’s database contains a total of 332 stories
(83 children × 2 stories × 2 time points), totaling approximately
166 minutes of speech.
2.3. Gesture and Speech coding system
To assess gesture-speech temporal alignment, all target video-
recordings were coded in terms of gesture type and gesture
phasing and the sections containing gestures were transcribed
orthographically. Each story was annotated following the
gesture labeling scheme proposed by [12]. Pitch-accented
syllables were also annotated to assess alignment between the
gesture stroke (i.e., the mandatory phase of the gesture) and
speech.
3. Results
A GLMM analysis was run for each dependent variable: the
overall number of gestures, and gesture alignment (coded as
either aligned or misaligned). In both analyses, Time, Gesture
Type and their interaction were set as fixed factors. Participant
was set as random factor.
Results showed a main effect of Time (χ2(1) = 18.977, p <
.001), showing that children produced more gestures at Time 2
(d = 0.76, p < .001) than at Time 1, regardless of Gesture Type;
and also, a main effect of Gesture Type (χ2(1) = 11.688, p =
.001), with more beats (d = 0.41, p = .007) than iconics,
regardless of Time. Moreover, an interaction between Time and
Gesture Type was found to be significant (χ2(1) = 5.212, p =
.022). Post-hoc analyses revealed that while gesture production
increased from Time 1 to Time 2 for both gesture types (beat: d
= 1.00, p < .001; iconic: d = 0.52, p = .020), there were
significantly more beat gestures than iconics at Time 2 (d =
0.65, p < .001) (Figure 1).
Figure 1: Number of non-referential beat gestures and
referential iconic gestures in Time 1 and Time 2
In terms of temporal alignment, results showed a main
effect of Alignment (χ2(1) = 194.949, p < .001), indicating that
both in Time 1 and in Time 2 there were more gestures aligned
(d = 1.36, p < .001) than misaligned. However, no interactions
were found to be significant (Time × Alignment, χ2(1) = 0.994,
p = .319; Gesture × Type Alignment, χ2(1) = 0.284, p = .594;
Time × Gesture Type × Alignment, χ2(1) = 2.435, p = .119).
These results show that there are no clear differences between
how children align gestures in the two time points, and that the
alignment patterns for non-referential beat gestures and
referential iconic gestures are similar (76.67% for beats and
84.62% for iconics in Time 1, and 79.42% and 76.96%,
respectively, in Time 2).
4. Conclusions
The current study is in accordance with previous research ([4],
[5]), as it shows that by the age of 5, children already produce
non-referential beats. Also, our results show that non-referential
beats increase prominently with age, but outnumber the
production of iconics, and thus suggest that the increase of beats
is an indication of gestural development in children’s narrative
discourse, as prior studies pointed out (e.g., [4], [5]).
Expanding on initial findings from the literature (e.g., [6]),
we claim that by the age of 5 and 6, children already produce
non-referentials which are aligned with the prosodic structure
of speech. This temporal integration of gesture and speech has
also been found in referential gestures. Therefore, we argue that
both referential and non-referential gestures align with prosodic
prominence in children’s narratives.
5. Acknowledgements
This research was benefited from funding awarded by the
Spanish Ministry and Competitiveness (FFI2015-66533-P), by
the Spanish Ministry of Science, Innovation and Universities,
Agencia Estatal de Investigación, and Fondo Europeo de
Desarrollo Regional (PGC2018-097007-B-100) and by the
Generalitat de Catalunya (2014 SGR_925; 2017 SGR_971).
6. References
[1] I. Vilà-Giménez and P. Prieto, “Encouraging kids to beat:
Children’s beat gesture production boosts their narrative
performance,” Developmental Science, in press, 2020.
[2] S. Shattuck-Hufnagel and A. Ren, “The prosodic characteristics
of non-referential co-speech gestures in a sample of academic-
lecture-style speech,” Frontiers in Psychology, vol. 9, no. 1514,
pp. 1–13, 2018.
[3] J. M. Colletta, “Comparative analysis of children’s narratives at
different ages: A multimodal approach,” Gesture, vol. 9, no. 1, pp.
61–69, 2009.
[4] J. M. Colletta, C. Pellenq, and M. Guidetti, “Age-related changes
in co-speech gesture and narrative: evidence from French children
and adults,” Speech Communication, vol. 52, no. 6, pp. 565–576,
2010.
[5] J. M. Colletta et al., “Effects of age and language on co-speech
gesture production: an investigation of French, American, and
Italian children’s narratives,” Journal of Child Language, vol. 42,
no. 1, pp. 122–145, 2015.
[6] M. Mathew, I. Yuen, and K. Demuth, “Talking to the beat: Six-
year-old’s use of stroke-defined non-referential gestures,” First
Language, vol. 38, no. 2, pp. 11–128, 2018.
[7] S. Shattuck-Hufnagel, A. Ren, M. Mathew, I. Yuen, and K.
Demuth, “Non-referential gestures in adult and child speech: Are
they prosodic?,” in Speech Prosody 2016, J. Barnes, A. Brugos,
S. Shattuck-Hufnagel, and N. Veilleux, Eds., 2016, pp. 836–839.
[8] N. Esteve-Gibert and P. Prieto, “Infants temporally coordinate
gesture-speech combinations before they produce their first
words,” Speech Communication, vol. 57, pp. 301–316, 2014.
[9] I. Vilà-Giménez, J. Florit-Pons, G. Gurrado, P. L. Rohrer, and P.
Prieto, “Audiovisual corpus of Catalan children’s narrative
speech development,” CHILDES corpus, in preparation.
[10] L. Bosch and N. Sebastián-Gallés, “Evidence of early language
discrimination abilities in infants from bilingual environments,”
Infancy, vol. 2, pp. 29–49, 2001.
[11] I. Vilà-Giménez, A. Igualada, and P. Prieto, “Observing
storytellers who use rhythmic beat gestures improves children’s
narrative discourse performance,” Developmental Psychology,
vol. 55, no. 2, pp. 250–262, 2019.
[12] P. L. Rohrer et al., “The MultiModal MultiDimensional (M3D)
labelling scheme for the annotation of audiovisual corpora,”
submitted at this conference.