BOOKS AND STORIES IN OTHER FORMS/MEDIA

Multimodal Texts in Children’s Literature

It is easy to envision a classroom that relies on the use of a

print textbook and resources that primarily use printed words and
visual images to represent meaning. However, print resources are
changing in ways that are reflective of the multiple ways, or modes,
that are used to communicate within digital contexts. It could be
recalled that modes of communication encompass all forms of
expression, including “Visual Meanings (images, page layouts,
screen formats); Audio Meanings (music, sound effects); Gestural
Meanings (body language, sensuality); Spatial Meanings (the
meanings of environmental spaces, architectural spaces); and
Multimodal Meanings” (New London Group, 1996, p. 80). Though
children’s literature, especially picture books, rely mainly on print
and visual modes (i.e., words combined with pictures), there are
growing numbers of children’s books that creatively incorporate
audio, gestural, and spatial modes as well. Multimodal texts are
capable of drawing on students’ strengths and preparing them for a
multimodal society where individuals communicate through audio,
gestural, visual, spatial, and print resources, as well as various
combinations of these modes.

² The audio mode, which includes both sound and music, is

present in many aspects of daily life, including use in film and
television, as well as content accessed on iPods, tablets,
smartphones, and at popular websites, such as YouTube. This
proliferation of sound and music in daily life heightens the
importance of teaching students how to use the audio mode to
understand and communicate messages.

For the purposes of this discussion, the audio mode is not

referring to books that talk about music or sound, as these do not
necessarily make meaning using the audio mode. Nor is it referring
to audio books (e.g. a CD or audio file in which someone is reading
the book out loud) which consist of spoken text or words that are
primarily a linguistic meaning. Rather, this discussion focuses on the
ways that books can represent meaning through the audio mode
using visual and linguistic information, or using other symbolic
representations that allow for the possibility of the reader to use the
audio mode to make meaning and comprehend the message. For
example, if, based on the information represented in a piece of text,
a reader makes a sound or sings a song to make meaning of that
information, the audio mode is being used to communicate. Kress
(2000) explained that to determine which mode is being used to
communicate a message, one can think about the sense that is used
to make meaning of represented information. Figure 1 provides an
example of how a linguistic (print) message, a visual message, and
an audio message might appear in a children’s book.

Another way picture books can represent the audio mode is by

including an external audio CD or a link to an audio file. In The
Yellow Umbrella by Jae Soo Liu (2002), the music for this text was
composed solely for the purpose of accompanying the illustrations.
The music can be listened to in two different ways. There is a one-
track short version that has about 20 to 30 seconds of music for
each page. There is also a long version which provides 1 to 2
minutes of music for each page, each on a different track. Without
the music, a reader might only pause for a moment on the page to
consider the small amount of action they see. With the musical
accompaniment, the mood and spirit of each page changes. In a
wordless picture book, the reader typically uses their own words to
form a story related to what is happening in the pictures. The music
adds more meaning than the pictures can convey alone to help the
reader construct this story. All of the above examples demonstrate
some of the ways authors and publishers work together to
communicate a wider range of messages, using not just print and
visual modes but the audio mode as well.

Audiobooks online. For as long as schools are closed, online

audiobooks are open. Right now, young learners everywhere can
instantly stream an incredible collection of stories, including titles
across eight different languages, that will help them continue
dreaming, learning, and just being kids. Stories are free to stream on
desktop, laptop, phone or tablet. For samples, you can access
https://www.penguin.co.uk/articles/children/2019/jan/children-s-
audiobooks-for-a-listen-aloud-adventure.html.

When selecting audio books for your children, the process is

much the same as what you'd use to evaluate other forms of
children's literature.

 How long is the audio book? The younger your child, the
shorter the story should be.
 What is the age group recommendation? It's true that these
recommendations are only a general guide, but they are still a
good starting point for choosing audio books that will hold your
child's interest.
 Does the story relate to a topic of interest to your child? Just as
when you're choosing regular books to read with your child, it's
helpful to choose audio books that tap into your child's
interests. Athletes love to hear sports stories, while animal
lovers might enjoy books about children and their pets.
 Who is the narrator? When choosing audio books, you want a
story read by someone who speaks as expressively as
possible. Many well-known actors have actually recorded audio
books for children, so keep this in mind as you're looking for
stories.

² Using children’s literature that includes multiple modes of

communication offers more opportunities to invite students to
respond using arts-based forms, such as visual art, drama,
music, and dance. Students may be more encouraged to
respond to literature if teachers use more familiar terms, such
as music, art or drawing, acting or drama, and dance or
movement rather than discussing modes, like gestural, spatial,
or audio, as terms. Arts-based responses allow students to use
all their senses as they make meaning. As an example, an arts-
based response might be one in which students act out what
 they think might happen in a story, create a rhythmic pattern or
tune to symbolize each character in a book, or move the same
way as they believe characters felt or acted in given situations
to help analyze a character’s emotions and motivations.
Multimodal books are not required tools for arts-based
responses in that teachers can encourage or create arts-based
response activities for any book, but when a text already
utilizes audio, gesture, movement, or space in creative ways, it
can offer students a model and set the stage for engaging in
arts-based responses.

 popular shows like Blue’s Clues and Ni Hao Kai-Lan. The

downside is that content is limited to Nickelodeon shows.
Noggin leans more “entertainment” than “educational.”

 Common Sense Media: Common Sense provides reviews and

ratings on children’s products, including movies, TV shows,
books, and games. While children’s maturity and abilities vary,
it provides good, age-specific guidelines. Common Sense is a
free site and is a great source to check products to save time
watching yourself, particularly if you’re worried about a potential
movie or video game.

In Technology and Interactive Media as Tools in Early Childhood

Programs Serving Children from Birth through Age 8, the National
Association for the Education of Young Children (NAEYC) and the
Fred Rogers Center state that “appropriate experiences with
technology and media allow children to control the medium and the
outcome of the experience, to explore the functionality of these
tools, and pretend how they might be used in real life.”
Lisa Guernsey, author of Screen Time: How Electronic Media—
From Baby Videos to Educational Software—Affects Your Young
Child, also provides guidance for families and early educators. For
example, instead of applying arbitrary, “one-size-fits-all” time limits,
families and early educators should determine when and how to use
various technologies based on the Three C’s: the content, the
context, and the needs of the individual child. They should ask
themselves following questions:

 Content—How does this help children learn, engage, express,

imagine, or explore?
 Context—What kinds of social interactions (such as
conversations with parents or peers) are happening before,
during, and after the use of the technology? Does it
complement, and not interrupt, children’s learning experiences
and natural play patterns?
 The individual child—What does this child need right now to
enhance his or her growth and development? Is this technology
an appropriate match with this child’s needs, abilities, interests,
and development stage?
(https://tech.ed.gov/earlylearning/principles/)