Research in Science Education (2023) 53:357–375

https://doi.org/10.1007/s11165-022-10058-6

Distribution of Visual Representations Across

Scientific Genres in Secondary Science Textbooks:
Analysing Multimodal Genre Pattern of Verbal-Visual
Texts

Kok-Sing Tang1

Accepted: 19 May 2022 / Published online: 15 June 2022

© The Author(s), under exclusive licence to Springer Nature B.V. 2022

Abstract
There is currently a lack of systematic study examining the integration of verbal-visual
ele- ments in science textbooks. In particular, few research has investigated how different
types of visual representations (e.g. photograph, diagram, table) vary across the major
written genres of science (e.g. information report, explanation). Based on the perspective
of social semiotics, this paper aims to understand the co-dependency between visual
representations and written genres by analysing a corpus of multimodal texts from a major
Australian text- book series. Quantitative and qualitative analyses reveal a number of
interesting patterns in terms of how several types of representations tend to appear with
certain genres, as well as possible reasons for this co-dependency. In particular, the
analysis accounts for why and how: (a) photographs and tables are used to support
information report, (b) diagrams and photographs for explanation, and (c) diagrams and
tables for experimental account. This study provides empirical evidence to support the
importance of analysing multimodal genre for science teaching and learning as well as
science education research.

Keywords Genre · Multimodality · Representation · Textbook analysis · Visualisation

Introduction

Representations such as diagrams, graphs, equations and tables are central to science
meaning-making (Nielsen & Yeo, 2022). They are also a ubiquitous feature of science
curriculum materials, including textbooks. Research examining visual representations in
school science textbooks is currently growing (Vojíř & Rusek, 2019), largely as a result of
the increasing visual appeal and presence seen in contemporary textbooks (Slough et al.,
2010). However, one persistent issue with the analysis of representations in textbooks (and
other kinds of curricular text in general) is an isolated attention towards visual represen-
tations with few consideration to the main body of text that contextualises and expands
those representations (e.g. Liu & Treagust, 2013; Papageorgiou et al., 2019). Conversely,

 Kok-Sing Tang
kok-sing.tang@curtin.edu.au

1
1 2 Research in Science Education (2023) 53:357–
STEM Education Research Group, School of Education, Curtin University, GPO Box
U1987, Perth, WA 6845, Australia

1
Research in Science Education (2023) 53:357–375

there has been a long-established research focusing on the linguistic features in science
textbooks with little attention to the accompanying non-verbal representations (e.g. Bryce,
2013; Muspratt & Freebody, 2013).
There is currently a lack of systematic study examining the integration of verbal-visual
elements in science textbooks. Although there are some studies that explored the connec-
tion between printed texts and images, these are mostly limited to diagram labels, figure
captions and text references (e.g. Akcay et al., 2020; Khine & Liu, 2017), rather than a
thorough analysis of the main body in the written text. For instance, it is quite common for
researchers to apply the Graphical Analysis Protocol developed by Slough et al. (2010) to
categorise whether an image is indexed and captioned in the main body. But such analy-
sis is unable to shed light on why certain types of visual representations tend to be used
in some parts of the written text, while others are used elsewhere. It also does not reveal
insights into how a visual representation supports the written text, and vice versa.
Building on the separate research strands focusing on either visual representation or
textual analysis, the study in this paper aims to investigate the co-dependency between
representations and written text in Australian secondary science textbooks. Specifically, it
explores how different types of representation (e.g. photograph, diagram, table) vary
across the major genres of science. A genre is a cultural way of doing things with language
(Mar- tin, 2007). Every culture, including science, has developed unique genres that are
charac- terised by how language (including representations) is used to mediate a specific
purpose. As such, genre plays a major role in influencing how representations are used
within a culture (Bateman, 2008). However, this role has only been theorised and there has
not been many empirical evidence in science education that support the connection
between genre and representations. As such, this study seeks to investigate the connection
in science text- books in order to better understand how visual representations are used in
relation to scien- tific genres. The research questions that guide this study are:

1. How are the types of visual representation distributed across scientific genres, and
vice versa, in junior secondary science textbooks?
2. How do visual representations support the communication of scientific genres in
junior secondary science textbooks?

Literature Review and Conceptual Framing

Research in Science Textbook Analysis

Science textbook analysis is a major area of research around the world for many years
(Vojíř & Rusek, 2019). Despite the explosion of multimedia sources available for teach-
ers and students to use in formal and informal contexts, textbooks in both print and
digital formats remain a key education resource through which students learn scientific
knowledge. There is a widespread belief that textbooks represent an outdated pedagogy
associated with didactic and teacher-centred instruction. However, such a view conflates
the difference in using a textbook as a resource versus a pedagogical approach. A sur-
vey of science textbook usage in 486 Australian secondary schools found that 87% of
the schools used textbooks, with 55% reporting using them in most lessons (McDon-
ald, 2015). More importantly, the survey also revealed many respondents stressed
that textbook was one of many tools in their teaching repertoire to support a range of

1
 Research in Science Education (2023) 53:357– 3
instructional activities. Due to the pervasive use of science textbooks in a range of peda-
gogies, it is important to analyse how the written language and visual representations of
science are used in them.

Type and Distribution of Visual Representations in Science Textbooks

In terms of the use of visual representations in science textbooks, many researchers

have categorised various visuals (e.g. Liu & Treagust, 2013; Papageorgiou et al., 2019);
examined their image functions (e.g. Tang et al., 2019; Peterson et al., 2021; Slough
et al., 2010); and compared the usage of visuals in textbooks across the years (e.g.
Akcay et al., 2020), grade levels (e.g. Liu & Treagust, 2013) and publishers (e.g. Han
& Roth, 2006; Slough et al., 2010). Collectively, these studies have examined different
types of representation and showed how they are distributed or used in school texts.
A visual representation in such research is typically defined as a standalone graphic
that is embedded in the main text, sometimes appearing with a descriptive caption.
There are various classification schemes developed by different researchers to catego-
rise visual representations (e.g. LaDue et al., 2015; Lohse et al., 1994; Pozzer & Roth,
2003). Most classifications typically include photographs, diagrams, network charts,
graphs, tables and equations.
Photographs are realistic images taken by optical instruments and have a direct cor-
respondence between the real world and the image. They are often reported as the most
common type of visual representation in primary and secondary science textbooks.
According to several studies, the percentage of photographs out of all the visual rep-
resentations is 56% in Korean middle school chemistry textbooks (Han & Roth, 2006),
53% in North American high school biology textbooks (Pozzer & Roth, 2003) and 47%
in Texan middle school general science textbooks (Slough et al., 2010).
Diagrams are natural or schematic drawings that represent some aspects of a physi-
cal object or process (LaDue et al., 2015). Some researchers further divide this category
into iconic (or naturalistic) and schematic diagrams (e.g. Khine & Liu, 2017; Liu &
Treagust, 2013; Papageorgiou et al., 2019). An iconic diagram looks like what it rep-
resents as the drawn objects share similar spatial relations with the objects in the real
world, while schematic diagrams are symbolic or imaginary depictions that do not pre-
serve the physical relationships between the drawn and real objects. According to some
studies (e.g. Han & Roth, 2006; Slough et al., 2010), diagrams constitute the second
most common visual representations in science textbooks, at about 20%.
Researchers tend to classify visual representations along an abstraction continuum from
the highest contextual details (i.e. photograph) to more symbolic representations like sche-
matic diagrams (e.g. Pozzer & Roth, 2003). Going further up this continuum are network
charts and graphs. A network chart depicts abstract relationships among different visual
components, such as flow chart or decision tree (LaDue et al., 2015). Although they are
sometimes classified and labelled as diagrams (e.g. Venn diagram), network charts are dif-
ferent from diagrams because they represent ideas, processes or relationships without any
direct correspondence to a real physical object. A graph is another symbolic representa-
tion as it represents quantitative information along conventional scales or number systems.
Finally, tables and equations are also abstract entities often classified as a visual represen-
tation in textbook analysis, even though some researchers regard them as more numerical-
symbolic rather than a “visual” representation (e.g. Van Rooy & Chan, 2017).

1
 3 Research in Science Education (2023) 53:357–

Modal Affordance of Representations

Besides identifying the types and usage of visual representations, it is also crucial for
this study to explain why certain representations tend to be used for a particular purpose
more so than others. This is where the notion of modal affordance from the perspective
of social semiotics provides a useful conceptual framework (Kress, 2010). Modal affor-
dance refers to the potentialities and constraints that are available to different modes of
representation. According to Kress and van Leeuwen (2006, p.232), all representations
“carry a set of affordances from which sign-makers and interpreters select according to
their communicative needs and interests in a given context”.
For instance, in order to draw light as a wave, the indication of its wavelength, ampli-
tude, direction and spatial expanse is inherent in the diagram and as such unavoidable
in the act of drawing (Fredlund et al., 2021). Thus, a diagram commits users to make
certain spatial and quantitative relations (e.g. about wavelength) that are not obligatory
in other modes such as verbal language. By contrast, words in verbal language have
a modal affordance in making categorical relations, such as the distinctions between
“light” and “radiation”. Similarly, making a mathematical equation commits users to
make both categorical and quantitative relations. For example, every symbol in F = ma
represents a categorical attribute of a phenomenon, but at the same time, also carries a
numerical value that bears a quantitative relation with other values in the equation (see
Tang et al., 2011).

Lexicon and Grammatical Analysis in Science Textbooks

Given that textbook is still predominantly a written medium (unlike a graphic novel or
video), there is a lot of attention on the use of verbal language in mediating or hinder-
ing student comprehension of science textbooks. Decades of research has shown that
scientific language poses a lot of challenges for many students (Yore & Treagust, 2006).
As such, researchers have examined the linguistic and cognitive demands on students to
read and understand science concepts from secondary and tertiary textbooks (e.g. Fang,
2005; Muspratt & Freebody, 2013; Rusek & Vojí, 2019).
The challenge of scientific language can be understood at three different levels —
lexicon, grammar and genre. Research focusing on lexicon level tends to examine the
technical vocabulary introduced in textbooks or the number of words per sentence
(Bryce, 2013). Such research tends to use quantitative measurements, such as lexical
density (percentage of content words per sentence) or the Automated Readability Index
(computed based on number of letters per word and words per sentence), to assess the
level of reading difficulty in science textbooks (e.g. Fang, 2005; Hu et al., 2021; Ram-
narain, 2012).
As for research focusing on grammar, several studies have examined the unique gram-
matical patterns and features in science texts (e.g. Fang, 2005; Jalilifar et al., 2017). For
instance, nominalisation (Halliday, 1993) — the process of making a noun from a verb or
adjective (e.g. evaporate to evaporation) is a common grammatical feature in scientific
lan- guage that accounts for the abstractness of science texts (Fang, 2005). A study of
elemen- tary and middle school science textbooks in the USA by Mueller (2015) found
that nomi- nalisation is commonly used from the third grade (4.6% of the total words) and
increases in middle school textbooks (average of 5.5%).

1
Research in Science Education (2023) 53:357– 3
Scientific Genres and Genre Analysis in Science Textbooks

Besides lexicon (on words) and grammar (on clauses and sentences), the next level of ana-
lysing language focuses on genre. A genre is often associated with text structure and
organ- isation, but this concept involves more than that. Genre is defined as a staged and
goal-ori- ented social activity, which is reflected in how texts are structured according to
the purpose of that activity (Martin, 2007). Thus, genre is associated with the recurring
social practices used by people to produce, distribute and interpret texts within a particular
discourse com- munity. A genre is often organised according to its purpose as well as its
distinctive stages that involve stable and predictable patterns of language use (Martin,
2007).
Past research has identified four major genres that are reflected in scientific written
texts: experimental account, informational report, explanation and argument (Fang et al.,
2010). An experimental account comprises the procedural steps and results obtained in
conduct- ing an experiment. An information report comprises descriptive or taxonomic
informa- tion about things or events in the natural world. An explanation comprises the
underlying causes or processes of a phenomenon while an argument comprises a claim or
position together with supporting or disputing evidence.
Research has found that secondary school students tend to face difficulty in reading and
writing scientific genres because these genres differ from the familiar genres they grow up
reading at home, such as narrative and biography (Fang et al., 2010). In genre analysis,
some researchers have investigated the type and distribution of genres in science textbooks
(e.g. Dimopoulos et al., 2005; May et al., 2020). For instance, in a study of Greek primary
and lower secondary science textbooks, Dimopoulos et al. (2005) reported information
report and explanation genres comprised the majority of the content at 80.0% (the study
combined these two genres under “Reports”). Experimental account constituted 13.2%
while historical account constituted the remaining 6.8%. No study has yet investigated the
connection between written genres and visual representations.

Methods

Data Sample and Unit of Analysis

The data used in this study are taken from four junior secondary (Grades 7–10) science
textbook series called Pearson Science (2nd edition; see appendix for list of references).
These textbooks were selected because according to McDonald’s (2015) survey study,
Pearson Science is the most commonly used junior secondary science textbook in Aus-
tralia, at 17% of the schools surveyed. Most Year 7 to 10 science textbooks in Australia do
not differentiate according to sub-disciplines (e.g. biology, physics), in alignment with the
structure of the Australian Science Curriculum. However, Pearson Science textbooks indi-
cate the sub-disciplines in individual chapters. The distribution of chapters according to
the sub-disciplines of science and the year levels is shown in Table 1.
Besides topical chapters, the textbooks are also organised according to the three strands
in the Australian Science Curriculum: Science Understanding (SU), Science Inquiry Skills
(SIS) and Science as Human Endeavour (SHE). SU focuses on the content knowledge of
science and it occupies the bulk of the textbook content. SIS emphasises the planning,
conduct, evaluation and communication of scientific investigations. This strand is easily

1
 3 Research in Science Education (2023) 53:357–

Table 1 Number of chapters by sub-disciplines and levels in Pearson Science textbooks

Year 7 Year 8 Year 9 Year 10 Total

Biological sciences 2 3 3 4 12
Chemical sciences 2 2 2 2 8
Earth and space sciences (ESS) 2 1 1 2 6
Physical sciences 1 1 3 1 6
General topic (e.g. working scientifically) 1 1 1 1 4
Total 8 8 10 10 36

identifiable from a section in every chapter entitled “practical investigations”. SHE

empha- sises the social development of science as a human activity and its importance in
contem- porary applications. It is identifiable in the textbooks from two chapter sections
entitled “working with science” and “science as human endeavour”.
In this study, the unit of analysis is a visual representation with its accompanying writ-
ten text. A visual representation is identified and counted whenever there is a caption (e.g.
Figure 4.1.1, Table 4.1.1) in the main text. Visuals that appear without caption are
excluded from the analysis as they are typically used for cover page and other decorative
purposes. An exception of this rule is mathematical equations as they often appear without
a caption, but instead are embedded in a text. In this case, every equation embedded within
a para- graph or between two consecutive paragraphs is counted as a visual representation.
The total number of visual representations in the data is 1931, and their distributions
according to sub-disciplines, year levels and strands are shown in Tables 2 and 3.
The figures and tables in Pearson Science are always directly referenced in the main
text (e.g. “Fig. 8.3.12 shows…”). This referencing allows a connection to be made
between the visual representation and its accompanying written text. The boundaries of the
written text are typically identified from one header (or sub-header) to another, and the
text herein determines the genre associated with that representation. With these criteria,
this is how
Table 2 Number of visual representations by sub-disciplines and levels

Year 7 Year 8 Year 9 Year 10 Total

Biological sciences 134 173 159 224 690

Chemical sciences 110 102 134 144 490
Earth and space sciences (ESS) 128 52 53 104 337
Physical sciences 81 61 206 66 414
Total 453 388 552 538 1931

Table 3 Number of visual

SU SIS SHE Total
representations by sub-
disciplines and strands
Biological sciences 531 68 91 690
Chemical sciences 338 76 76 490
Earth and space sciences (ESS) 250 39 48 337
Physical sciences 298 72 44 414
Total 1417 255 259 1931

1
 Research in Science Education (2023) 53:357– 3
a multimodal text is identified in this study (see examples from Figs. 1, 2 and 3). Most
multimodal texts contain one visual representation (e.g. Figure 1), but it is also common
for some multimodal texts to contain two or more representations of the same or different
types (e.g. Figure 2).

Classification Schemes

Every multimodal text is analysed and tagged along three dimensions. First, it is tagged
along the following content attributes, namely grade level (Year 7, 8, 9, 10), strand (SU,
SIS, SHE), sub-discipline (biology, chemistry, ESS, physics) and topic (e.g. habitats, mix-
tures, forces). This information is directly provided in the textbooks.
Second, the type of visual representation is coded according to a coding scheme
adapted from LaDue et al. (2015). This scheme consists of photograph, diagram, network
chart, graph, map, table and equation, which are categories previously described in the
literature review. In addition, three new categories not observed in LaDue et al (2015)
study were added to the coding scheme. These are scientific image, mixed diagram-
photograph and mixed table-visual. A description of all these 10 types of representations
is summarised in Table 4.
Lastly, the genre of the multimodal text is coded as informational report, explanation,
experimental account, argument, technical procedure or story. Besides the four scientific
gen- res that were discussed earlier, technical procedure and story are new categories
added to the coding scheme. Technical procedure consists of a series of steps to instruct
readers on how to perform a task or procedure, such as drawing vectors or balancing a
chemical equation. Story

Table 4 Coding scheme for visual representations

Representation type Characteristics

Photograph A realistic image of an object captured through an optical camera

Diagram A hand-drawn or computer-generated graphic depicting physical aspects of
objects, e.g. shape, size, parts, relationships. Diagram can be iconic —
images look like what they represent, schematic — images are symbolic or
imaginary depictions (e.g. circuit symbol, field lines), or both
Network chart A graphic depicting abstract and qualitative relationships among different
visual components, which comprise geometric shapes, lines and words.
Examples: Flow chart, decision tree, mind map, Venn diagram
Graph A graphic depicting quantitative information through position and
magnitude of visual components. Number lines and coordinate systems are
typically used, but pie chart is also included here
Map A graphic depicting information layered on specific locations on Earth
Table An organised arrangement and display of words, numbers or symbols to
high- light set relationships among them
Equation A symbolic expression containing scientific notations and algebraic
symbols (e.g. =, +)
Scientific image A non-photorealistic image produced by specialised scientific equipment,
such as X-ray, scanning electron microscope (SEM) and scanning tunnelling
microscope (STM)
Mixed diagram-photograph A hybrid combination of diagram and photograph, either juxtaposing next to
or superimposing onto one another
Mixed table-visual A hybrid combination of visual (photograph or diagram) embedded in a table,
for example inside one of the cells

1
 3 Research in Science Education (2023) 53:357–

consists of recent news, historical events, and biographies of past and contemporary
scientists. See Table 5 for the functional and linguistic characteristics of these genres.

Analytical Procedures

Nvivo12® was used to collate, code and analyse the multimodal texts in the data. Two
analysts were involved in the processing and coding of data. The interrater reliability as
Table 5 Coding scheme for genres
Genre Characteristics

Information report Purpose: To organise information about things or events in the natural
world Stages: Identification, description, classification
Linguistic features:
• High usage of relational clauses (e.g. is, has, consist of, type of) centring on non-
human objects
• Sentences are often non-sequential
Explanation Purpose: To account for the underlying causes or processes of a known
phenom- enon
Stages: Phenomenon identification, sequence,
closure/generalisation Linguistic features:
• High usage of material clauses (e.g. move, block) centring on non-human objects
• High usage of external conjunctions (e.g. because, so, when)
• Sentences are sequential according to temporal or causal logic of events
Experimental account Purpose: To present the procedures and results of an experiment
Stages: Goal, procedure, results
Linguistic features:
• High usage of behavioural clauses centring on the reader or doer (e.g. set up the
apparatus, hold the magnet)
• High usage of numeric system or bullet points for sequences
• Sentences are sequential according to the procedural steps required to do the
experiment
Argument Purpose: To present evidence to support or refute a disputable claim
Stages: Thesis, evidence, discussion
Linguistic features:
• Lower degree of modality suggesting prediction or hypothesis (e.g. believe,
sug- gest, could be, propose) instead of objective and fact-like statements
• Human subjects are explicit in some sentences (e.g. scientists believe, Darwin
thought)
• Sentences are sequential according to temporal or causal logic of events
Technical procedure Purpose: To instruct readers on how to perform a task or procedure
Stages: Task, method
Linguistic features:
• High usage of behavioural clauses centring on the reader or doer (e.g. write,
balance)
• High usage of personal pronouns (e.g. you can, you will)
• Sentences are sequential according to the procedural steps required
Story Purpose: To narrate stories of recent news, recounts of historical events, and
biog- raphies of past and contemporary scientists
Stages: Orientation, record of events
Linguistic features:
• Human subjects and their names are explicit in most sentences (e.g. Rosalind
Franklin, a British scientist used her skills…)
• High usage of material verbs (e.g. investigate, perform) centring on humans
• Sentences are often sequential according to the chronological events

1
 Research in Science Education (2023) 53:357– 3
measured by Cohen’s kappa coefficient is 0.951 for the coding of visual representation
type and 0.831 for the coding of genre. The discrepancy for genre mainly centres around
argu- ment and story as the boundaries between these two genres are sometimes not easy
to dis- tinguish. Most of the arguments found in Pearson Science are based on historical
debate among scientists (steady state vs. Big Bang theory). Hence, such texts can be
mistaken as personal biographies of how a scientist determined those theories instead of
scientific argu- mentation based on evidence and contestation of claims.

Results

To address RQ 1, the following quantitative results show the number of visual representa-
tions distributed according to different types and genres (Table 6), the percentage of repre-
sentation types used within a particular genre (Table 7) and the nominalised percentage of
genres using a particular representation type (Table 8). To address RQ 2, these quantitative
results are expanded through a qualitative interpretation to illustrate how the visual repre-
sentations are used to support the major genres (see Figs. 1, 2 and 3).

Distribution of Visual Representations

Table 6 provides an overview of the numbers in every category. In terms of the total
number of visual representations, we can see that photographs (42.4%) and diagrams

Table 6 Number of visual representations across types and genres

Information Explanation Experi- Argument Technical Story Total

report mental procedure
account

Photograph 469 198 40 14 10 87 818

(42.4%)
Diagram 182 262 178 7 12 17 658
(34.1%)
Table 54 2 44 0 0 5 105
(5.4%)
Equation 38 34 4 0 20 0 96
(5.0%)
Mixed diagram- 33 49 1 3 1 4 91
photo (4.7%)
Network chart 28 9 10 1 0 1 49
(2.5%)
Scientific image 23 15 2 1 1 4 46
(2.4%)
Graph 14 6 4 4 2 0 30
(1.6%)
Map 10 9 1 5 0 2 27
(1.4%)
Mixed table- 9 0 0 1 0 1 11
visual (0.6%)
Total 860 584 284 36 46 121 1931
(44.5%)
(30.2%) (14.7%) (1.9% (2.4% (6.3%)
) )

1
 3 Research in Science Education (2023) 53:357–

(34.1%) are the two most common types used in the textbooks. This distribution is con-
sistent with the findings from previous studies (e.g. Han & Roth, 2006; Slough et al.,
2010). In terms of the total number of genres, information report and explanation are
the most common, followed by experimental account and story. Argument and technical
procedure are rare but they do appear several times in the textbooks. The combined per-
centage of information report and explanation (74.7%) and percentage of experimental
account (14.5%) are close to the findings from Dimopoulos et al. (2005). In terms of the
novel findings from this study (i.e. connection between representation and genre), every
cell in Table 6 shows the number of a representation type used in a particular genre.
Table 7 shows the percentage of representations by each genre. For information
report and story, it is clear that photographs are highly dominant as their usage far
exceeds (more than 2 times) the next common representation — diagram. In explana-
tion, diagram appears to be the most common, although it is also quite close to photo-
graph. Experimental account contains predominantly diagram, followed by photograph
and table. For argument, it is common to use photograph, followed by diagram, map and
graph. For technical procedure, the dominant representation is equation, largely because
the text often instructs students to manipulate a formula or balance a chemical equation.
It should be noted that as the number of argument and technical procedure genres in the
data are small (N = 36 and 46), the distribution for these two genres may not be general-
isable unlike the more common genres.
Table 8 shows a normalised distribution of each representation type across different
genres. The normalisation takes into account the uneven number where some genres (e.g.
information report) appear much more often than other genres in the textbooks. In effect,
Table 8 shows how likely a representation type would appear if the textbook contains the
same number for every genre.
From Table 8, we can see a number of interesting patterns. First, photographs are most
likely used to support story, followed by information report. Diagrams are most likely
used to support experimental account, followed by explanation. Tables are most likely
used in experimental account, followed by information report. Lastly, equation appears
most likely in technical procedure.

Table 7 Percentage of representation types used within a particular genre (bold denotes highest percentage
within a genre)
Informa-
Explanatio Experi- Argument Technical Story
n
tion report (N = 584) mental (N = 36) procedure (N = 121)
(N = 46)
(N = 860) account
(N =
284)
Photograph (N = 818) 55% 34% 14% 39% 22% 72%
Diagram (N = 658) 21% 45% 63% 19% 26% 14%
Table (N = 105) 6% 0% 15% 0% 0% 4%
Equation (N = 96) 4% 6% 1% 0% 43% 0%
Mixed diagram-photo (N = 91) 4% 8% 0% 8% 2% 3%
Network chart (N = 49) 3% 2% 4% 3% 0% 1%
Scientific image (N = 46) 3% 3% 1% 3% 2% 3%
Graph (N = 30) 2% 1% 1% 11% 4% 0%
Map (N = 27) 1% 2% 0% 14% 0% 2%
Mixed table-visual (N = 11) 1% 0% 0% 3% 0% 1%
Total 100% 100% 100% 100% 100% 100%

1
 Research in Science Education (2023) 53:357– 3
Table 8 (Normalised) percentage of genres using a particular representation type

Infor- Explanation Experi- Argument Technical Story Total

procedure
mation mental
account
report

Photograph (N = 818) 23% 14% 6% 17% 9% 31% 100%

Diagram (N = 658) 11% 24% 33% 10% 14% 7% 100%
Table (N = 105) 24% 1% 59% 0% 0% 16% 100%
Equation (N = 96) 8% 11% 3% 0% 79% 0% 100%
Mixed diagram-photo 15% 32% 1% 32% 8% 13% 100%
(N = 91)
Network chart (N = 49) 27% 13% 30% 23% 0% 7% 100%
Scientific image (N = 46) 19% 18% 5% 20% 15% 23% 100%
Graph (N = 30) 8% 5% 7% 57% 22% 0% 100%
Map (N = 27) 6% 8% 2% 75% 0% 9% 100%
Mixed table-visual (N = 11) 23% 0% 0% 60% 0% 18% 100%

Qualitative Interpretation and Explanation

To provide a more in-depth understanding of why and how different visual representations
are used in coordination with the written genres, a qualitative interpretation of three rep-
resentative multimodal texts is provided in this section. Due to space constraint, only the
genres of information report, explanation and experimental account, and their dominant
visual representations, are reported here.

Information Report (with Photograph and Table)

Figure 1 shows a typical information report with a photograph and table from Pearson
Science. The purpose of this text is to distinguish solute and solvent as well as highlight
different types of solutions. Information report often contains two types of semantic rela-
tionships: meronym (part of; composition) and hyponym (type of; classifying). Meronym
relationships are found in the beginning of the second paragraph with the use of possessive
(“has”) and composing (“is… in it”) relational clauses to list the solvents in a soft drink
solution. Hyponym relationships are found in the last sentence talking about “types of
solu- tions”. They are also implied in Table 4.1.1, which lists the four types of solutions
and their respective examples.
Based on the modal affordance of a table, the table’s layout commits the authors and
readers to make categorical meanings by systematically comparing across both the vertical
columns (various types) and horizontal rows (example of a particular type). As such, the
use of a table to classify different entities and describe the components, attributes or
factors of an entity can be seen as a common reason to support the dominance of
meronyms and hyponyms in information report, as observed not only in this example but
also most of the other tables found in the data.
As for the photograph, this was referenced in the second paragraph: “The soft drink
in Fig. 4.1.1 has sugars, preservatives and flavourings dissolved in water”. This ref-
erence provides an important clue towards the purpose of the photograph and how it

1
 3 Research in Science Education (2023) 53:357–

Fig. 1 A multimodal text in the genre of informational report (reproduced with permission from Greg Lin- ▸
stead, David Madden, Malcolm Parsons, Lana Salfinger, Maggie Spenceley, Christina Bliss, Louisa Len-
nard, Craig Tilley, Julie Williams, and Rebecca Wood, Pearson Science - Year 7: Student Book, 2nd edition,
©2017, Pearson Australia, pages 141 (text only). Figure 4.1.1 belongs to author’s original photograph)

supports the text. The photograph focuses on what are the solutes inside a bottle of soft
drink, although it is somewhat ironic that the solutes mentioned (sugars, preservatives,
flavourings) were not visible. However, what was visible is the carbon dioxide bubbles
that were described in the next sentence. By showing the bubbles (the parts) inside the
bottle (the carrier), we can see how the visual representation also complements the mer-
onym (part-carrier) relationship found in the written text.

Explanation (with Diagram and Photograph)

Figure 2 shows a typical explanation from Pearson Science that accounts for the causes
or processes of a known phenomenon, which is lunar eclipse in this text. In this particu-
lar multimodal text, there are two visual representations: a photograph with several layers
superimposed onto one another and a diagram that contains both iconic (Earth and Moon
resembling the real things) and schematic features (orbit line, shadow lines and shades).
This dual usage of photograph and diagram is sometimes combined into one visual rep-
resentation, as coded in the category of “mixed diagram-photograph” in the data. Not
surprisingly, an explanation genre has the highest number of mixed diagram-photographs
(N = 49), which is 54% of all the mixed diagram-photographs used in Pearson Science.
To understand why a photograph and diagram were used for Fig. 8.3.13 and Fig. 8.3.12,
respectively, we need to examine their functions in relation to their modal affordances. For
the photograph, its function is to enable the viewer to, as quoted in the textbook, “see what
the Moon looks like”. As photographs use optical imaging technology to reproduce the
things we can see, their modal affordance centres on realism to depict what a phenomenon
looks like in real life, as compared to other types of representation. This reason not only
explains why photographs are sometimes (34%) used in explanation, but also why
informa- tion report often (55%) uses photographs more than diagrams in order to show
what real objects actually look like (as was the case in Fig. 1).
As for the diagram, its function is evident from this statement in the text:
“Fig. 8.3.12 shows how this happens”. This “how” aspect underlies the causal and
temporal process of the phenomenon. A photograph is not ideal here as its modal
affordance does not give the author much control over what can be depicted. By con-
trast, the construction of a diagram commits the drawer to make certain choices about
the positions, sizes, shapes, colours, shades and other visual elements as an object
or phenomenon is drawn. Thus, the modal affordance of a diagram constraints the
authors and viewers towards specific spatial and quantitative relations during the meaning-
making process. In this particular example, Fig. 8.3.12 requires the size of and
distance between the Earth and Moon to be drawn in a disproportionate way in
order to highlight the effect of the shadow. The choices of the size and distance in
turn determine the Moon’s orbit and the Earth’s umbra and penumbra to be shown and
interpreted in the diagram.
In sum, for an explanation genre, the modal affordance of a photograph explains why
it is frequently used (at 34%) to support its phenomenon identification stage (the what),

1
Research in Science Education (2023) 53:357– 3

1
 3 Research in Science Education (2023) 53:357–

Fig. 2 A multimodal text in the genre of explanation (reproduced with permission from Greg Linstead,
David Madden, Malcolm Parsons, Lana Salfinger, Maggie Spenceley, Christina Bliss, Louisa Lennard,
Craig Tilley, Julie Williams, and Rebecca Wood, Pearson Science - Year 7: Student Book, 2nd edition, ©
2017, Pearson Australia, pages 368 (text only). Figures 8.3.12 and 8.3.13 belong to author’s original dia-
gram and photograph)

while the affordance of a diagram explains why it is frequently used (at 45%) to support
its sequences stage (the how or why).

Experimental Account (with Diagram and Table)

Figure 3 shows a typical multimodal text appearing in an experiment account genre. In

Pearson Science, the format of such text is highly standardised according to the organisa-
tional headings shown in Fig. 3 (e.g. purpose, materials, procedure, results).
A high percentage (63%) of such texts use an iconic diagram with labels to show
and indicate all the different parts of the apparatus needed for the experiment. Such
visual is needed to show the physical connections and spatial arrangements that can-
not be adequately described in the written text. A diagram is often preferred over
a photograph for its modal affordance in two aspects. The first aspect is related to
modality, which is the credibility or truth value of a statement or representation. In
scientific discourse, the modality of an image depends on the universality and objec-
tiveness of a representation, rather than its realism (Kress & van Leeuwen, 2006).
Thus, diagrams and graphs have a higher modality in science because they are deemed
to be applicable and universal across time and place. By contrast, a photograph of an
experimental setup has lower modality because the experiment would be seen as lim-
ited to a localised laboratory. The second aspect is due to its pedagogical simplicity.

1
 Research in Science Education (2023) 53:357– 3

Fig. 3 A multimodal text in the genre of experimental account (reproduced with permission from Greg
Rickard et al. author, Pearson Science – Year 9: Student Book 2 ©2017, Pearson Australia, page 262, fig-
ure 6.4.17)

As an author has more control over the construction of a diagram compared to a pho-
tograph, they can highlight the different parts of the apparatus more clearly with a
diagram.
Experimental accounts also frequently use a table to systematically record and
organise the collected data in accordance to the modal affordance of table, as discussed
earlier. Such data can include qualitative observation or quantitative numbers. While
the percentage of table used within an experimental account is not very high (at 15%)
compared to diagram, it is noteworthy to point out experimental accounts take up 59%

1
 3 Research in Science Education (2023) 53:357–

of all usage of table in terms of normalised percentage (see Table 8) in Pearson Science
textbooks.

Discussion

This study aims to understand the co-dependency between visual representations and writ-
ten genres by examining a corpus of multimodal texts (N = 1931) from a major Australian
textbook series. The analysis reveals a number of interesting patterns in terms of how sev-
eral types of representations tend to appear with certain genres, as well as possible reasons
for this co-dependency. In particular, the analysis accounts for why and how: (a) photo-
graphs and tables are used to support information report, (b) diagrams and photographs for
explanation, and (c) diagrams and tables for experimental account.

Limitations

Due to the labour-intensive nature of textbook analysis, this study only examined one pub-
lisher for the series of four science textbooks from Years 7 to 10. The quantitative results
must thus be interpreted with this limitation in mind. Nevertheless, the purpose of this
study is to examine the variation of representations across genres instead of publishers. As
genres are fairly stable within a well-established culture (Martin, 2007), it is quite unlikely
the distribution will be very different across publishers. Furthermore, most textbooks are
written by a multi-authored team comprising experienced science educators contracted by
the publisher. The language and representation choices in the textbooks are mostly made
by the authors (who are familiar with scientific genres) instead of the publishers. As such,
it is more accurate to report that this study focuses on textbooks produced by a dozen of
science educators (see appendix for list of authors) instead of just Pearson Science.
Another limitation in this study, and of text analysis in general, is the danger of ascrib-
ing too much meaning on printed texts alone. We must remember that representations and
genres are always dynamic despite their static appearances in textbooks. The word “repre-
sentation” is not just a thing or artefact, but it is also a nominalised noun from the action
verb represent (Tang, 2013). Thus, representation is more accurately a process of repre-
senting something by someone for someone else (c.f., Peirce, 1986). Research on repre-
sentation therefore needs to also consider the users (e.g. teachers, students) by analysing
their discourse and meaning-making with representations in situ (e.g. Tang, 2013; Tang
et al., 2014; Tang, 2020), and not just the representations per se. Nevertheless, what is
written and represented in the textbooks have a huge influence on students’ and teachers’
science meaning-making in the classroom. Future research can build on the findings from
this study to investigate how classroom participants interpret or construct the dominant
representations as they read or write specific scientific genres.

Significance and Implications

This study supports the importance of examining multimodal genre in science educa-
tion (Tang et al., 2022). Multimodal genre is a construct that combines the research work
from multimodality (the study of multiple modes of representation) and genre (the study
of recurring social practice). Multimodal genre analysis an emerging area of research in

1
 Research in Science Education (2023) 53:357– 3
linguistic and media studies (Hiippala, 2014), but it is currently not widely known and
applied in science education. A key argument in multimodal genre analysis is that repre-
sentations and genres are intrinsically intertwined. Thus, any analysis cannot examine one
aspect independently from the other. As highlighted in this study based on a large corpus
of multimodal texts, there is a recurring pattern of using certain representations for certain
genres. This provides strong empirical evidence to support the importance of analysing
multimodal genre in science teaching and learning.
The role of multimodal genre has implications for future research in textbook analy-
sis. While there is an active research focusing on visual representations in school science
textbooks (Vojíř & Rusek, 2019), there is a need to examine how these representations
are used in conjunction with the written genres. Such verbal-visual analysis needs to go
beyond looking at the captions or labels in a diagram, as has been done in previous studies
(e.g. Akcay et al., 2020; Slough et al., 2010), and examine extensively how the written
lan- guage in the main body and the visual representations are integrated.
This study also has several implications for science teaching. While science teachers
use a range of representations to support their teaching, they seldom explicitly highlight
why those representations are used or when to use them (Prain & Waldrip, 2006). Most
teachers also do not explain to students the form and function of scientific genres even
though they speak, read and write those genres in the classroom (Tang, 2020). There is
thus a need to point out the multimodal genre patterns that are implicit in the recurring
use of visual representations for certain genres. For instance, students need to understand
how to interpret the photographs that are embedded in information report, explanation
and story by knowing their respective purpose in those genres. They also need to dis-
tinguish how to use diagrams to support the sequencing stages of an explanation versus
the procedural setup of an experiment or investigation. More research building on the
initial investigation in this study to explore the various combinations of visual repre-
sentations and genres will shed more light into the use of representations for science
meaning-making.

Appendix – Science textbooks from Pearson Science used in the study

Linstead, G., Madden, D., Parsons, M., Rickard, G., Salfinger, L., Spenceley, M., Bliss,
C., Lennard, L., Tilley, C., Willams, J., & Wood, R. (2019). Pearson Science 7 Student
Book. Pearson.
Clarke, W., Devlin, J., Linstead, G., Parsons, M., & Spenceley, M. (2019). Pearson Sci-
ence 8 Student Book. Pearson.
Rickard, G., Clarke, W., Devlin, J., Linstead, G., Madden, D., Parsons, M., Salfinger,
L., Spenceley, M., & Moore, T. (2019). Pearson Science 9 Student Book. Pearson.
Rickard, G., Clarke, W., Devlin, J., Linstead, G., Madden, D., Maher, F., Salfinger, L.,
Spenceley, M., Bliss, C., Hogendoorn, B., Lennard, L., Tilley, C., & Wood, R. (2019).
Pearson Science 10 Student Book. Pearson.

Acknowledgements I wish to thank Isabelle Tang for her assistance in the data analysis. I also wish to
acknowledge and thank Pearson for granting free digital access to Pearson Science (Year 7–10, 2nd
edition) as well as the permission to use the excerpts and graphics included in this article. Nonetheless,
there is no conflict of interest in this independent study and I remain solely responsible for the views and
statements made herein.

1
 3 Research in Science Education (2023) 53:357–

Declarations
Competing Interests The author declares no competing interests.

References
Akcay, H., Kapici, H. O., & Akcay, B. (2020). Analysis of the representations in Turkish middle school sci-
ence textbooks from 2002 to 2017. Participatory Educational Research, 7(3), 192.
Bateman, J. (2008). Multimodality and genre: A foundation for the systematic analysis of multimodal docu-
ments. Palgrave Macmillan. https://link.springer.com/book/10.1057/9780230582323
Bryce, N. (2013). Textual features and language demands of primary grade science textbooks: The call for
more informational texts in primary grades. In M. Khine (Ed.), Critical analysis of science textbooks
(pp. 101–120). Springer.
Dimopoulos, K., Koulaidis, V., & Sklaveniti, S. (2005). Towards a framework of socio-linguistic analysis
of science textbooks: The Greek case. Research in Science Education (australasian Science Education
Research Association), 35(2), 173–195. https://doi.org/10.1007/s11165-004-8162-z
Fang, Z. (2005). Scientific literacy: A systemic functional linguistics perspective. Science Education, 89(2),
335–347.
Fang, Z., Lamme, L. L., & Pringle, R. M. (2010). Language and literacy in inquiry-based science class-
rooms, grades 3-8. Corwin Press.
Fredlund, T., Beate Remmen, K., & Knain, E. (2021). The epistemological commitments of modes: Oppor-
tunities and challenges for science learning. Visual Communication. https://doi.org/10.1177/14703
572211038991
Halliday, M. A. K. (1993). On the language of physical science. In M. A. K. Halliday & J. R. Martin (Eds.),
Writing science: Literacy and discursive power (pp. 54–68). University of Pittsburgh Press.
Halliday, M. A. K., & Matthiessen, C. M. I. M. (2004). An introduction to functional grammar (3rd ed.).
Arnold.
Han, J., & Roth, W. M. (2006). Chemical inscriptions in Korean textbooks: Semiotics of macro- and
microworld. Science Education, 90(2), 173–201. https://doi.org/10.1002/sce.20091
Hiippala, T. (2014). Multimodal genre analysis. In S. Norris & C. D. Maier (Eds.), Interactions, images
and texts: A reader in multimodality (pp. 111–125). De Gruyter.
Hu, J., Gao, X., & Qiu, X. (2021). Lexical coverage and readability of science textbooks for English-
medium instruction secondary schools in Hong Kong. SAGE Open, 11(1), 21582440211001868.
https://doi.org/10.1177/21582440211001867
Jalilifar, A., White, P., & Malekizadeh, N. (2017). Exploring nominalization in scientific textbooks: A
cross-disciplinary study of hard and soft sciences. International Journal of English Studies, 17(2), 1–
20. https://doi.org/10.6018/ijes/2017/2/272781
Khine, M., & Liu, Y. (2017). Descriptive analysis of the graphic representations of science textbooks.
European journal of STEM education, 2(3). https://doi.org/10.20897/ejsteme/81285
Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication.
Routledge.
Kress, G., & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd edition ed.).
Routledge.
LaDue, N. D., Libarkin, J. C., & Thomas, S. R. (2015). Visual representations on high school biology,
chemistry, earth science, and physics assessments. Journal of Science Education and Technology,
24(6), 818–834. https://doi.org/10.1007/s10956-015-9566-4
Liu, Y., & Treagust, D. (2013). Content analysis of diagrams in secondary school science textbooks. In
M. S. Khine (Ed.), Critical Analysis of Science Textbooks (pp. 287–300). Springer.
Lohse, G. L., Biolsi, K., Walker, N., & Rueter, H. H. (1994). A classification of visual representations.
Communications of the ACM, 37(12), 36–49. https://doi.org/10.1145/198366.198376
Martin, J. R. (2007). Genre, ideology and intertextuality: A systemic functional perspective. Linguistics
and the Human Sciences, 2(2), 275.
May, L., Crisp, T., Bingham, G. E., Schwartz, R. S., Pickens, M. T., & Woodbridge, K. (2020). The
durable, dynamic nature of genre and science: A purpose-driven typology of science trade books.
Reading Research Quarterly, 55(3), 399–418. https://doi.org/10.1002/rrq.274

1
 Research in Science Education (2023) 53:357– 3
McDonald, C. V. (2015). Evaluating junior secondary science textbook usage in Australian schools.
Research in Science Education (australasian Science Education Research Association), 46(4), 481–
509. https://doi.org/10.1007/s11165-015-9468-8
Mueller, B. M. (2015). Analysis of nominalization in elementary and middle school science textbooks.
School of Education Student Capstone Theses and Dissertations. 247. https://digitalcommons.
hamline.edu/hse_all/247
Muspratt, S., & Freebody, P. (2013). Understanding the disciplines of science: Analysing the language of
science textbooks. In M. S. Khine (Ed.), Critical analysis of science textbooks: Evaluating instruc-
tional effectiveness (pp. 33–59). Springer. https://doi.org/10.1007/978-94-007-4168-3_3
Nielsen, W., & Yeo, J. (2022). Introduction to the special issue: Multimodal meaning-making in science.
Research in Science Education. https://doi.org/10.1007/s11165-022-10051-z
Papageorgiou, G., Amariotakis, V., & Spiliotopoulou, V. (2019). Developing a taxonomy for visual rep-
resentation characteristics of submicroscopic particles in chemistry textbooks. Science Education
International, 30(3), 181.
Peirce, C. S. (1986). Writings of Charles S. Peirce: A chronological edition. Indiana University Press.
Peterson, M., Delgado, C., Tang, K.-S., Bordas, C., & Norville, K. (2021). A taxonomy of cognitive
image functions for science curriculum materials: Identifying and creating ‘performative’ visual
displays. International Journal of Science Education, 43(2), 314–343. https://doi.org/10.1080/
09500693.2020.1868609
Pozzer, L., & Roth, W.-M. (2003). Prevalence, function, and structure of photographs in high school
biology textbooks. Journal of Research in Science Teaching, 40(10), 1089–1114. https://doi.org/10.
1002/tea.10122
Prain, V., & Waldrip, B. (2006). An exploratory study of teachers’ and students’ use of multi-modal
representations of concepts in primary science. International Journal of Science Education, 28(15),
1843–1866.
Ramnarain, U. (2012). The readability of a high stakes Physics examination paper. Acta Academica,
44(2), 110–129. https://doi.org/10.10520/EJC125960
Rusek, M., & Vojí, K. (2019). Analysis of text difficulty in lower-secondary chemistry textbooks. Chem-
istry Education Research and Practice, 2(1), 85–94. https://doi.org/10.1039/c8rp00141c
Slough, S. W., McTigue, E. M., Kim, S., & Jennings, S. K. (2010). Science textbooks’ use of graphical
representation: A descriptive analysis of four sixth grade science texts. Reading Psychology, 31(3),
301–325. https://doi.org/10.1080/02702710903256502
Tang, K. S. (2013). Out-of-school media representations of science and technology and their relevance
for engineering learning. Journal of Engineering Education, 101(1), 51–76. https://doi.org/10.
1002/jee.20007
Tang, K. S. (2020). Discourse strategies for science teaching & learning: Research and practice.
Routledge. https://doi.org/10.4324/9780429352171
Tang, K. S., Tan, S. C., & Yeo, J. (2011). Students’ multimodal construction of work-energy concept.
Inter- national Journal of Science Education, 33, 1775–1804.
https://doi.org/10.1080/09500693.2010.508899
Tang, K. S., Delgado, C. & Moje, E. B. (2014). An integrative framework for the analysis of multiple and
multimodal representations for meaning-making in science education. Science Education, 98,(2), 305–
326. https://doi.org/10.1002/sce.21099
Tang, K.S., Won, M., & Treagust, D. F. (2019). Analytical framework for student-generated drawings.
Inter- national Journal of Science Education, 41(16), 2296–
2322.https://doi.org/10.1080/09500693.2019. 1672906
Tang, K. S., Park, J. & Chang, J. (2022) Multimodal genre of science classroom discourse: Mutual
contextu- alization between genre and representation construction. Research in Science Education, 52,
755–772. https://doi.org/10.1007/s11165-021-09999-1
Van Rooy, W. S., & Chan, E. (2017). Multimodal representations in senior biology assessments: A case
study of NSW Australia. International Journal of Science and Mathematics Education, 15(7), 1237–
1256. https://doi.org/10.1007/s10763-016-9741-y
Vojíř, K., & Rusek, M. (2019). Science education textbook research trends: A systematic literature review.
International Journal of Science Education, 41(11), 1496–1516. https://doi.org/10.1080/09500693.
2019.1613584
Yore, L. D., & Treagust, D. F. (2006). Current realities and future possibilities: Language and science lit-
eracy - Empowering research and informing instruction. International Journal of Science Education,
28(2), 291–314.

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.

1
Research in Science Education is a copyright of Springer, 2023. All Rights Reserved.