Multimodal Discourse Analysis

JOURNAL OF ADVANCES IN LINGUISTICS Vol 10 (2019) ISSN: 2348-3024 https://rajpub.com/index.
php/jal
DOI: https://doi.org/10.24297/jal.v10i0.8196
A Multimodal Discourse Analysis of the Interactive Meaning in Public Service Advertisement
Shuting Liu
School of Interpreting and Translation Studies, Guangdong University of Foreign Studies,
No.2 Baiyun Dadao Bei, Baiyun District, Guangzhou, China
liushuting0124@qq.com
Abstract
On the basis of Kress and van Leeuwen’s Visual Grammar based on Halliday’s Systemic Functional Linguistics,
this study explores the interactive meaning in three public service advertisement multimodal discourses, adding
evidence to the assumption that Systemic Functional Linguistics can be applied to the multimodal discourse
analysis of public service advertisement in a feasible and operational manner.
Keywords: Multimodal Discourse Analysis, Systemic Functional Linguistics, Visual Grammar, Interactive
Meaning, Public Service Advertisement
1 Introduction
Nowadays, advertisements are increasingly penetrating into people's daily life, playing an increasingly important
role. Whether through traditional media tools, such as newspapers, books, letters, or modern media tools, such
as television, computers, mobile phones or other network terminals, advertisements can be found anywhere at
any time, as if advertising has become one of the indispensable elements of human society. Generally speaking,
advertisement can be divided into two categories, namely, commercial advertisement and public service
advertisement (PSA). The criteria for classifying the two types of advertisements are also quite simple, that is,
whether they are profit-oriented or not. From research in recent years, it can be seen that experts and scholars
on advertising intend to set their research focus on commercial advertising, while research on PSA is
comparatively less. In addition, research perspectives need to be expanded in an all-round way, and research
connotations need to be further explored.
With information technology rapidly developing, visual language becomes more and more important in all
aspects, especially in advertising. The multimodal feature of advertising is an evidence of its use of various
semiotic resources, such as language, image, sound, and color, to better convey the connotation and obtain the
best advertising effect. Multimodal discourses exist widely and experts have carried out a lot of research on the
interpretation of multimodal discourses. Among these studies, multimodal discourse analysis (MDA) has taken
a place, taking Systemic Functional Linguistics (SFL) as its theoretical basis and providing new research ideas
and perspectives for discourse analysis. As a new research method of discourse analysis, MDA has become a
research focus of western scholars in recent years, especially in the fields of linguistics and social semiotics. Kress
and van Leeuwen [1, 2], Lemke [3], Royce [4], O 'Halloran [5, 6] and Baldry [7] all made positive and significant
contributions to MDA. Chinese linguistics Li [8], Hu [9] and Zhu [10] introduced the theory of MDA. Nowadays,
with the gradual development of MDA, more and more researchers are focusing on this field. However, there is
a lack of MDA focusing on PSA, which leaves a lot of gaps to be filled.
2 SFG, VG, MDA and PSA
In 1990s, Halliday’s SFL theory started to be applied in the study of MDA. Among the scholars who applied SFL
to MDA, Kress and van Leeuwen [1] put forward a theory that becomes prevailing in the field - they proposed
1523
JOURNAL OF ADVANCES IN LINGUISTICS Vol 10 (2019) ISSN: 2348-3024 https://rajpub.com/index.php/jal
Visual Grammar (VG) to analyze visual images. On the other hand, studies on discourse analysis starts from
1950s attracting wide attention from scholars and researchers.
2.1 Systemic Functional Grammar (SFG)
As mentioned previously, much research conducted in the field of MDA are based on the theory of social
semiotics, which in its narrow sense, refers to the Systemic Functional Grammar (SFG) of Halliday. The tenet of
SFG is to regard language as a kind of social semiotic and study language from the perspective of its interaction
with the social context. According to Eggins [11], SFG “has been applied to interpreting the grammar of other
semiotic modes, such as visuals [1, 2], sound [12] and action [13].
As Halliday [14] indicates, “In a functional grammar, a language is interpreted as a system of meanings,
accompanied by forms through which the meanings can be realized”. Therefore it can be said that SFG focuses
on the means of realizing the functions of language. According to Halliday’s framework, in language structure
and human communication there are three meta-functions, namely, ideational function, interpersonal function
and textual function.
1) Ideational Function: Language is used for describing, reflecting or interpreting the world around us, the
individual experience of ourselves, consisting of “goings-on, happening, doing, sensing, meaning, and being
and becoming” [14].
2) Interpersonal Function: Language is used for having communication or interaction with people, establishing
and maintaining relations with them, influencing other people’s behavior, expressing viewpoints of our own
on the things in the world, and eliciting or changing other people’s points of view. In this way, language is
used to enable people to participate in communications, take own their roles, and express their own and
understand others’ feelings, attitude and judgements.
3) Textual Function: It refers to the fact that language itself has a mechanism to turn a random list of sentence
into a living passage by organizing any stretch of written or spoken discourse into a coherent and unified
text [14]. Therefore, in textual function, language is used to relate what is said or written to the real world
or other linguistic events.
2.2 Visual Grammar (VG)
Inspired by Halliday’s ideas of social semiotics, Kress and van Leeuwen proposed and developed a method of
social semiotic analysis of visual communication and established a descriptive framework of multimodality. This
theory of Kress and van Leeuwen carried out for reading images is called Visual Grammar (VG). In light of
Halliday’s three meta-functions of language - ideational, interpersonal and textual meanings, they put forward
three meta-functions of images – representational, interactive and compositional meanings. The corresponding
relation between the three meta-functions in SFL and VG is shown in Table 1[15].
SFL VG
Ideational Representational
Interpersonal Interactive
Textual Compositional
Table 1 Three strands of meta-functions in SFL and VG
1) Representational Meaning: “Any semiotic system has to be able to represent, in a referential or pseudo-
referential sense, aspects of the experiential world outside its particular system of signs” [1]. That is to say,
1524
the semiotic system has to be able to represent objects and their relations in the world outside the
representational system. According to SFL, language “represents and constructs our perception of reality in
the form of ‘goings-on’ or processes of various types, which incorporate different categories of participants
(e.g. actors, goals, receivers, sensors, attributes)” [15].Adapting this model to the analysis of images, Kress
and van Leeuwen [1] divide representational meaning into two main categories of processes: narrative and
conceptual.
2) Compositional Meaning: “Any semiotic system has to have the capacity to form texts, complexes of signs
which cohere both internally and with the context in and for which they were produced” [1]. The visual and
verbal elements in multimodal discourses form a compositional whole. When expressing the textual meta-
function, compositional resources would appear to be: (1) horizontal structure when presenting visual
information as Given or New and (2) vertical structure when presenting visual information as Ideal and Real
[1].
3) Interactive Meaning: “Any semiotic system has to be able to project the relations between the producer of
a sign or complex sign, and the receiver/ reproducer of that sign” [1]. That is to say, any semiotic system has
to be able to project a particular social relation between the producer, the viewer and the object represented.
According to Kress & van Leeuwen [1], there are two kinds of participants involved in an image, namely,
represented participants (the people, the places and things depicted in images) and interactive participants (the
people who communicate with each other through images, namely the producer and the viewers). Between
these participants, there are three kinds of relations. First, relations between represented participants; Second,
relations between interactive and represented participants, which refers to the interactive participants’ attitudes
towards the represented participants; And third, relations between interactive participants, which refers to the
things interactive participants do to or for each other through images.
Interaction happens whenever the producers who produce the images and the viewers who appreciate the
images expressed their own ideas on the ways of images interpretation. Some of this interaction is direct and
immediate where the two parties have conversations face to face. While other interactions may be neither direct
nor immediate due to the absence of both sides. When examining an interaction, four key factors shall be taken
into careful consideration: visual contact, social distance between the represented participants and the viewers,
perspective and modality accounting for the degree of factuality and truthfulness of visual images with regard
to the real world [1].
2.3 Multimodal Discourse Analysis (MDA)
According to Xin [16], there are many well-known researchers studying MDA: O'Toole, Kress, van Leeuwen,
Lemke, O'Halloran, Thibault and Ventola who studies MDA from the perspective of social semiotics; Scolion,
Norris and Jewitt from the perspective of interactive sociology; Forceville and Holsanova from the perspective
of cognitive. Kress and van Leeuwen's [1] framework documents the system of meaning for the ideational,
interpersonal and textual which are respectively labeled as representational, interactive and compositional
in visual communication. Reading Images explores how figures, places and things in the image compose the
visual design to realize the compositional, interactive and representative meanings. As cited from Li (2003), many
multimodal discourse analysts made great contributions to MDA. For example, O'Toole [17] analyzes images
from the viewpoint of stratification; Lemke [3] explores the interrelations between the graphics and the verbiage
in scientific articles. Royce [4] explores the image-verbiage relations in advertisements. In analyzing three-
dimensional discourses, a few analysts have done tentative studies. For example, Martinec [13] makes a
multimodal study of body movements and the space design of the play.
In comparison with the prosperous studies on MDA overseas, China has just witnessed a budding stage in recent
years. Several Chinese scholars have realized the importance of multimodality. A series of introductory papers
to multimodality [8, 9, 10, 16, 18, 19, 20] and studies which apply the theories of MDA in specific types of
discourses [21, 22, 23, 24] have been done. Li [8] is the first to introduce the theory of MDA and reviews Kress
1525
and van Leeuwen's VG in her article. Afterwards Hu [9], Zhu [10], Xin [16] and Zhang [20] propose their opinions
on MDA, systemically introducing the theories of MDA and outlining the research status, the limitations and the
future of MDA. As for the application of the theory, plenty of research has been done. Hu and Dong [25] makes
a multimodal analysis of PPT; Ding [26] probes the modality in the illustrations in fairy tales. Several researchers
such as Li [27], Wang [23] and Hu [9] make a multimodal analysis of print advertisements and they mainly focus
on the interaction between the image and the verbiage.
2.4 Public Service Advertisement (PSA)
Public Service Advertisement is to diffuse beneficial social values to the public by using diverse artistic
demonstrational styles which can easily attract audience’s attention, affecting their views and behavior towards
the society and get a better solution to social problems. There are plentiful studies on advertising discourse in
social context in the past few decades. Leech [28] first studied advertising discourses from the view of stylistics.
Cook [29] put an emphasis of advertising discourse on the social implication of advertising language. One of
the Chinese frontiers, Huang [30], applies Systemic-Functional Grammar in advertising analysis and gives a
description of advertising from some aspects. However, they lack systemic analysis of inter-semiotic grammatical
connections in PSA. Therefore, this thesis chooses PSA as data to explore the interactive meaning by using MDA.
3 Methodology
The purpose of this study is to analyze how the multimodal discourses of PSA interact and exert the effects.
Thus, the present study attempts to provide an answer to the following questions:
Q1: How is the interactive meaning constructed in visual mode?
Q2: How is interactive meaning facilitated in verbal mode?
Q3: How are the verbal and visual modes combined to construct the interactive meaning?
In terms of data collection, three PSA examples are chosen for example analysis to have their multimodal
discourse analyzed in this study. The author mainly choose the examples for analysis from the Internet by
browsing through websites entitled “world’s creative public service advertisement” and alike. These examples
are respectively issued by World Wildlife Fund (WWF), goodparent.pl, and Cancer Patient Aid Association.
This study will analyze the visual and verbal messages in the PSAs respectively and then study the interaction
relations and effects of the verbal and visual modes. Data analysis is composed of three sections. The first section
is concerned with the interactive meaning in terms of the four elements of VG, namely, visual contact, social
distance, perspective and modality; the second section analyzes how the interactive meaning is constructed in
verbal language with respect to mood system and attitude system. The last part explores how the visual modes
and the verbal modes cooperate to construct the interactive meaning in PSA multimodal discourse.
4 Discussions
In this section, three PSA examples will be analyzed respectively in detail in order to inquire how various
modalities work together to realize the interactive meaning and to persuade the public in advertising discourses.
4.1 Analysis on Example 1 “Everyone Pays”
1526
Figure 1 “Everyone Pays”
(Source: http://www.ad-cn.net/read/6524.html)
4.1.1 Analysis on the Interactive Meaning in the Visual Image
(1) Visual contact
In terms of visual contact, this image can be counted as both a “demand image” and an “offer image” because
that some of the represented, the two big polar bear, look directly at the viewers, while others, the two small
polar bear and the penguin, do not look directly at the viewers. On one hand, “the participant’s gaze demands
something from the viewers, demands that the viewers enter into some kind of imaginary relation with him or
her” [1]. That is to say, the represented participant wants the viewers to take some action or to establish some
kind of social rapport with the viewers. In this case, the represented, polar bear and penguin standing on isolated
ice, want the viewers, human, to connect the light turning on with the severe conditions of their habitats and
then come up with solutions to ease the issue to protect these endangered animals. On the other hand, the
small polar bear and penguin do not look directly at the viewers but gaze somewhere else. According to Kress
and van Leeuwen [1], this kind of image “offers the represented participants to the viewers as items of
information, objects of contemplation, impersonally, as though they specimens in a display case”. In this case,
the represented, as objects of contemplation, provides information to the viewers. Viewers should make use of
this information, which is that the earth’s climate is undergoing severe changes, and do some thinking on taking
measures to save the animals.
(2) Social distance
The represented in this image could be divided into two parts, namely, the light and the picture with animals on
the wall. In terms of the light, a long shot is employed with the object shown out of the viewer’s reach with much
background information, which suggests a far social distance; while in terms of the animals, a very long shot is
employed with a total of eight torsos shown to the viewers, which entails a public distance. Both the long shot
and the very long shot indicate a rather far distance, which gives the viewers a feeling of objectivity. That is to
say, the viewers are more easily convinced of the truthfulness of the advertisement and thus earnestly take
actions to solve the objective problem.
1527
(3) Perspective
For the horizontal dimension, a frontal angel is adopted to depict both the picture and the light. With a frontal
angle, the image transmits involvement, indicating that climate issue and earth-protecting mission are world-
wide and in need of every one’s efforts and contributions. For the vertical angle, an eye-level angel is chosen to
transmit equality between the represented and the viewers, showing that the viewers are all in the same boat
when facing the issue shown in the image.
(3) Modality
The degree of modality of an image is assessed from the following aspects: color, contextualization,
representation, depth, illumination and brightness. This advertisement is truthful and reliable for the reason that
it has high modality. For contextualization and representation, much details of the background as well as the
represented participants are depicted, for instance, the wallpaper pattern on the wall, the small pieces of ice, the
frame of the picture and the posture of the penguin. For color, the image has fully saturated colors, several
colors and different shades of colors. In addition, there are also the play of light and shade as well as different
degrees of brightness, especially areas around the lighting lamp.
4.1.2 Analysis on the Interactive Meaning in the Verbal Language
The verbal components identified in this PSA mainly include this one sentence on the upper side of the image:
(1)“When you leave the light on, you are not the only one who pays” with the words “light on” in bold. The
words (2)“panda.org/climate” on the right lower corner and the sign beside indicate the institution issuing this
advertisement which is “WWF (World Wildlife Fund)” and its website. The verbal text (1) is a declarative sentence
to give a moderate description on the consequences of not saving the energy, which is to make climate warming
worse and leave some animals in an endangered situation. What is particular about this sentence is that it
intends to convey the idea in an implicit but not explicit way, leaving spacious room for contemplation to the
readers. The latter part of the sentence “you are not the only one who pays” intrigues the readers to reflect on
the question “who else will pay” and may come up with answers themselves.
4.1.3 Analysis on the Visual-Verbal Inter-Semiotic Interaction
The visual and the verbal text of the advertisement cooperate to construct the interactive meaning of this image.
To begin with, in terms of visual contact, this image is a “demand image” and an “offer image”, which demands
people to take immediate actions and have deep contemplation by making direct visual contact and offering
information of the worsening climate change; the far social distance in this image delivers objectivity;
involvement and equality is conveyed through the perspective of the image; and this image is of high modality.
As for the verbal text of the image, a declarative sentence is used to lead the viewers into thinking about the
consequences of not being environmentally friendly. Therefore, the visual and the verbal text of the image
integrate to address the topic issue – environmental protection and wildlife protection and to urge the human
society to take actions to save the planet by doing everyone’s own bit.
4.2 Analysis on Example 2 “Broken Child”
1528
Figure 2 “Broken Child”
(1) Visual contact
In terms of visual contact, this image can be counted as an “offer image” because the represented, the child
crying heart-brokenly against the wall with his limbs of serious injury, do not make direct eye contact with the
viewers. Therefore, the image offers the represented participants to the viewers as an item of information or an
object of contemplation. The viewers shall consider the meaning this image intends to convey carefully to
explore the messages contained.
(2) Social distance
The represented in this image is depicted using a long shot for the reason that the viewers can see the whole
human figure with space around it. By employing a long shot, a far social distance is suggested. At far social
distance, the phenomenon in the image is presented in an objectively and impersonally manner. With the long
shot, the image presents the objects for the viewer’s contemplation, leading the viewers to think about the
possible reasons behind such phenomenon as well as to find out effective and practical solutions to such issues.
(3) Perspective
For the horizontal dimension, an oblique angel is adopted to depict the child crying his heart out in the room.
The oblique angle detaches the viewers from the represented, correspondent with the detachment conveyed by
the far social distance between the represented participant and the viewers. For the vertical dimension, a low
angle is adopted since the viewers are requested to look up to the represented participant. The low angle is
suggestive of the importance of the portrayed participant and the significance of having the problem solved.
(4) Modality
This advertisement has high modality in terms of contextualization and representation, much details of the
background as well as the represented participants are depicted, such as the door and the bolt and the lock on
1529
the door, the connector plugs on the wall, the doorsteps, the tiles on the floor, the broken pieces of the child,
the facial expression of the child and so on. For color, the image has several colors and different shades of colors.
In addition, there are also the play of light and shade such as the changing of light on the doorstep.
The verbal components identified in this PSA mainly include this one sentence on the upper right side of the
image: (1)“YOU CAN LOSE MORE THAN YOUR PATIENCE” with all the letters capitalized. On the lower right
corner of the image is the sign of the institution who issued this advertisement and its website
(2)“www.goodparent.pl”. The letters in verbal text (1) are all in uppercase, which shows the producer’s emphasis
on the issue. This sentence brings the viewers to contemplate on the question “what else will I lose if I lose my
patience” and they may come up with unexpected answers themselves. From the name of the website, readers
may assume that this advertisement is to teach parents how to be good parents. And in this case, this
advertisement is to teach parents to be patient to their children.
To begin with, in terms of visual contact, this image is an “offer image”, which offers the viewers information for
contemplation; the far social distance in this image delivers objectivity; an oblique and low angle is adopted in
terms of perspective; and this image is of high modality. As for the verbal text of the image, a declarative
sentence is used to lead the viewers into thinking about the consequences of not being patient to the children.
Therefore, the visual and the verbal text of the image integrate to address the topic issue – parenting and child
education and to urge parents to have contemplation on their loss-of-patience issue to their children.
4.3 Analysis on Example 3 “Cigarettes Smoke People”
Figure 3 “Cigarettes Smoke People”
1530
(1) Visual contact
In this image, the represented object do not have their eyes depicted, not to mention whether they are looking
directly at the viewers. In this case, the viewers may take it as both a “demand image” and an “offer image”, or
consider it as neither a “demand image” nor an “offer image”, which depends on the viewers’ imagination and
association. As far as the author is concerned, the represented in the first image, the persona sitting on the bed,
may make direct eye contact with the viewers; while the two persons sitting face to face at a dinner may not
look directly at the viewers because it seems like they are having a conversation so they may just look at each
other. In this way, both the “demand” and the “offer” images are shown to the viewers, leaving the viewers to
intrigue their imagination and take relevant practices to the issue addressed.
(2) Social distance
In this image, there are two parts when referring to the represented. The first part of the represented, the persons
with cigarettes in their fingers, is depicted in the medium long shot since the full figure is shown, entailing a
close social distance with the viewers. The second part of the represented, the cigarettes being hold in persons’
hands, is depicted in the long shot since the object is shown out of the viewer’s reach with much background
information. The object is there for the viewer’s contemplation. In this case, an invisible barrier exists between
the viewer and the object possibly due to the extreme harm of the object – cigarettes can do harm to people’s
health as the image shows.
(3) Perspective
For the horizontal dimension, an oblique angel is adopted to depict the represented, both the people holding
cigarettes and the cigarettes. The oblique angel suggests that there is detachment rather than involvement
between the represented and the viewers. For the vertical dimension, a high angle is chosen for the viewers may
need to look down on the represented participants, which suggests the inferiority of the represented participants
in terms of health conditions for the reason that cigarettes are doing harm to their physical quality.
(4) Modality
This image is highly truthful for it has moderately high modality. To begin with, this image has high modality in
terms of contextualization and representation. A great deal of details of the background and the presented
participants are depicted, for example, the jewelries on the women’s hands, the smoke of the cigarettes, the
lighting lamps on the both sides of the bed, the ashtray and phone on the night table, the coffee cups and sugar
containers on the table, the lipstick on the brink of the cup, etc.
Different from the previous two examples, this PSA consists of two images, one above and the other below. The
verbal components of these two images is the same, which is on the lower right corner and upper right corner
of the images respectively. The verbal message contained is this image include (1)“CIGARETTES SMOKE PEOPLE”
with all the letters in uppercase and the sign and name of the institution who issued this advertisement which
is (2)“Cancer Patient Aid Association”. The verbal text (1) is a simple declarative sentence but with a logic quite
unmatched with the rules of English grammar or daily conversation because in normal cases, this sentence
should be “people smoke cigarettes”, but not the other way round. With the subject and object switched around,
the viewers may have a deeper impression on the image and be more easily influenced by this advertisement.
To begin with, in terms of visual contact, it is not sure whether this image is a “demand image” or an “offer
image” because the presented participants’ faces and bodies are “smoked up” by cigarettes; the close and far
1531
social distance between the two represented participants and the viewers embody different meanings; an
oblique and low angle is adopted in terms of perspective; and this image is of high modality. As for the verbal
text of the image, a declarative sentence is used to impress the viewers with unfamiliar logical relations.
Therefore, the visual and the verbal text of the image integrate to address the topic issue – smoking as bad
habits and to urge smokers to rethink their smoking habits to improve their health conditions.
5 Conclusions
Based on the previous analysis on three PSA examples in the field of wildlife protection, parenting, and health
respectively, there are some major findings. Firstly, although there is some distinction in image representation,
for example, in terms of visual contact, social distance, perspective and modality, the four aspects work together
rather than separately to construct the interactive meaning. Their corresponding to each other is evidence to
their cooperation. Secondly, the verbal text in the images tend to leave some space for contemplation to the
readers. Through the way the producers organize the verbal text, the viewers may start asking themselves
questions on the issue proposed, and may come up with their own new ideas on the current issues, but not
reading somebody else’s solutions to the problems without self-reflection. Thirdly, these images are generally
of high modality to make them truthful and reliable. In terms of representation and contextualization, these
images are all of high modality, with detailed environment description and exquisite background setting. Images
of high modality turn out to be more truthful and can leave deeper impression on the readers, thus increasing
the possibility of influencing their behaviors.
Needless to say, this study, like any other studies, has its limitations, which further research may refer to and
make some improvements. Firstly, the example analysis part may feature subjectivity due to the author’s
personal ideas or the reason that some visual images are difficult to define and analyze in a precise manner.
This could be a disadvantage of the study. For further research, the research methods for this kind of MDA may
be improved and more other analytical methods shall be developed or used for MDA. Secondly, due to
limitations on time and space, the data collected in the thesis may not suffice for the findings. For further
research, more PSA examples shall be collected and analyzed for a more comprehensive study in the future.
Thirdly, this study explore the static images only, while in this era of information technology, dynamic
multimodal discourses such as videos, films, and television advertisements are prevailing. Therefore, for future
research, dynamic multimodal discourse analysis shall be taken into consideration, studying how sound, action
and other semiotic resources work together to construct interactive meaning. Lastly, due to the author’s limited
theoretical ability and rare literature, the findings in this study may need to be testified with the advances of
academic research in the future.
Conflicts of Interest
The authors hereby state that there is no conflicts of interest in terms of the writing and publishing of this paper.
References
1. Kress, G. and van Leeuwen, T. Reading Images: The grammar of visual design[M]. London: Routledge. 1996.
2. Kress, G. and van Leeuwen, T. Multimodal Discourse: The modes and media of contemporary
communication[M]. London: Arnold. 2001.
3. Lemke, J. Multiplying meaning: visual and verbal semiotics in scientific text[A]. In Martin, J. & Veal, R. (eds.),
Reading Science: Critical and Functional Perspectives on Discourses of Science[C]. London: Routledge,
1998: 87-113.
4. Royce, T. Synergy on page: exploring inter-semiotic complementarity in page-based multimodal text[A].

In JASFL Occasional Papers (No.1) [C]. Tokyo: Japan Association of Systemic Functional Linguistics. 1998.
1532
5. O 'Halloran, K. Independence, interaction and metaphor in multimodal texts[J]. Social semiotics. 1999, 9(3):
317-354.
6. O 'Halloran, K. Towards a systemic-functional analysis of multi-semiotic mathematics texts[J]. Social

Semiotic. 1999, 124(1/2):1-29
7. Baldry, A. Phase and transition type and instance: patterns in media texts as seen through a multimodal
concordance[A]. In O’Halloran, K. (ed.), Multimodal Discourse Analysis: Systemic-Functional
Perspectives[C]. London: Continuum. 2004:83-108.
8. Li Zhanzi. Social Semiotic Analysis of Multimodal Discourse[J]. Foreign Languages Research, 2003,5:1-8.
9. Hu Zhuanglin. Multi-patterning in the study of social semiotics[J]. Language Teaching and Research,
2007,1:1-10.
10. Zhu Yongsheng. Theoretical Basis and Research Methods of Multimodal Discourse Analysis[J]. Foreign
Languages Journal, 2007, 5:82-86.
11. Eggins, S. An introduction to systemic functional linguistics[M]. London: Continuum. 2004.
12. van Leeuwen, T. Speech, Music, Sound[M]. New York: St Martin’s Press. 1999.
13. Martinec, R. Cohesion in action[J]. Semiotic, 1998:120(1/2), 161-180.
14. Halliday, M. An introduction to functional grammar[M]. London: Edward Arnold. 1994.
15. Cui Yan. Multimodal Discourse Analysis of Interactive Meaning of Flat Public Service Advertisements[D].
Master Thesis of Shandong University. 2010.
16. Xin Zhiying. New Development of Discourse Analysis and Multimodal Discourse Analysis[J]. Journal of
Social Sciences, 2008,5:208-211.
17. O'Toole, M. The language of displayed art[M]. London: Leicester UP. 1994.
18. Li Miaoqing. Analysis of Multimodal Discourse Analysis of Adapted Films - Take "Raise the Red Lantern"
as an Example [J]. Film Literature, 2007,8:98-100.
19. Gu Yuguo. Analysis of Multimedia and Multimodal Learning [J]. Technology Enhanced Foreign Language
Education, 2007, 2:3-12.
20. Zhang Delu. Exploring the Comprehensive Theoretical Framework of Multimodal Discourse Analysis[J].
Foreign Languages in China, 2009,1:24-30.
21. Ye Qichang. Social Semiotic Analysis of Hypertext Multilingualism[J]. Foreign Language Teaching and
Research, 2006,6:437-442.
22. Wang Hongyang. Interpretation of Multimodal Function in Cummings's Poetry[J]. Foreign Language
Teaching, 2007,5:22-25.
23. Wang Hongyang. Research on the Interactive Meaning of Multimodal Advertising Discourse[J]. Journal of
Sichuan International Studies University, 2007,6:31-34.
24. Zheng Haicui, Zhang Maizeng. Multimodal Discourse Construction of Harmony Concept [J]. Foreign
Languages Journal, 2008,2:107-112.
1533
25. Hu Zhuanglin, Dong Jia. Multimodal Construction of Meaning-Discourse Analysis of One-on-One

Demonstration Competition[J]. Technology Enhanced Foreign Language Education, 2006, 3:3-12.
26. Ding Jianxin. Research on the Social Semiotics of Modality in Visual Grammatical Fairy Tale Illustrations[J].
Journal of Sun Yat-sen University, 2007, 5:39-42.
27. Li Songqing. Social Semiotics and Advertising Discourse Analysis[J]. Journal of Anhui University of Science
and Technology, 2005,2:56-61.
28. Leech, G. English in advertising[M]. London: Longman. 1966.
29. Cook, G. The discourse of advertising[M]. London: Routledge. 1992.
30. Huang Guowen. Theory and Practice of Discourse Analysis - A Study of Advertising Discourse[M]. Shanghai:
Shanghai Foreign Language Education Press. 2001.
1534
English Language Teaching; Vol. 9, No. 8; 2016
ISSN 1916-4742 E-ISSN 1916-4750
Published by Canadian Center of Science and Education
A Multimodal Discourse Analysis of Tmall’s Double Eleven

Advertisement
Chunyu Hu1 & Mengxi Luo1
1
School of English for International Business, Guangdong University of Foreign Studies, China
Correspondence: Chunyu Hu, Professor of School of English for International Business, Guangdong University
of Foreign Studies, Guangzhou 510420, China. Tel: 86-20-3631-7183. E-mail: gwhcy@gdufs.edu.cn
Received: May 2, 2016 Accepted: July 1, 2016 Online Published: July 3, 2016
doi: 10.5539/elt.v9n8p156 URL: http://dx.doi.org/10.5539/elt.v9n8p156
Abstract
From the 1990s, the multimodal turn in discourse studies makes multimodal discourse analysis a popular topic in
linguistics and communication studies. An important approach to applying Systemic Functional Linguistics to
non-verbal modes is Visual Grammar initially proposed by Kress and van Leeuwen (1996). Considering that
commercial advertisement is an indispensable part of the modern society that bears rich meanings worth
discussing, this paper analyzes visual components of the advertisement produced by Tmall for the Double
Eleven Shopping Carnival from the perspective of Visual Grammar. By analyzing representational, interactive
and compositional meaning presented in the advertisement, this article illustrates how visual components serve
as a huge attraction to the viewers and effectively justifies the consumption behavior by appealing to the cultural
and social state. It also sheds some light on raising the awareness of consumers by presenting how advertisement
producers practice psychological manipulation on the viewers.
Keywords: visual grammar, multimodal discourse analysis, the Double Eleven Shopping Carnival, commercial
advertisement, multimodal literacy
1. Introduction
Chinese’s interest in festivals stretches through the ages, but changes also take place in terms of celebration over
time. For many people, consumption has become the central feature of modern festivals, and the irreversible
trend gave rise to the appearance of man-made festivals. For example, Foodie’s Day was created by merchants in
May 17th, because 517 resembles the sound of “I want to eat” in Chinese; May 20th was made a day for couples
for 520 resembles “I love you” in Chinese. The most famous example, however, is the Double Eleven Shopping
Carnival. Molded by Tmall, November 11th was initially made a day for singles in 2009. In the first year, its sales
volume reached 5 million RMB (AdMaster, 2015), which proved to be a huge success. The following years has
witnessed the steady increase in the sales volume, and Double Eleven Shopping Carnival became the most
famous man-made online shopping festival in China.
Under the circumstance, the reason behind Double Eleven’s popularity is worth researching. Although it started
as a shopping carnival, now it has become a part of our popular culture with a total sales volume of over 91
billion RMB in 2015’s event and a participation rate up to 79% among Netizens in 2014 (Ipsos, 2013).
For many years, text analysis or discourse analysis in linguistic field has been the central focus, and language has
been an exclusive interest for research. Profound studies in the field of multimodal analysis have been focusing
on some of the static texts like newspapers and magazines (Zhang, 2007), print advertisements and posters (Yu,
2013), textbooks, and dynamic ones like Public Service Advertisement (Wang, 2012; Qian, 2014), forensic texts
(Guo, 2014) and movies (Luo, 2010). However, little attention has been paid to the analysis of commercial
advertisement’s structuring and how it resonates powerfully with consumer thus trigger consumption.
As the embodiment of the spirit of Double Eleven, the commercial advertisement produced by Tmall makes a
good subject for the study. This paper employs the Tmall Double Eleven advertisement launched in 2014,
discussing the cultural background and marketing motives of Tmall’s advertisement in Double Eleven, and hopes
to raise more attention on multimodal study.
This study aims to address the following two questions:
a. How does the advertisement appeal to and resonant with the consumers through multimodal construction?
156
www.ccsenet.org/elt English Language Teaching Vol. 9, No. 8; 2016
b. How does the advertisement justify the man-made festival through visual presentation?
2. Literature Review
2.1 Multimodal Discourse Analysis
The earliest scholar studying multimodal discourse is R. Barthes, who discussed the interaction of language and
image in expressing meanings in Rhetoric of the Image (1977). In 1990s, the field of study raised more concern
in academia. In the prosperity of multimodal discourse analysis, two research approaches are widely applied,
namely multimodal metaphor approach and systemic functional linguistic (SFL) approach. Multimodal metaphor
approach, represented by Forceville (1996), studies multimodal discourse from a cognitive perspective. The
representative of SFL is Halliday, who interpreted the functional relationship between language and social
structure, worked on the definition of linguistics and set the theoretical groundwork of SFL in his book An
Introduction to Functional Grammar in 1994. Based on his contribution in functional linguistic theories,
O’Toole (1994) and Kress and van Leeuwen (1996) laid the landmark of multimodal discourse analysis (MDA).
Apart from these two sociological theorists, other scholars including Martinec (2000) and O’Halloran (2004)
have also made theoretical contribution at the early stage of development of multimodal discourse analysis from
a systemic functional perspective.
In recent years, more theorists started doing interdisciplinary studies which put more emphasis the integration of
different research methods in their analysis instead of merely discussing social semiotics (Forceville &
Urios-Aparisi, 2009; Feng & Jing, 2011). Digital technology was borrowed for the annotation and analogue of
complex multimodal discourse analysis (Lim, 2011; O’Halloran, et al., 2012). Multimodal corpus was
established and multimodal retrieval software was built for further analysis (Baldry & Thibault, 2008; Gu, 2006,
2009).
Apart from the variation of research methods, subjects for research are also expanding, such as
three-dimensional space (Stenglin, 2011; Ventola, 2011), paralinguistic features (Hood, 2011; Knight, 2011),
picture and comic books (Feng & O’Halloran, 2012); Painter, Martin, and Unsworth (2013) and situated
discourse (Gu, 2006, 2009). From the previous studies, it can be concluded that the most innovative progress was
made on the multimodal analysis of movies and new media. However, the existing multimodal theories are still
in need of improvement, for example, insufficiency visual grammar and multimodal metaphor in terms of
theoretical framework. Also, the trend of multimodal analysis is going beyond intradisciplinary study, and
whether it is moving toward interdisciplinary or transdisciplinary is still unpredictable, requiring more research
and discussion.
2.2 Visual Grammar
Proposed by Kress and van Leeuwen in 1996, the framework of Visual Grammar is based on the contribution of
the former semiotic school’s findings which are originally used to illustrate linguistic texts. Halliday sees
language as a semiotic mode which represents three metafunctions: the ideational metafunction, the interpersonal
metafunction and the textual metafunction. Based on Halliday’s theory, Kress and van Leeuwen use different
terms for the same subjects: representational instead of ideational; interactive instead of interpersonal; and
compositional instead of textual.
The representational meaning deals with the way images represent the relations between represented participants
in the picture. It is divided into narrative representation and conceptual representation, and narrative
representation can be further categorized into action process, reaction process and speech and mental process,
which referred to the six processes in Halliday’s Transitivity system. Action process is similar to material process
in SFL, and reaction process and verbal and mental process to verbal and mental process. While narrative
representation presents unfolding actions and events, processes of change, transitory spatial arrangements,
conceptual representation represents participants in terms of their more generalized and more or less stable and
timeless essence, in terms of class, or structure or meaning (Kress & van Leeuwen, 2006, p. 79). Conceptual
representation is divided into classificational processes, analytical process and symbolic process, with the former
two resembling relational process in SFL.
The interactive meaning is mainly about the social relations between interactants and the evaluative orientations
that participants adopt towards each other and to the world represented by the text. Its realization relies on four
elements, namely contact, social distance, attitude and modality. Contact, which discusses the demand or offer
relation between represented participant and viewer, conveys meaning through ‘demands’ and ‘offers’, which is
related to Halliday’s description of four ‘speech acts’. Social distance represents social relations between the
producer, the viewer and the represented participant, which is realized by size of frame. Attitude, which classifies
157
images into subjective and objective ones, is deemed naturally neutral yet altered by angles. Modality, which
comes from linguistics and refers to the truth value or credibility of (linguistically realized) statements about the
world (Kress & van Leeuwen, 2006, p. 155), explores the role of modality markers including color saturation,
color differentiation, color modulation, etc.
The compositional meaning relates the representational and interactive meanings of the image to each other
through three interrelated systems (Kress & van Leeuwen, 2006, p. 177), the three principles of composition are
information value, salience and framing. Information value studies the placement of elements and their values
attached accordingly, with the left and right arrangement represents given and new information, top and bottom
arrangement resembles ideal and real information, center and margin arrangement is also discussed in respect of
culture context. The construction also referred to Halliday’s viewpoint that the combination of known and new
information is the most common information structure. Salience refers to how a hierarchy of importance among
the elements is created through variations in size, sharpness of focus, tonal contrast, etc. Framing discusses the
relationship between the degree of connectedness and the significance of individuality or differentiation. The
latter two components of the compositional meaning provide practical insights on the interpretation of images,
however, the theory does not elude itself from over-interpretation or misinterpretation of images considering the
possible subjectivity of the researcher or his/her unfamiliarity with unusual attribution bearing decisive
meanings.
Recent years have witnessed new development of visual grammar in theoretical refinement (Painter, Martin &
Unsworth, 2013) and attempt in interdisciplinary discussion (Feng & Jing, 2011) and renewed research
perspective (Bateman, 2014). However, we are still in the trial stage where the quality of output from the
exploration is still in need of refinement.
2.3 Previous Studies on Advertising
Advertising discourse produces social meanings and symbolic value through various channels and influences
social-culture and social relationships (Yang, 2007). In the field of study, traditional analysis focused on the
lexical, grammatical, stylistic, rhetorical features of language and its social functions (Leech, 1966; Williamson,
1978; Huang, 2001; Zhang, 2007; Hu, 2007a & 2007b). Leech (1966) is the pioneer in advertising research, he
investigates personal, imperative, passive voice and other linguistic features in language. Barthes (1977) and
Williamson (1978) are the representatives of scholars who analyze advertising discourse from the semiotic
perspective, Barthes introduced the relationship between images and information in advertisements, and
Williamson thinks that advertisements analysis is not only a process of coding and decoding (Han, 2011).
Vestergaard and Schroder (1985) explained the social motivation in advertising discourse from the pragmatic
perspective. Geis (1982) studied television commercial advertisements with conversation implication and
cooperative principles and found that advertising language is persuasive and spreading. Cook (1992) discussed
the interactive function of text, music, picture and participants in advertising. Huang (2001) made use of
systemic functional grammar to study advertising discourse.
Research on online shopping festival has been a hot topic in recent years, Double Eleven Shopping Carnival, as
the most successful one, is discussed most frequently. Previous studies can be divided into marketing analysis
(Feng, 2012; Liu, 2013) and statistical research (Ipsos, 2013), which focused more on broad tactical analysis and
assessment, the cultural and social background contributed to Tmall’s success was rarely mentioned. Thus, this
study combines the less discovered aspects by conducting multimodal analysis on the advertisement Tmall
produced for Double Eleven, hopefully, the paper would inspire new thoughts for more original insights and a
broader perspective on the study of multimodal analysis and the study of advertisement.
3. Tmall’s Double Eleven Advertisement: A Multimodal Discourse Analysis
The core objective of advertising is to specify and substantiate the value of product or service to its potential
buyers. In order to present and transmit as much effective information to the consumers as possible in a society
where visual culture has become increasingly important, commercial advertisement has evolved from
single-mode to multimodal expression in advertising.
This part mainly discusses the visual components of the Tmall advertisement for the purpose of having a clearer
understanding of how advertisement appeal to and resonant with the consumer through multimodal construction
while propagating its value. In the ad, four scenes are presented, depicting female friends gathering, a man
crashing a piggy bank with a wrecking ball, a woman and a wall with shaped holes and a hardworking man
respectively. In order to be clearer, screenshots from the ad are presented accordingly.
158
3.1 Representational Meaning

According to Kress and van Leeuwen, conceptual patterns represent participants in terms of their class, structure
or meaning, in other words, in terms of their generalized and more or less stable and timeless essence, narrative
patterns serve to present unfolding actions and events, processes of change, transitory spatial arrangements. The
hallmark of a narrative visual ‘proposition’ is the presence of a vector: narrative structures always have one,
conceptual structures never do (Kress & van Leeuwen, 2006: 59).
(1) Narrative Processes
Narrative process is classified into three sub-processes: action processes, reaction processes, and verbal and
mental processes.
In action processes, the Actor is the participant from which the vector emanates. When images or diagrams have
only one participant, this participant is usually an Actor, the resulting structure is non-transactional process;
when there are two participants respectively serving as the Actor and the Goal, we call it transactional process.
The Actor, in whole or in part, forms the vector, which can be analyzed through size, place in the composition,
contrast against background, color saturation or conspicuousness, sharpness of focus, and through the
‘psychological salience’ which certain participants have for viewers (Kress & van Leeuwen, 2006, p. 63). In the
Tmall ad, transactional processes overweigh non-transactional ones, especially when it comes to the construction
of the highlights in each scene where salient Actors are shown specifically ‘aimed at’ the Goal, creating a
concrete context to the viewer who, in this case, is saved the trouble of needing to ponder over the profound
meanings behind. However, non-transactional processes do exist, and the occasionally appearance is not
randomly arranged or edited, on the contrary, the process usually appear at the early stage of a scene with the
gesture of represented participant forming a vector without specific Goal in the frame, forming an open-ended
situation for consideration, and leaving the viewer imagining the psychological activities of the Actor, in other
words, relating to the image, thus creating a sense of empathy or identification.
In reactional processes, the vector is formed by an eye line, the direction of the glance of one or more of the
represented participants (Kress & van Leeuwen, 2006, p. 67). The Reactor, the participant who does the looking,
are humans in the Tmall ad, they convey the spirit of the holiday mainly through their facial expressions and
direction of glances, leading the viewer to see what they see, feel the way they feel and believe what they believe.
The Phenomena may be formed either by another participant or a visual proposition, and when there is no
Phenomenon, reactions are non-transactional. In Tmall advertisement’s case, the Phenomena is formed by the
piggy bank, the wall with shaped holes, wall of piles of files, etc., resembling the release of stress, fulfilling of
mental void and shattering of pressure from work respectively, which are all realized by purchasing behaviors.
The function of the Phenomena is rather important for the advertisement in that it forms the transactional
reaction, which is quite prominent in the recreational processes and presents the concrete and
easy-to-comprehend imagery to the viewer to enhance the validity and legitimacy of the man-made festival.
In verbal and mental processes, a special kind of vector, in the forms of thought balloons and dialogue balloons
that connect drawings of speakers or thinkers to their speech or thought, is discussed for its appearance in comic
strips. However, the vector does not appear in the ad, thus the discussion of verbal and mental process is omitted
from the analysis.
(2) Conceptual Processes
The conceptual representation is comprised of classificational processes, analytical processes and symbolic
processes. The former two study the relationship between represented participants, which resembles the
relational process in SFL, marked by whether there exists superordinate apart from subordinate, classificational
processes are categorized into Covert Taxonomy and Overt Taxonomy. Analytical processes mainly discuss the
part-whole structure, involving two kinds of participants, namely Carrier (the whole) and Possessive Attributes
(the parts). The symbolic processes, which discuss the meaning of participants, can be divided into Symbolic
Suggestive and Symbolic Attributive depending on whether there exist one or two participants respectively.
Symbolic process is ubiquitous in the Tmall ad and Symbolic Attributive appears more frequently.
159
Figure 1. Wall with shaped holes
Take one scene as a representative example, in the ad, a high wall with shaped holes (figure 1), which resemble
the shape of dresses, bags, shoes, dolls, furniture, etc., is presented. The wall itself, as a represented participant,
is made salient through its size in the picture and eye-catching color, and the holes are visualized source of
mental void. With the red-dressed woman standing before the wall displaying a sense of yearning and fulfillment
after the void is fulfilled, the ad equivalent mental void with unsatisfied material needs and provides a solution
for achieving fulfillment through the quickest way—consumption.
3.2 Interactive Meaning
The interactive dimension of images is the “writing” of what is usually called “non-verbal communication”, a
“language” shared by producers and viewers alike. It involves two kinds of participants, represented participants
(the people, the places and things depicted in images) and interactive participants (the people who communicate
with each other through images, the producers and viewers of images), and three kinds of relations: (1) relations
between represented participants; (2) relations between interactive and represented participants (the interactive
participants’ attitudes towards the represented participants); and (3) relations between interactive participants
(the things interactive participants do to or for each other through images) (Kress & van Leeuwn, 2006, p.114).
The interactive meaning is realized by four factors: contact, social distance, attitude and modality.
(1) Contact
The visual configuration has two related functions. On the one hand, an image may create a visual form of direct
address, in which contact is established by direct eye gaze or gestures of represented participants, in this case, we
call this kind of image a ‘demand’; on the other hand, an image may also address the viewer indirectly , in this
case, it ‘offers’ the represented participants to the viewer as items of information, objects of contemplation,
impersonally, as though they were specimens in a display case (Kress & van Leeuwen, 2006, p. 119).
In some contexts—for instance, television newsreading and the posted magazine photograph—the ‘demand’
picture is preferred: these contexts require a sense of connection between the viewers and the authority figures,
celebrities and role models they depict. In other contexts—for example, feature film and television drama and
scientific illustration—the ‘offer’ is preferred: here a real or imaginary barrier is erected between the represented
participants and the viewers, a sense of disengagement, in which the viewer must have illusion that the
represented participants do not know they are being looked at, and in which the represented participants must
pretend that they are not being watched (Kress & van Leeuwen, 2006, p. 120).
In the Tmall ad for Double Eleven, ‘offer’ images are presented. The represented participants, women and men,
do not create direct eye contact with viewers, however, their facial expression and conduct display the happiness
and enjoyment gained from the man-made festival. The intangible yet undeniable link between happiness and
consumption is strongly indicated to the viewers. What the ad offers for its viewers is the sense of fulfillment
people feel after buying the commodities they wanted, which insinuates that the viewers can also feel satisfied
and live a happier life mimicking what the represented participants do, in other words, buying commodities from
Tmall.
(2) Social Distance
The choice of distance can suggest different relations between represented participants and viewers, the
construction of image itself can make the viewer feel close or far away from the represented participant. In our
daily interactive activities, regulation imposed on social relation and interaction determines the distance we
should keep from each other and influences the way we interact with each other.
In advertisement, size of frame is invariably defined in relation to the human body. According to Kress and van
Leeuwen, the close shot (close personal distance) shows head and shoulders of the subject, and the very close
shot (intimate distance) anything less than that. The medium close shot (far personal distance) cuts off the
160
subject approximately at the waist, the medium shot (close social distance) approximately at the knees. The
medium long shot (far social distance) shows the full figure. In the long shot (public distance) the human figure
occupies about half the height of the frame, and the very long shot is anything ‘wider’ than that (Kress & van
Leeuwen, 2006, pp. 124-125). In conclusion, the distance of depicted participants from the viewer is one of the
material realizations of social interaction in a given context.
In the ad, images showing the head and shoulders and the whole figure with space around it appear the most
frequently, accompanied by occasional long shot. The rare showing of intimate distance and detachment is by no
means a coincidence. In some ads, in order to enhance the feeling of involvement and intimacy, a close shot is
employed. In other cases, a long shot is employed, and although the represented participants are looking at the
viewers, the distance weakens the impact on us.
In the Tmall ad, however, not all scenes are shot at close distance, instead, medium shot and medium long shot
are used more frequently along with occasional appearance of other sizes of frame. Beside, flexible camera
movement between close and long shot appears frequently. Several reasons can be concluded based on the aim
and feature of the ad: (1) the ad attempts to form the atmosphere of joyfulness by sketching in the details of
every scene, in this case, the ad does not only try to capture the revealing of individual feelings, but more
importantly, heighten the over-all joyous and appealing atmosphere of the very special day, apart from building
up and spreading the spirit, as a commercial, it is also important to trigger acquisition, thus (2) the delicate
distance between the represented participants and viewer in most scenes, to a certain extent, prevents the viewers
from immersing themselves fully in the scene the ad created as the viewers are watching at a far social distance,
in other words, the distance of the ‘stranger’. The fact that the image brought by the passionate ad alone cannot
fulfill the viewers’ craving for fulfillment subconsciously trigger the buying behavior. It is also important that the
distance is not so far as to create an unreachable sense for the audience, hence the long shot, which indicates
public distance is rarely used. Thus, the images in the ad create a subtle distance, and in order to fill the gap, it is
insinuated that the audience would need to make purchases; and (3) the frequent camera movement between
close and long shot also plays a role in triggering consumption. In the ad, the movement of camera either zooms
in from a long shot to close shot, or zoom out from presenting a scene showing several represented participants
to the face or head of a specific represented participant.
(3) Attitude
Attitude discusses the relations between represented participants and the viewer, and the system of perspective
realizes “attitude”, which is often socially determined, is usually realized by the selection of the angle. The
selection of an angle, a ‘point of view’, implies the possibility of expressing subjective attitudes towards
represented participants, human or otherwise (Kress & van Leeuwen, 2006, p.129).
Horizontal angle forms either a frontal or an oblique point of view, which respectively signifies different degree
of involvement or detachment. Represented participant shown by frontal angle indicates that special attention
need to be paid to him or her, participants shown by oblique angle, on the other hand, usually suggests otherwise.
In the Tmall ad, frontal angle is widely used in the construction of images so that the audience has a greater
feeling of being involved into the holiday atmosphere. However, there is one exception, in the scene depicting a
wall of shaped holes filled with commodities (Figure 2), the represented participant is given a back view. This
scene suggests us to pay more attention to the wall, meanwhile the woman does not only represent an individual
who has specific identities, she become ‘us’, viewers of the ad.
161
Figure 2. Wall filled with commodities
Vertical perspective is realized by camera height and signifies different degree of power. A high angle, it is said,
makes the subject look small and insignificant, with the tendency of diminishing the individual; on the other
hand, a low angle makes it looks imposing and awesome, giving an impression of superiority, exaltation and
triumph (Kress & van Leeuwen, 2006, p. 140), and eye-level angle indicates an equal relationship and gives out
a sense of neutral documentation.
In the Tmall ad, the eye-level shot is used generally to suggest the normality of being happy with material
sufficiency. One scene worth paying special attention to, however, is the one that depicts a hard-working man
surrounded by piles of documents from a high angle (Figure 3), which adds to the feeling of exhaustion from the
heavy workload. The following scene witnesses a drastic change in his over-all state from a low angle, showing
his smiling and relieved face from a medium close shot, his figure accordingly becomes more significant, and the
mood of the very scene changes into a light-hearted one as Double Eleven comes. The indication that the
man-made festival can save people from heavy workload is presented through the carefully constructed
perspective.
Instead of using high or low angle, eye-level angle is used in presenting the image, which resembles equality.
Shooting from a physically higher angle and look down on the subject usually makes the subject looks small or
weak whereas a low-angle shot taken from below makes the subject looks more powerful and spectacular. A
neutral shot or eye-level angle, however, attaches little psychological effect on the viewer with the camera looks
straight on with the subject.
Figure 3. Man surrounded by document piles
(4) Modality
The term ‘modality’ comes from linguistics and refers to the truth value or credibility of (linguistic realized)
statements about the world (Kress & van Leeuwen, 2006, p. 155), and the specific degrees of modality are
measured by modality markers including color saturation, differentiation and modulation, contextualization,
162
representation, depth, illumination and brightness. In the Tmall ad, full color saturation, high diversified range of
colors and fully modulated color are prominent, displaying articulated and detailed background, low degree of
abstraction and high brightness value. By presenting an ad with the features above, a light-hearted and cheerful
atmosphere is presented, echoing with the theme of the festival. The detailed depiction, along with a sense of
approval created by the positive atmosphere, helps to justify willfully consumption on this very day.
3.3 Compositional Meaning
Composition relates the representational and interactive meaning of the image to each other through three
interrelated systems (Kress & van Leeuwen, 2006, p. 177): information value, salience and framing.
(1) Information Value
The placement of elements (participants and syntagms that relate them to each other and to the viewer) endows
them with the specific informational values attached to the various ‘zones’ of the image: left and right, top and
bottom, centre and margin (Kress & van Leeuwen, 2006, p. 177). The left to right composition shows the
information from given to new, top to bottom shows the information from ideal to real, and centre and margin
composition signifies the greater emphasis on the central part of images.
In the Tmall ad, most scenes follow the centre and margin construction in which the represented participants are
usually the highlighted part in images, reemphasizing the role of general public as the central point of the festival.
The central composition is relatively uncommon in contemporary Western visualization, though. According to
Kress and van Leeuwen, perhaps it is the greater emphasis on hierarchy, harmony and continuity in Confucian
thinking that makes centring a fundamental organizational principle in the visual semiotic of Asian culture
(Kress & van Leeuwen, 2006, p. 195).
Figure 4. Wine glass
Apart from that, at the beginning of the ad, it shows a cherry dropping into a glass of wine (Figure 4) right after
showing a happy woman falls from mid-air. Goatly (2007, pp. 36-37) stated that “high is also a metaphor for
power and dominance, as in high places, high handed”, and “the symbolism of height as power is especially
noticeable in the penchant for tall buildings”. The scene of a woman falling from mid-air is a typical downward
movement, so does the dropping cherry, the vertical movement from top to bottom indicates the change of state
from being ideal and unreachable to real and accessible. Unlike automobile advertisement, in which cars often
display upward movements indicating the outstanding mechanical performance as well as the driver’s capability
of moving upward in the society (Feng & Xing, 2011, p. 60), the Tmall ad aims to highlight accessibility. Along
With the voice-over saying “On this day, I’ll taste the sweetness of losing control”, the glass of wine can be seen
as the routine life before the shopping carnival, and the cherry, resembles the breaking of routine, can be viewed
as a sweet, surprising gift worth embracing and made reachable through the festival. The ad hereby justifies the
day by comparing the shopping carnival to a sweet surprise.
(2) Salience
The elements (participants as well as representational land interactive syntagms) are made to attract the viewer’s
attention to different degrees, as realized by such factors as placement in the foreground or background, relative
size, contrasts in total value (or color), differences in sharpness, etc. (Kress & van Leeuwen, 2006, p.177). Based
on their thought, Machin (2007, p. 130) states that “Salience is where certain features in composition are made to
stand out to draw the viewer’s attention”.
One representative scene is Figure 1. In the screenshot, we can see that although the woman in red dress is in the
163
foreground, the wall in the background, featured by the golden light filtered in through the shaped holes in the
wall, is the more salient composition, the salience and eye-catching golden color highlight the commodities and
glorifies its importance.
Figure 5. Man riding a wrecking ball
In one scene, the ad shows a piggy bank and a man riding a wrecking ball (Figure 5). The color contrast between
the cool white and garish red is impactive: strongly saturated red dots all over the room creates an intensive and
anxious atmosphere, which echoes with the stressful life. In this way, keeping bank saving is depicted as being
equivalent to the root cause of anxiety. Piggy bank, which has been used for a long time as a money-saving
suggestion, is crashed into by a silver wrecking ball rode by a young man. The gesture is depicted in a way
which makes it looks like a light-hearted game, insinuating that spending money on this very day can ease the
anxiety brought by the social norm which encourages frugality, thus encourages the viewers to do the same
thing—to take out their money in the bank and buy themselves some pleasure. The color contrast contributes
greatly to the accomplishment of the construction of the hidden message.
(3) Framing
The presence or absence of framing devices (realized by elements which create dividing lines, or by actual frame
lines) disconnects or connects elements of the image, signifying that they belong or do not belong together in
some sense (Kress & van Leeuwen, 2006, p. 177). The more the elements of the spatial composition are
connected, the more they are presented as belonging together, as a single unit of information (Kress & van
Leeuwen, 2006, pp. 203-204). Connectedness is the mainstream feature in terms of multimodal construction in
the ad, nevertheless, there is one exception. In Figure 6, the image is cut into two parts featured by the
discontinuous color of the double door and the space in between. The scene does not last long, but it is critical
and plays an important role in the ad. In the scene the lady running across the closing door creates a sense of
tension for the door might close before she enters, which insinuates the fact that being a part of the shopping
carnival is an event worth going for, and since the festival only lasts for one day, it is advised that viewers need
to take the seemingly precious chance. In the scene, although the lady is at the centre of the image, the focal
point is on the closing door, heightening the diminishing chance. At this point, he vague figure of the represented
participant suggests that she no longer represent a specific individual, but the general public.
Figure 6. Woman running through a door
164
4. Discussion
The aim of this section is to discuss the previous findings in the analysis and examine how the advertisement
successfully applied AIDA, the advertising principles, through multimodal construction, advertising patterns
brought up by Cook (2001) are also included and discussed in this section.
According to Lewis (1903), the mission of an advertisement is to attract a reader, so that he will look at the
advertisement and start to read it; then to interest him, so that he will continue to read it; then to convince him, so
that when he has read it he will believe it. Later on, the formula was further developed and widely used in
marketing analysis. In Habermas’s theory of Communicative Action, this type of business action belongs to the
strategic action in which the advertiser appeal to the desires of the consumer so as to motivate consumption that
is required for advertiser’s success. Cook (2001, p. 232) puts it with commoner expression by stating that
advertising attracts receivers’ attention, then trick them into buying the product, appealing to greed, vanity, lust
and fear, and suggesting that purchase will make the receiver like the people portrayed.
(1) Attention
As Crawford (2015) puts it, “Attention is a resource—a person has only so much of it”. When it comes to
advertising, it is even harder to get attention since ads, for many people, are either not at the centre of attention
or do not hold attention for long (Cook, 2001, p. 222).
Advertising does not only provide sources of information, but more importantly, employs verified methods for
persuasion to grasp the consumer’s attention. Three ways can be employed (Suggett, 2014), namely location,
shocking factor and personalization. In Tmall’s case, emphasis on personalization is the most salient feature.
a. Represented participants
There are six represented participants, four women and two men, in total, all between the age of 20 to 30. The
directivity in age implies that audience and major consumers are the younger generation who are relatively more
proactive in terms of online shopping. The fact that female outnumbers male further indicates that women are
more of the main target of the ad. Another feature is that no celebrities are involved in any of the scene, instead,
regular-looking people are the leading figures. the advertisement aims to shorten the social distance with the
general public by presenting participants similar to average people, in this way, it appeals more accessible for the
viewers to live a similar happy life, also, it is easier for the viewers to relate to the scenes. By bring ordinary
people in the limelight, the ad tries to convey its spirit that Doble Eleven is a festival for the general public.
b. Specified context
As an ad which encourages shopping, Tmall did not attract the viewers by presenting manifold commodities it
can provide, nevertheless, the ad constructs four different scenes with their themes close to the daily life of its
audience.
In the ad, four scenes depicting female friends gathering, a man crashing a piggy bank with a wrecking ball, a
woman and a wall with shaped holes and a hardworking man surrounded by document files respectively are
presented. While friends gathering and hard-working employees are more commonplace, the man riding a
wrecking ball and woman filling a wall with shaped holes with commodities may seem less likely to happen in
daily life, however, it is the themes conveyed through the ad that are common. By crashing the piggy bank, the
represented participant of the very scene visually practices what many viewers would not dare to do in daily
life—purchase commodities without overly worrying about the bank account, by presenting the scene, the ad
offers the audience a day in which the guilt of buying without planning can be exempted. The wall with shaped
whole is another metaphor which equivalents the mental void with insufficiency in material requisites, in a
society where loneliness and void brought by fast-paced modern life has become increasingly overwhelming, the
ad visualizes and directs at the specific context.
(2) Interest
To increase and maintain viewers’ attention, ‘Unrealism’ is sometimes used in describing a bland and
problem-free world: the families are happy; the days are sunny; the meals tasty; the Christmases snowy; the
grannies kind; the roads uncongested; the countryside unspoiled; the farming traditional (Cook, 2001, p. 224).
Nonetheless, the feature of being selective and avoiding problems is not overriding in the Tmall ad. In four
scenes depicted in the ad, two deeds normally deemed as unconventional and prevailing phenomenon are
addressed.
Losing control and spending money as we wish without planning ahead are often deemed as inappropriate,
however, the ad presents the inappropriateness in a positive and joyous atmosphere. In the first scene, the guilty
165
pleasure of enjoying the feeling of losing control with friends are presented light-heartedly, and in the second
scene, casual consumption is depicted as a thrilling game. The prominent feature heightening the playful scenes
is the use of color. In modern life, color plays important role in expressing meaning. For instance, in traffic light
red is used on warning signs, and green on safe signs. In hospitals, the room is painted white for the purpose of
calming patients down (Zhou, 2012, p. 42). The definition of modality in visual analysis refers to the usage
degree of the certain image representing devices (color, represented detail, depth, tone, etc.) (Norris, 2004, p.
256). The Tmall advertisement features heavily with bright, warm colors, the full palette of diverse colors
renders a cheerful and positive feeling. Specifically, the viewer can connect the bright colors with happiness,
hope and cheerfulness, and the happier viewers become, the more likely they are going to complete the purchase
procedure. The artistic expression of hyperbole creates a sense of resonance for the viewers.
Instead of presenting an abstract idea of “consumption makes people happy”, the articulation of detail is finely
grained in the ad. Through detailed depiction, the ad tries to equal sufficient material wellbeing with happiness
and fulfillment. To make the abstract idea of happiness more concrete, images in the ad presents details to their
fullest.
The prevailing phenomenon of modern society presented is also a key factor which maintains the audiences’
attention. In the third scene, a young female who choose to fill the mental void with commodities is portrayed.
Mental void has become a commonplace issue that individuals need to deal with on their own in modern society.
In the ad, the void is visualized as a wall with wholes, and by filling them with products, the void is filled as well.
The visualization of filling our void with accessible products which can be easily bought from Tmall presents a
soothing way of dealing with the problem. In the last scene, a man surrounded by paperwork is depicted from a
high angle, the presentation of a commonly seen situation, the ad raises empathy among viewers since a large
proportion of the targeted audience are employees whose workload can also be excessively heavy. By relating to
the viewers, interest is maintained.
(3) Desire and Action
After successfully gained attention and maintained viewers’ attention accordingly, it is necessary to transform the
thought into desire for the product or service. By showing four different yet specific situations, the desires of the
represented participants are light-heartedly presented and satisfactorily met, the persuasiveness and accessibility
justify the reasonableness of the man-made festival.
Though shot from staged scenes, the scenes reflect realistic problems which relate to its viewer. Seeing how the
men and women enjoyed themselves and solved their problems, there is no need to shout slogans to win the
audience’s heart since the ad speaks for itself well. By showing that the festival is a day for the general public,
the ad creates attention; by presenting down-to-earth scenarios in an acceptable approach, interest is maintained;
the specification of scenes presented and the cheerfulness in atmosphere relate successfully to the viewers and
offers a quick solution—consumption, the ad arises interests and creates desire, which could very likely turn into
consumption, and even if it does not, the spirit and essence of Double Eleven has ran deep into the consumers’
cognition toward the day, which to a certain degree, has become part of the consumer culture.
5. Conclusion
In the Tmall Double Eleven advertisement, the multimodal construction successfully conveys and realizes the
purpose of the ad—promoting the festival culture and trigger consumption. Based on the previous analysis, the
research questions are hereby answered.
The main tactic employed to appeal and resonant with the consumer is to create a delicate balance between
immersion and distance through multimodal construction. The use of bright color presents a vivid and positive
image to the viewers, by creating a warm and relaxing phenomenon for the consumers. Images in the ad convert
between short and medium social distance, accompanied by occasional long shot. The feature aims to create a
situation which is not too immersive to the viewers but also accessible, thus trigger the behavior of consumption.
Also, the avoidance of direct contact establishes a strong link between happiness and consumption.
As for the justification of material consumption, the advertisement does not bluntly display its products or
services for sale, instead, the ad caters to the common insufficiencies in our society and provides solutions, thus
making the festival a day for people to fix problems and enjoy themselves. Consumption is necessary at times,
yet also viewed as something need to be done with rationality. However, the advertisement dispels the
discomfort by placing the the shopping carnival as a means of letting go. It is not only about letting go of tightly
saved money, but also the uptight nerves, accumulated pressure, and the despairs aroused by emotional
blankness. By echoing to the social need, the advertisement serves as more of a spiritual release other than a
166
mere promotion.
With multimodal components playing important roles in the justification of Double Eleven Shopping Carnival,
the ad reflects the spirit and motivation of the carnival, as well as current social state. By revealing the successful
tactics employed by Tmall in the ad for promotion, the analysis not only provides another perspective for
multimodal discourse analysis, but also serves as a reference for other enterprises with intention to renew and
expand its social influence, apart from that, the writer also hopes to help raising the cautious of consumers by
presenting how advertisement producers practice psychological manipulates on the general public.
The analysis also hopes to shed light on the cultivating student’s ability in multimodal literacy, therefore forming
a better understanding and more critical perspective in detecting and analyzing the underlying purpose of
advertisement, which is prominent in today’s society.
In spite of the findings listed above, several deficiencies are in need of discussion. First, the influence of culture
affects the applicability and authenticity of visual grammar. As Kress and van Leeuwen says, visual grammar is
“quite a general grammar of contemporary visual design in ‘Western’ culture” (Kress & van Leeuwen, 2006, p.3).
The irreversible trend of globalization brought about better understanding and communication among cultures,
however, the differences are not to be overlooked. Similarity, it would be negligent not to take cultural difference
into consideration when borrowing foreign theories in domestic study. Although this study tries to take Chinese
societal status quo into consideration under the framework brought up by Kress and van Leeuwen, a more
comprehensive explanation on Chinese culture and factors influencing domestic consumer’s current buying
behavior can be taken into consideration to better adjust to the very specific context. Apart from that, in the
discussion of compositional meaning, size and placement are two of the decisive factors in determining the
degree of importance. However, the attribution in different society may vary greatly, and the balance between
generalization and specification still need refinement.
Second, the inherent insufficiencies of visual grammar also need improvement. For example, the function of
color is deemed significant in visual grammar and discussed under different meaning. The complexity and
possible overlap in discussion and elaboration might be averted in the future with modification in structure. The
subjectivity in the process of analysis is also a problem in need of discussion. For example, in the determining
the attitude of images, the degree of objectivity formed by different angles can be altered and influenced by other
prominent factors in the very same image.
Third, though the advertisement produced by Tmall serves as a good example in terms of academic research, it is
also possible to expand the research object for a more comprehensive study. And due to the complexity of the
nature of video advertisement, it is unlikely to analyze every frame in detail, thus the most critical ones which
are also less likely to cause bias and confusion are picked for discussion.
Last, due to the limited personal academic capacity, the study only takes into consideration the visual grammar
for analysis. More modes and related theories can be included for future research in the attempt to produce more
valuable academic results.
Acknowledgements
This study was supported by the Key Research Project of Philosophy and Social Science of Ministry of
Education of China (15JZD048).
References
AdMaster. (2015). Double 11 is coming! Consumers have been looking forward two weeks ahead, are you ready?
Retrieved from http://www.admaster.com.cn/?c=downloads&a=view&id=6
Baldry, A., & Thibault, P. (2008). Applications of multimodal concordances. Journal of Language and
Communication Studies, 41, 12-41.
Barthes. R. (1977). Image music text. London: Fontana Press.
Bateman, J. (2014). Using multimodal corpora for empirical research. In C. Jewitt (Ed.), Routledge handbook of
multimodal analysis (pp. 238-252). London: Routledge.
Cook, G. (2001). The discourse of advertising. London: Routledge.
Crawford, M. B. (2015). Attention as a cultural problem. In M. B. Crawford (Ed.), The world beyond your head:
On becoming an individual in an age of distraction (pp. 3-27). Farrar, Straus and Giroux.
Feng, D., & Jing, C. (2011). Spatial metaphor and the construction of multimodal meaning: The case of car
advertisements. Journal of Foreign Languages, 34, 56-61.
167
Feng, D., & O’Halloran, K. (2012). Representing emotion in visual images: A social semiotic approach. Journal
of Pragmatics, 44, 2067-2084. http://dx.doi.org/10.1016/j.pragma.2012.10.003
Forceville, C. (1996). Pictorial metaphor in advertising. London: Routledge.
http://dx.doi.org/10.4324/9780203272305
Forceville, C., & Urios-Aparisi, E. (Eds.). (2009). Multimodal metaphor. Berlin/New York: Mouton de Gruyter.
http://dx.doi.org/10.1515/9783110215366
Geis, M. (1982). The language of television advertising. Salt Lake City: Academic Press.
Goatly, A. (2007). Washing the brain: Metaphor and hidden ideology. Amsterdam: John Benjamins Publishing
Company. http://dx.doi.org/10.1075/dapsac.23
Gu, Y. (2006). Multimodal text analysis: A corpus-based approach to situated discourse. Text and Talk, 26,
127-167. http://dx.doi.org/10.1515/TEXT.2006.007
Gu, Y. (2009). From real-life situated discourse to video-stream data-mining. International Journal of Corpus
Linguistic, 14, 433-466. http://dx.doi.org/10.1075/ijcl.14.4.01gu
Guo, W. (2014). On the construction of legal facts in court discourse: A multimodal perspective. Journal of
Guangdong University of Foreign Studies, 1, 53-57.
Halliday, M. (1994). An introduction to functional grammar. London: Edward Arnold (Publishers) Limited.
Han, S. (2011). A social semiotic approach to the multimodal discourse analysis of advertisements. Chongqing
Normal University, Chongqing, China.
Hood, S. (2011). Body language in face-to-face teaching. In S. Dreyfus, S. Hood, & M. Stenglin, (Eds.), Semiotic
Margins (pp. 31-52). London: Continuum.
Hu, D. (2007a). A social semiotic approach to the multimodal discourse of and advertisement of perfume.
Journal of East China Jiaotong University, 3, 126-130.
Hu, D. (2007b). Multimodal discourse analysis of Emmett Williams’ poem ‘She loves me not’. Foreign
Languages and Their Teaching, 11, 16-19.
Huang, G. (2001). Theory and practice of discourse analysis: A study in advertising discourse. Shanghai:
Shanghai Foreign Education Press.
Ipsos. (2013). Consumer stories during Double 11 Shopping Craze. Marketing Research, 12, 9-13.
Kress, G., & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd Ed.). London:
Routledge.
Knight, N. (2011). The interpersonal semiotics of giving a laugh. In S. Dreyfus, S. Hood, & M. Stenglin (Eds.),
Semiotic Margins (pp. 7-30). London: Continuum.
Leech, G. N. (1966). English in advertising: A linguistic study of advertising in Great Britain. London:
Longman.
Lewis, E. St. E. (1903). Catch-Line and argument. The Book-Keeper, 15, 124.
Lim, F. A. (2011). Systematic Functional Multimodal Analysis Approach to Pedagogical Discourse. National
University of Singapore, Singapore.
Liu, J. (2013). From festive ceremony culture to marketing: Case study of T-Mall “Double Eleven” online
shopping event. Journal of Advertising Study, 2, 84-90.
Luo, Y. (2010). Multimodal discourse analysis of the movie The Da Vince Code. Heilongjiang University, Harbin,
China.
Machin, D. (2007). Introduction to Multimodal Analysis. London: Hodder.
Martinec, R. (2000). Types of processes in action. Semiotica, 130, 243-268.
http://dx.doi.org/10.1515/semi.2000.130.3-4.243
Norris, S. (2004). Analyzing multimodal interaction: A methodological framework. London: Routledge.
O’Halloran, K. (ed.). (2004). Multimodal discourse analysis: Systemic-Functional perspectives. London:
Continuum.
O’Halloran, K. & Smith, B. A. (2012). Multimodal Studies. In K. L. O’Halloran, & B. A. Smith (Ed.),
Multimodal Studies: Exploring Issues and Domains (pp. 1-13). London: Routledge.
168
O’Tool, M. (1994). The language of displayed art. London: Leicester University Press.
Painter, C., Martin, J. R., & Unsworth, L. (2013). Reading visual narratives: Image analysis of children’s picture
books. London: Equinox.
Qian, M. (2014). A comparative analysis of Chinese and American public service advertisements: The
perspective of multimodal discourse analysis. Southwest University, Chongqing, China.
Stenglin, M. (2011). Spaced out: An evolving cartography of a visceral semiotic. In S. Dreyfus, S. Hood, & M.
Stenglin (Eds.), Semiotic Margins (pp. 73-100). London: Continuum.
Suggett, P. (2014). Get to know, and use, AIDA: Attention-Interest-Desire-Action. Retrieved from
http://advertising.about.com/od/successstrategies/a/Get-To-Know-And-Use-Aida.htm
Ventola, E. (2011). Semiotization processes of space. In K.L. O’Halloran, & B. Smith (Eds.), Multimodal Studies
(pp. 220-238). London: Routledge.
Vestergaard, T., & Schroder, K. (1985). The language of advertising. Massachusetts: Blackwell Publishers.
Wang, H. (2007). The construal of interactive meaning in multimodal advertising discourse. Journal of Sichuan
International Studies University, 6, 31-34.
Wang, J. (2012). Multimodal discourse analysis of TV PSAs: A case study of “Welcoming the Olympic Games,
stressing civilization, fostering a new trend”. Henan University, Kaifeng, China.
Williamson, J. (1978). Decoding advertisements: Ideology and meaning in advertising. London: Martin Boyars.
Yang, X., & Gu, H. (2007). Advertising discourse analysis: A possible theory for the ontological study of
advertising. Journal of Jinan University (Philosophy and Social Sciences), 6, 31-34.
Yu, T. (2013). On the meaning construction of visual-verbal beverage advertising discourse. Jiangnan University,
Wuxi, China.
Zhang, J. (2007). English in news publications. Shanghai: Shanghai Foreign Education Press.
Zhang X. (2010). On the Multimodality of the visual poem ‘Me up at does’. Foreign Language Research, 1,
85-90.
Appendices
Appendix 1. Link for Discourse analysis data
Name: Tmall 2014 Double Eleven Advertising (0:00:30)
Produced by: Tmall
Date: 2014
Source URL: https://www.youtube.com/watch?v=DwiVCflrMq4&feature=youtu.be
Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution
license (http://creativecommons.org/licenses/by/3.0/).
169
English Language Teaching; Vol. 11, No. 4; 2018
ISSN 1916-4742 E-ISSN 1916-4750
Published by Canadian Center of Science and Education
Multimodal Discourse Analysis of the Movie Argo

Xu Bo1
1
Binhe Road 778#, Shizhong District, School of Foreign Languages, Leshan Normal University, Leshan,
Sichuan, China
Correspondence: Xu Bo, Binhe Road 778#, Shizhong District, School of Foreign Languages, Leshan Normal
University, Leshan, Sichuan, China. E-mail: 419934369@qq.com
Received: November 4, 2017 Accepted: March 24, 2018 Online Published: March 26, 2018
doi: 10.5539/elt.v11n4p132 URL: http://doi.org/10.5539/elt.v11n4p132
Abstract
Based on multimodal discourse theory, this paper makes a multimodal discourse analysis of some shots in the
movie Argo from the perspective of context of culture, context of situation and meaning of image. Results show
that this movie constructs multimodal discourse through particular context, language and image, and successfully
publicizes western mainstream ideology.
Key words: Argo, movie, multimodal discourse analysis, context
1. Introduction
Multimodal discourse analysis refers to the analysis of various and different semiotic modes in a discourse. The
aim of it is to integrate the representational, interactive and textual meanings achieved by various elements. Its
task is to analyze how these elements work together to form a complete discourse. As a matter of fact, in current
writing, textual structure is realized not only by linguistic signs, but also by layout, color, and typography both at
the level of “clause” and at the level of “discourse” (Wei, 2015).
Multimodal discourse analysis can be done in many ways; content analysis, conversation analysis, and social
semiotic analysis belong to it. And different perspectives can be taken to analyze them. For example, we can
select elements such as layout, modality, color, genre, discourse, style, etc. to do multimodal discourse analysis.
Multimodal discourse analysis is a new and rapidly developing research hot spot in discourse analysis field. It
has become very popular in the western world since 1990s.
Since then, discourse analysis has laid emphasis on linguistic sign the mono-modality. Recently, discourse
analysis has changed its field from static to dynamic multimodal discourse analysis —including sound, language,
image and action. Multimodal discourse refers to using visual, audio, sense multi-lingual symbol system to
communicate through multi-means and symbol recourses such as character, image, sound and action. (Zhang,
2009).
Movie Argo is about an agent Tony Mendez from CIA. He was adept at camouflage technology and designed a
rescue plan in which six American diplomats trapped in Embassy of Canada in Iran were saved. Obviously,
movie belongs to multimodal discourse, thus, the movie Argo can be analyzed from three perspective of
multimodal discourse analysis: context of culture, context of situation and the meaning of image.
2. Literature Review
Scholars both in China and abroad have done researches on multimodal discourse from different aspects. Kress
and Van Leewuen (2006) discussed image and visual design. O’Toole (2010) applied systemic functional mode
to arts, painting, sculpture and architecture symbols. Kay O’Halloran (2011) made a multimodal discourse
analysis of Australian multi-party debate.
Recently, scholars in China have been doing research on multimodal discourse analysis. Linguistic semiotics has
made an immense contribution to the development of general semiotics, but it has met challenges from some
researchers in this domain. Based on Halliday’s theory of language as social signs, social semiotics puts
emphasis on the semiotic practices of a particular community of a particular culture, which pushes forward the
study of multimodal presentations, i.e. the multimodal semiotics. Hu (2007) differentiated multimodal semiotics
from multimedia semiotics; the former was based on the view that all discourses inherently possess the nature of
multimodality; then it went on discussing the role of computer semiotics. He also dealt with some relevant
132
elt.ccsenet.org English Language Teaching Vol. 11, No. 4; 2018
problems, such as semiotic resources as semiotic system, the replacement of books with screens in the designing
of multimodal signs, and coherence realized by discourse participants. Since we live in a new era characterized
by multimodal social semiotics, we should pay attention to the training of one’s multi-literacy.
So far, the discourse analysis has been focusing on language itself, some on the language system and semantic
structure, others on their relation to culture and cognition. Zhu (2007) made an attempt to discuss many issues
relevant to multimodal discourse analysis: the origin, definition, nature, theoretical basis, content, methodology
and significance of multimodal discourse (Zhu, 2007).
Taking advantage of a PowerPoint presentation sponsored by Renmin University’s Australian Studies Center, Hu
and Dong (2006) made an analysis of 23 PowerPoint presentation texts, thinking that PowerPoint presentation is
a very important information communication tool and has become a new genre. Results show that natural
language is still the main modality to convey meaning. The power of PowerPoint presentation lies in application
of the image and sound in conveyance of thematic meaning. It is necessary that we should distinguish
PowerPoint is a tool, a text or a genre.
Ye (2006) interpreted multimodality of hypertext on the basis of people’s website. He made an analysis of a
social semiotic account of multimodal texts, the visual space, states of affairs and the social relations of the
viewer and the image as well.
Based on the theory of systemic functional linguistics, language functions and interactive mode, Huang and Liao
(2008) explored the process of communicative interaction and meaning negotiation in CNN TV interview
program: Larry King Live. Results show that there are three linguistic interactive mode: cooperation,
provocation and avoidance. Xu applied multimodal discourse analysis of the disaster Movie 2012(Xu, 2012).
Tian and Zhang (2013) applied Kress and van Leeuwe’s Visual Grammar to a multimodal discourse analysis of
images on the Olympic torch lighting ceremony. The ideologies hidden in the Chinese and British media was
revealed by them．Study shows that the media agents have expanded its image from representative meaning，
interactional meaning and compositional meaning to different ideologies.
Based on visual grammar in semiotics, Li (2013) interpreted multimodality of advertisement hypertexts,
including multimodality in semiotics, space division in visual discourse, meaning of tourism image
advertisement and social relations between reader and images. It is feasible to apply visual grammar in
advertisement hypertexts, which is good to understand text and significant to improve multi-literacy and
appreciation competence.
The multimodal discourse has been paid much more attention to. People were interested in the multimodality of
discourse long time ago. However, it took a long time for scholars to realize its importance (Dai, 2013).
Although language is a very important resource in discourse，yet it needs to coordinate with other resources in
the process of meaning-making. Therefore, discourse analysis should take other modalities into consideration. In
the near future, multimodal discourse will need more concern from researchers. Zhao (2016) made a multimodal
discourse analysis of movie posters, taking movie Tiny Times as an example.
Dai (2017) discussed the relationship between language，image，motion，posture，sound and other modalities .It
is very important to realize the construction of meaning depending on the coordination of different semiotic
resources. Interactive sociolinguistics and cognitive linguistics pool their researching efforts into the relationship
of different modalities in multimodal discourse．He attempted to probe into and revealed the discourse features
of multimodal discourse，hence, enhanced the understanding and cognition of the relationship between different
modalities.
Researchers at home and abroad has applied more and more modalities into the filed of multimodality, such as
sound and music, science textbook, education research, action and gesture, multi-literacy. Multimodality
discourse analysis has been expanded to field such as newspaper, advertisement, television interview, PPT,
hypertext and movie, etc.
3. Multimodal Discourse Analysis of the Movie Argo
Based on systemic functional grammar put forward by Hallliday (2006), Zhang (2009) formed the theoretical
framework of multimodal discourse analysis, which includes five levels: culture, context, meaning, form and
media. Based on the framework, the author of the paper will make a multimodal discourse analysis of the movie
Argo from the perspective of context of culture, context of situation and meaning of the image.
In the first place, context of culture is realized via ideology and genre, which is an important element in
multimodality communication. Context of culture is so vital that context of situation is dependent on it. In
133
context of culture, Argo can be categorized as Hollywood theme movie in which American pursues liberty and
respects the value of life. Democracy and liberty of American government comes up in the movie, which sets a
good example of American government. In addition, the movie sets a positive image which caters for the United
States, Britain, and Canada through intelligence cooperation and advocates western mainstream ideology. Under
the circumstance, Iran is seen as a villain role, which opposes western world.
Scene one: Burning of American national flag
Splendid shot one: American national flag. In the beginning of the movie, the director tells the origin and
background information of the movie through cartoon, pictures and a series of historical video. American
national flag has shown in the movie for several times, from extreme political action——burning of American
national flag to the flag hung in the laundry, American national flag drawn by the American people, praying for
the American people, American national flag in the Iran embassy. The burning of Iran national flag is also
publicized in the perspective of American discourse privilege. It is obvious that American political space and
discourse power should be advocated in the movie. Burning American national flag, “God bless American”
called out by American people, American people’s support for war advocates a fact that the United States of
America is a democratic political country. In America, people can make comment on politics freely in spite of
any restrictions from institution, for their country is a nation with discourse power in which the society is
harmonious and stable. Any political event will not exert any influence on people’s stable life.
Although Iran in the movie is a country free of political discourse power, people in the country is out of order. In
the beginning of the movie, frame is continually shaking and people in demonstration are ferocious and brutal:
some are tearing Reza’s picture; some are covering American diplomat’s eyes…the scene testifies that only
America is the most democratic and beautiful country. The movie advocates American spirit in a classic way.
Scene two: Six diplomats in security office
134
In the second place, the context of situation is achieved via field of discourse, mode of discourse and tenor of
discourse. The field of discourse of Argo is that agent Tony Mendez designed a recue plan through making a
movie named Argo and successfully brought six American diplomats trapped in Canadian embassy in Iran away.
The tenor of discourse of Argo is realized through dialogues made by Tony Mendez, Jack Donnell, Lester Siegel,
the Central Intelligence agency leader, and John Chambers, Hollywood movie producer. The tenor is achieved
through formal and informal dialogues, mainly formal language, interactive communication or a combination of
formal and informal language.
The mode of the discourse of Argo is a movie discourse which is realized through language, image, sound and
sensory organs and the four elements are supplementary to each other.
Finally, at meaning of image level, field of discourse, mode of discourse and tenor of discourse of the situational
context are achieved through representational meaning, interactive meaning and compositional meaning. One
classic shot is selected here. The time duration of Argo from the beginning to the parade burning American
national flag outside American embassy in Iran, and assault to the embassy is 3 minutes 30 seconds. This movie
clip tells Iran history and the process how Reza Baleway supported by America and Britain is overturned and
sought for asylum in America through a serious of actions: parade’s burning of American national flag and
assault to the embassy through black and white documentary, caricature and colorful video.
Based on Kress and van Leewuen’s (2006) distinction, frame belongs to narrative image. The representational
meaning of the image is embodied by action process, reactional process, and speech and mental process. Action
process are Mohammad Mosaddegh’s nationalizing petrol company held by American and British company,
returning Iran petrol to Iran people; Reza Baleway supported by America and Britain lived on the fat of the land
whereas people had not sufficient food to eat. The whole nation was held down by his brutal rule; Parade people
tore apart the picture of the king and assaulted embassy, etc. Reactional processes include people’s overthrowing
the king, the parade outside the embassy, burning of American national flag, calling for extradition of the king
back to Iran. Shouting and expressions of the parade belong to speech and mental process.
Interactive meaning of the image is mainly achieved through contact, social distance and attitude (Wei, 2015).
Scenes in Argo such as parade people’s burning of American national flag outside American embassy in Iran,
impact on embassy make the audience feel the change of Iran political power. Burning of American national flag
and impact on American embassy in Iran construct a negative image of Iran leader. As for the people in parade,
the movie found the view from far to near or a combination of far view and near view. Burning of American
national flag finds the view from near to far, which closes the distance between actors and audience and
enhances the emotional communication to each other and makes the audience think that political situation in Iran
is in chaos: people in parade detained diplomats at their own will. Attitude expresses a point of view. In visual
communication attitude is about angels or perspectives. High angle is used in parade, which makes audience be
afraid of the parade. For the image of the burning of American national flag at the top of the embassy building,
low angles are used first and then high angles are used too, which makes the audience have a felling of awe. For
the image of people tearing the photo of Reza Baleway, horizontal perspective is used to make the audience feel
that they are personally on the scene.
Compositions, that is, the representational and interactive meanings are combined into a unity of significance.
Information value, salience and framing constitute the compositional meaning of image (Kress & Van Leeuwen,
2006). Different elements are placed in different places, thus conveying specific information, which is called
information value. For instance, given information is put in the left whereas the new information is placed in the
right. The ideal information is placed at the top while the real information is put at the bottom.
When introducing background information about the movie, the photographer placed Iran King and Prime
Minister in the center and the information value is in the center of the picture, which is consistent with the centre
composition principle: in the centre is the important information and around them are the less important elements.
Certain features in a composition comprise salience. In the beginning of the background information, the
introduction to Iran leader, Reza Baleway and the maltreated person are put in the centre of the image, which
makes the audience know the background of Iran leader. Framing devices includes lines or the outlines and so on.
Using these framing devices can separate or relate different elements. The clip is a combination of backdrop and
theme from the perspective of framing. The picture and cartoon in the beginning of the movie clip belongs to
background element, thus black and white color is used here, which make the audience think that it is real
historical fact. People in the parade burning American national flag and assault to American embassy are theme
elements which use chromatic tone, thus making the audience return to the reality.
135
4. Conclusion
In conclusion, based on the theoretical framework of multimodal discourse analysis, the paper made a
multimodal discourse analysis of the movie Argo clip (3 minutes 30 seconds) from the perspective of context of
culture, context of situation and image meaning. Results show that the clip constructs multimodal discourse
through specific context (namely context of culture and context of situation), language, image, and consequently,
advocates western mainstream ideology successfully.
The paper applies multimodal discourse analysis to feature and horror movie, which expands the research field
of multimodal discourse analysis, testifies the feasibility and operability of applying multimodal discourse
analysis to feature and horror movie. At the same time, the paper integrates discourse, movie and television
media which also develops interface research field of linguistics, movie and television media. Hence, it provides
a new research perspective for discourse analysis of movie and television media.
Acknowledgements
This paper is supported by a grant from Sichuan Provincial Department of Education (Project name: Research of
Multimodal Discourse of Chinese and American TV News Magazine; Project number: 17SB0171).
References
Dai, S. L. (2013). The Origin and Development of Multimodal Discourse Analysis. Foreign Language Research,
2, 17-23.
Dai, S. L. (2017). On the Relationship of Different Modalities in Multimodal Discourse. Foreign Language
Research, 6, 1-7.
Halliday, M. A. K. (2006). An Introduction to Functional Grammar (2nd ed.). London: Edward Arnold/ Beijing:
Foreign Language Teaching and Research Press.
Hu, Z. L. (2007). Multimodalization in Social Semiotics. Language Teaching and Research, 1, 1-10.
Hu, Z. L., & Dong, J. (2006). How is Meaning Construed Multimodally-A Case Study of a PowerPoint
Presentation Context .Computer Assisted Foreign Language Education, 6, 3-12.
Huang, G. W., & Liao, H. Q. (2008). Function and Interactive Mode of TV Interview Program, Foreign
Languages Research, 4, 1-9.
Kress, G., & van Leeuwen, T. (2006). Reading Images: The Grammar of Visual Design (2nd ed.). London:
Routledge.
Li, D. Z. (2013). Visual Grammar Analysis of Multimodality Advertisement Hypertext. Foreign Language
Research, 2, 7-11.
O’Halloran, K. L. (2011). Multimodal Discourse Analysis. In K. Hyland, & B. Paltridge (Eds.), Companion to
Discourse (pp. 1-33). London and New York: Continuum.
O'Toole, M. (2010). The Language of Displayed Art (2nd ed). London and New York: Routledge.
Tian, H. L., & Zhang, X. J. (2013). Meanings in Images and Ideologies of Media：A Multimodal Discourse
Analysis, Foreign Language Research, 2, 1-6.
Wei, Q. H. (2015). Studies on Multimodality and Multimodal Discourse in Visual Surroundings. Beijing: Science
Press.
Xu, B. (2011). A Multimodal Discourse Analysis of the Movie 2012.Movie Literature, 9, 81-82.
Ye, Q. C. (2006). A social semiotic analysis of hypertext’s multimodality. Foreign Language Teaching and
Research (bimonthly), 6, 437-442.
Zhang, D. L. (2009). On A Synthetic Theoretical Framework for Multimodal Discourse Analysis .China Foreign
Language, 1, 24-30.
Zhao, C. P. (2016). Multimodal Discourse Analysis of Movie Posters—Taking Tiny Times as an Example, Movie
Review, 12, 110-112.
Zhu, Y. S. (2007). Theory and Methodology of Multimodal Discourse Analysis. Foreign Language Research, 5,
82-86.
136
Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution
license (http://creativecommons.org/licenses/by/4.0/).
137
Alyousef Functional Linguistics (2016) 3:3
DOI 10.1186/s40554-016-0025-1
RESEARCH Open Access
A multimodal discourse analysis of the

textual and logical relations in marketing
texts written by international
undergraduate students
Hesham Suleiman Alyousef
Correspondence:
hesham@ksu.edu.sa Abstract
Department of English Language &
Literature, King Saud University, Thematic progression and the composition of information value contribute to the
Riyadh, Saudi Arabia development of well-structured meaningful text. Discourse-based research of the
textual features has been confined to language learning and workplace contexts,
and in particular advertising and marketing research. However, no published study
has explored and analysed the multimodal textual and the logic-semantic features in
tertiary marketing texts. This case study employed a Systemic Functional Multimodal
Discourse Analysis (SF-MDA) to investigate and compare thematic progression
patterns and composition of information value in five business marketing plan
reports. It also aimed to investigate the logico–semantic expansions between the
visual semiotic resources and the text surrounding them. The SF-MDA of the
marketing plan texts is underpinned by Halliday’s (1994) systemic functional
linguistics, Kress and van Leeuwen’s (1996) composition of information value and
Martinec and Salway’s (2005) system for multimodal discourse analysis of logico–
semantic relations. The findings of the SF-MDA revealed the extensive use of Theme
reiteration pattern, followed by linear thematic progression pattern. Although
multiple-Theme pattern was minimally employed, disciplinary-specific uses of this
pattern in marketing discourse emerged. The findings suggest that the students
have managed professional workplace practices by showing their understandings of
marketing as part of overall business management. A number of other interesting
findings related to textual and logico–semantic relations that exist between the
tables and the graphs and the accompanying text were revealed. The SF-MDA of
informational choices in the marketing texts extends Kress and van Leeuwen’s (1996)
functional interpretations of visual artefacts in terms of compositional zones. As the
intersemiotic logic-semantic relations between graphs and tables and the
orthographic texts surrounding them play a vital role in marketing discourse, tutors
can introduce ways of expanding the meaning-making potential in these semiotic
resources. Other pedagogical implications for the teaching and learning of writing
are presented, particularly in the teaching of English for Business students.
Keywords: Business discourse, Marketing discourse, Composition of information
value, Thematic progression, Systemic Functional Linguistic (SFL), Multimodal
Discourse Analysis (MDA)
© 2016 Alyousef. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International
License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,
provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and
indicate if changes were made.
Alyousef Functional Linguistics (2016) 3:3 Page 2 of 29
Introduction
Thematic progression and the composition of information value contribute to the de-
velopment of well-structured meaningful message, thereby providing cohesion within a
text. Theme is defined as the “point of departure for the message; it is that which lo-
cates and orients the clause within its context” (Halliday and Matthiessen 2014: 64).
Theme identifies what the clause is about, and Rheme is the remaining elements of the
message that develop the Theme. For example, ‘The budget in the sentence “The budget
will increase in each period” is the Theme and the reminder of the sentence is the
Rheme. Students’ awareness of thematic progression patterns helps them produce nat-
urally flowing texts, thereby enhancing their language literacy skills. Academic literacies
are construed in the present case study as set of socially situated multimodal literacy
practices. Halliday’s (1978, 1994) social semiotic approach to language, Systemic Func-
tional Linguistics (hereafter SFL), suits the context of the present study: it views know-
ledge as the product of ongoing situated social literacy practices, contested around the
meaning making processes. SFL sets out the explanation of how humans create mean-
ing of language and various semiotic resources that represent the mode of discourse, a
textual feature that will be discussed next.
SFL postulates language as a meaning making semiotic potential that embodies three
kinds of language metafunctions: ideational oriented towards the field of discourse and
construed by the experiential and the logical meanings, the interpersonal meaning ori-
ented towards the tenor of discourse, and the textual oriented towards the mode of dis-
course. The textual metafunction weaves the ideational and the interpersonal meanings
into a textual whole. Martinec (1998: 162) states that feature selections and structures
of the textual meaning “enable the ideational and interpersonal ones to form the cohe-
sive wholes called phases”. Thus, any stretch of written text can be said to be cohesive
when it realises the ideational, interpersonal, and textual meanings.
The three language metafunctions provide powerful linguistic research tools for a
Systemic Functional Multimodal Discourse Analysis (henceforth SF-MDA) (Alyousef
2013; O’Halloran 2008b, 2009, 2011) of texts: TRANSITIVITY (participants, processes,
and circumstances) and conjunctions, MOOD (speech function) and modality (obliga-
tion and degree of certainty or usuality), and Theme and Information structure and the
non-structural system of Cohesion (reference, substitution and ellipsis). Lea and Street
(2006: 373) argue that multimodal analysis aids in theorising “the multimodal nature of
literacy, and thus of different genres, that students need to master in order to represent
different types of curriculum content for different purposes, and therefore to participate
in different activities”.
Due to space constraints, however, the SF-MDA aims to investigate and compare the
construal of the textual metafunction in two multimodal data sets that comprised three
students’ major assignments and a tutor’s two sample texts on a key topic in the Inte-
grated Marketing Topics course, namely a marketing plan. The textual metafunction is
construed by thematic progression and the composition of information value in the
visual-verbal mode, as they play a major role in the unfolding of multimodal marketing
texts. The SF-MDA also aims to investigate the logico–semantic relations between the
tables and graphs and the text surrounding them. Garzone (2009: 156) points out that
“so far, contributions from linguists specifically dealing with multimodality in business
discourse have been relatively few.” As multimodality in business discourse is an
increasingly growing field of research, Integrated Marketing Topics course suits the aim
of my study since it is one of the foundation courses in the Business of Management
(Marketing) undergraduate program. Moreover, insights gained from these analyses
may be valuable for English for Specific Purposes (ESP) and English for Business Pur-
poses (EBP) courses. The present study is, to the best of my knowledge, the first of its
kind to explore the way international undergraduate business students construct cohe-
sive and logically connected multimodal marketing texts.
Literature review
As the SF-MDA aims to investigate and compare the construal of the textual metafunc-
tion and the logico–semantic relations in the multimodal marketing discourse, it is per-
tinent to provide an overview of Halliday’s (1994) Theme and Information Structure
systems, Kress and van Leeuwen’s (1996) system for the composition of information
value, and Martinec and Salway’s (2005) system for image-text logico–semantic
relations.
Theme and Information Structure are the major structural systems within the textual
metafunction in Halliday’s (1994) SFL approach since they facilitate the development of
a meaningful message, thereby providing cohesion within language. Theme involves
three major systems: choice of marked or unmarked Theme, choice of predicated or
unpredicated Theme, and choice of Theme type.
An unmarked Theme means “the most typical/usual” (Eggins 2007: 318), while a
marked Theme refers to “atypical, unusual” choice whereby the Theme in a declarative
clause is something other than subject. A marked Theme is a variation of the unmarked
whereby focused information is foregrounded. The unmarked Theme conflates with the
MOOD structure constituent- i.e. Subject (in a declarative clause), Finite (in an inter-
rogative), Predicator (in an imperative), or WH (in a WH-interrogative); the marked
Theme conflates with adverbial and prepositional group/phrase to provide circumstan-
tial details about an activity, as in: “in this market plan of Alfa, we will include a finan-
cial performance from the beginning up until period 9”. The italicised Theme in this
example moved to thematic position. Marked Themes add coherence and emphasis to
texts through the use of Theme Predication, which includes thematic and informational
choices. Clauses with predicated Themes always have it + verb be + theme and the non-
Table 1 Theme types

Topical theme Participant
Process
Circumstance
Interpersonal theme Vocatives (e.g., John!, Madam!, …)
Modal or comment adjuncts (e.g., surely, probably, usually, …)
Finite elements (e.g., modal auxiliaries, ‘be’ auxiliary, …)
WH-question words (e.g., why, who, what, where, how)
Textual theme Continuatives (e.g., well, yeah, …)
Conjunctions (e.g., and, or, but)
Conjunctive adjuncts (e.g., then, therefore, because, although, …)
Wh-relatives (e.g., which, who, …)
predicated agnates: e.g. “It was 2 years ago that he quitted smoking”/“He quitted smok-
ing 3 years ago”.
There are three different Theme types (Table 1): topical, interpersonal and textual.
All clauses contain a topical Theme, whereas interpersonal and textual Themes are op-
tional. Topical Themes can be ellipsed in clauses that form a hypotactic relation. A
hypotactic relation is set up when a dependent clause is connected to an independent
clause by a conjunctive device. For example, the following sentence has two topical
themes, “our plant capacity” in the first clause and an ellipsed one in the second, in
addition to the textual theme ‘and’: “Our plant capacity is expected to experience a
growth and will require re-assessment and evaluation of our current manufacturing
capacity”.
The first thematic progression pattern, Theme reiteration (or constant Theme) pat-
tern, reiterates or maintains the Theme focus rather than developing it. Another the-
matic pattern is the linear (or ‘sequential’/‘zig-zag’) pattern, in which information
placed in Rheme position is packaged in a subsequent Theme, as illustrated in Fig. 1.
The cumulative development of Theme in this pattern makes a text cohesive through
the use of newly introduced ideas. The theme in a subsequent clause is picked up from
the Rheme of the previous one. A third form of thematic progression is referred to as
the multiple-Theme (or fan) pattern or split Rheme progression pattern (Fig. 2).
A Rheme may involve a number of different pieces of information, each of which
may be picked up and used as the Themes in subsequent clauses. Theme/Rheme con-
flates with the information focus functions of Given/New.
The system of Information Structure consists of two functional elements, Given (or
Known) and New, that are marked off in speech by tone or a pitch contour. New infor-
mation is typically marked by tonic prominence since it refers to “what is new or un-
predictable” and, therefore, carries the information focus; Given information precedes
New information, and it refers to “what is already known or predictable” (Halliday and
Matthiessen 2014: 89). Given information is shared by the reader/writer or speaker/
hearer. As the present study is concerned with written texts, the phonological indices
of the Information Structure system were not investigated in the present study; instead,
Kress and van Leeuwen’s (1996) system of the composition of information value is
employed in the analysis of the textual organization in tables and graphs.
In his book The Language of Displayed Art, O’Toole (1994) was the first to utilise
SFL in multimodal discourse analysis (MDA). Kress and van Leeuwen (1996) were the
first to outline methods for the analysis of the textual layout in images by proposing
three main systems of meaning-making potential: composition, framing and salience.
The first system is achieved through the principle of information value compositional
zone layout (Fig. 3), which is presented in terms of zones: Centred vs. Polarised, Circu-
lar, Mediator composition, horizontal (or left-right) Given-New and vertical (or up-
down) Ideal-Real compositions (or dimensions).
Fig. 1 Linear (or ‘sequential’/‘zig-zag’) Theme pattern

Fig. 2 Multiple-Theme or split Rheme progression pattern
A Centred composition is perceived by some element(s) appearing in the centre,

whereas a Circular refers to a composition which includes non-central elements that
are spatially spread around the centre. A Mediator composition forms a bridge between
dissimilar non-central elements. The second system, framing, is expressed through vis-
ual framing, whereas the salience system is realised through salience features. The
semiotic potential of visual framing is the separation of elements by frame-lines, pictor-
ial framing devices, empty spaces, and so on. Finally, salience attracts a reader’s atten-
tion through the semiotic resources of position, size, tonal value or colour, sharpness,
and so on (van Leeuwen 2005a). Bateman (2011: 52) presses the need for “more empir-
ically grounded analysis of a broader range of multimodal documents” in order to verify
or disprove Kress and van Leeuwen’s interpretations of the composition of information
value. Following Bateman’s (2011: 52) claim, it is essential to investigate Kress and van
Leeuwen’s functional interpretation of Given-New/ Ideal-Real to find out if these ‘tech-
nical labels’ are capable of bringing “a concrete, readily verbalised meaning to the inter-
pretation” of the marketing multimodal texts.
Fig. 3 The grammar of visual artefacts in terms of compositional zones (Kress and van Leeuwen 1996)
The SF-MDA of the composition of information value in terms of Given-New and

Ideal-Real lends itself to the investigation of the logico-semantic image-text relations
that construe these artefacts. Martinec and Salway (2005: 339) state, systemic functional
semiotics is “the one theoretical framework whose followers have concerned themselves
with [intersemiotic] relations between images and texts”. Martinec and Salway (2005)
system for image–text logico–semantic relations is based on Halliday and Hasan’s
(1976) three main sub-types of expansion in texts: elaboration, in the case when a
mode (image) clarified the other (text), extension, if images added information to texts,
and enhancement, if images prompted information such as how, when, where, or why
the text was taking place. In elaboration relation, the same participants, processes and
circumstances are depicted and referred to. In extension, images add new but related
information is referred to or depicted. Finally, in enhancement relation, related tem-
poral, spatial or causal information is provided. What follows is a review of the litera-
ture which takes SFL and multimodal communication and representation into account.
Discourse-based research of the textual features has been confined to language learn-
ing (Ebrahimi and Ebrahimi 2012; Jalilifar 2010; Medve and Takač 2013; Mellos 2011)
and workplace (Bargiela-Chiappini 2009; Camiciottoli 2010; Clatworthy and Jones
2001; Fitchett and Caruana 2015; Forey 2002; Thomas 1997) contexts, and in particular
advertising and marketing research (Ardley and Quinn 2014; Copley 2010; Kim et al.
2014; Oakes and Oakes 2012). Fitchett and Caruana (2015), for example, reviewed the
development of studies in marketing and consumer research. Thomas (1997) investi-
gated thematic progression in a series of management messages in the annual reports
of a company. The results highlighted significant decrease in the use of the personal
pronoun ‘we’ from 1984 to 1988, along with a corresponding increase in the use of in-
animate nominal groups as themes. Camiciottoli (2010) investigated discourse conjunc-
tive devices in financial disclosure texts. The findings indicated that the pragmatic use
of disclosure influences the interpretation of the message. Similarly, Forey (2002) inves-
tigated the function performed by Theme in workplace texts, which included 30
memos, 22 letters and 10 reports. Marked Theme played a crucial role in representing
the workplace as a depersonalised, material world. The interpersonal Theme was rea-
lised by similar features across the three types of texts. Clatworthy and Jones (2001) in-
vestigated the effect of thematic structure on the extent of readability variability in 60
chairman’s statements. The findings showed that the introduction was systematically
easier to read than other parts of the statement, contradicting prior objective research
on accounting narratives. The results indicated that the thematic structure in the state-
ment was a key driver of the variability of annual report readability.
Some studies conducted discourse analysis and interviews to identify key themes in
marketing workplace discourse. For example, Ardley and Quinn (2014) examined the per-
ceptions of senior marketers in regards to their approaches to the development and im-
plementation of marketing plans. The findings suggest that senior marketers written plans
function as cues and guides, rather than prescriptive actions which stifle creativity and
innovation. A diverse breadth of language and ways of enacting contextual tasks and
accomplishing goals is what characterises marketing activities. This view contrasts with
Lee’s (2005) and Oakes and Oakes (2012) view that the sequential steps in market plan-
ning model create a sense of order. Along similar lines, Copley (2010) conducted a dis-
course analyses of 14 interview exchanges to investigate the ways in which marketing is
seen and conducted in Sales and Marketing Executives (SMEs) by SME principals and
support agency practitioners in England. The findings identified key themes, which indi-
cate the need to include critical marketing issues when considering the nature of SME
marketing, which challenges accepted notions of orthodox and relational SME marketing.
Oakes and Oakes (2012) argue that the rhetorical use of verbal and visual metaphor to en-
hance legitimacy in accounting and marketing suggest a strong link between them.
Whereas multimodal communication research in tertiary contexts has been confined
to mathematics (de Oliveira and Cheng 2011; Guo 2004; O’Halloran 1996, 2000, 2004,
2005, 2008a, 2009), history (North 2005), science and computing (AlHuthali 2007;
Alshammari 2011; Drury et al. 2006; Hsu and Yang 2007; Jones 2006), journalism and
media (Hawes 2015), and nursing (Okawa 2008), SFL-based investigations of tertiary
business discourse were limited to a few studies (Alyousef 2013, 2015a, 2015b; Alyousef
and Alnasser 2015a, b; Alyousef and Mickan in press). Hawes (2015) for example, in-
vestigated and compared thematic progression in professional journalists’ texts with
those of comparable length written by 18 international students in a pre-MA course,
Inter-Communication. The findings suggest teaching the principles of thematic pro-
gression, particularly to non-native speakers of English. Okawa (2008) investigated the
process of constituting academic literacy practices of a Japanese first year nursing stu-
dent. She employed SFL in the analysis of texts to investigate, respectively, discourse at
the epistemological level and texts at the lexico-grammatical level. The findings showed
that discipline-specific knowledge is acquired through socialisation into a particular dis-
cipline. Whereas Alyousef (2013) examined the experiential meanings within the multi-
modal finance texts, Alyousef and Mickan (in press) investigated the ideational
meanings in management accounting texts. Along similar lines, Alyousef and Alnasser
(2015a, b) investigated the use of cohesive devices in a tertiary finance and manage-
ment accounting texts. The findings revealed that lexical cohesion is the most common
resource for cohesion in the two Business texts, and in particular repetition of the same
lexical items, followed by reference. Theme reiteration is used to define the numerical
values in the tables. Alyousef (2015a) employed an SF-MDA of Theme and the com-
position of information value in tertiary finance texts that encompassed tables and
graphs. The findings showed the high frequency of Theme reiteration pattern, the rare
occurrence of linear Theme pattern, and the minimal use of multiple-Theme pattern.
In a similar way, Alyousef ’s (2015b) investigation of management accounting texts re-
vealed the frequency of Theme reiteration and the linear pattern. The present study
aims to contribute to this line of research by investigating a key topic in tertiary multi-
modal marketing texts.
Having presented an overview of the literature relevant to the present study, next I
describe the data and method of analysis.
Data and method of analysis

The corpus is composed of three students’ major assignments (13,664 words) and a tu-
tor’s two sample texts (6949 words) written in English on a key topic in the Integrated
Marketing Topics course, namely a marketing plan. The students were enrolled in the
Business of Management (Marketing) undergraduate program, and they were given the
pseudonyms Nura, Hind, and Zohoy. The five texts were comparable since the main
topic underlying the tasks was similar.
The SF-MDA of the two data sets is based on the theoretical tools of Halliday and Mat-
thiessen’s (2014) Theme system and Kress and van Leeuwen’s (1996) system of the com-
position of information value. As marketing texts are multimodal, I employed Martinec
and Salway’s (2005) system for the investigation of image-text intrasemiotic (within a vis-
ual semiotic mode, a table/graph) and intersemiotic (across a visual semiotic mode and
the text surrounding it) logico–semantic relations. These tools seem to be suitable for the
purpose of this study because they reveal the textual and the logical configurations made
at the intersection of tables and graphs and the accompanying texts.
Following Halliday and Matthiessen (2014), independent clauses in tables and graphs
were numbered and annotated in order to calculate the frequency of occurrence of each
Theme type across the two data sets. I utilised the students’ intuitive understandings or the
intended reading paths (van Leeuwen 2005b) of the graphs and tables in order to transcribe
and annotate the frequency and the nature of Theme patterns and the composition of infor-
mation value. Ellipsed experiential Themes were included in the SF-MDA. Instances of im-
plicit Theme patterns in the tables and graphs were italicised and placed in square brackets,
whereas implicit finites in Rheme position were placed in square brackets. Numerical values
in the tables were annotated as New if they were not provided in the assignment task sheet.
Consideration is given to validity and reliability, which are crucial features of all re-
search. Reliability was achieved through iterative cross-checking of the analyses,
followed by member checking agreement to verify the annotations of thematic patterns.
As for validity, I calculated the percentage for the frequency of occurrence of each
Theme pattern per the total instances of Theme patterns in a text. This was calculated
by dividing the sub-total number of occurrences of each pattern by the total number of
occurrences of the overall Theme patterns and then multiplying this number by 100.
The percentage total adds up to 100 %, equivalent to the total number of Themes. The
use of numerical/quantitative data in this study aims to make statements such as
“higher,” “fewer,” and “most” more precise.
What follows is the findings and the discussion of the SF-MDA of marketing texts.
Results and discussion

This section presents and discusses the findings of the SF-MDA of market planning
texts, including an overview of the context of the study.
An overview of context
Students were required to engage in workplace practices by developing a business market-
ing plan report to show their understanding of marketing, their ability to comprehensively
analyse their environment, and their creative marketing oriented solutions and written
communications. The terms marketing plan and market planning (and more generally
business planning) are used interchangeably to refer to “a technology, a set of techniques
and activities, that assists an organization in achieving an appropriate alignment of external
environment and internal capabilities in order to achieve a desired outcome” (Pulendran
et al. 2003: 478). So businesses need a marketing plan report since it details the analysis,
ideas, and plans, linked to what one wants to do with the company. A marketing plan
report also communicates the substance of the marketing effort (and the professionalism
of its author) to other areas of the organization.
Whereas Nura achieved the highest mark, 81 out of 100, Hind and Zohoy’s marks
were 72 and 75 respectively. The word limit for this individual major assignment is
3000 words. This assignment is allotted 40 % of the total mark for the course. Table 2
outlines the key statistics of the three students’ texts and the tutor’s two sample texts.
This includes word count, and the number of tables and graphs:
All the four participants used tables and graphs in the analyses of their products and to
assist them in making appropriate decisions. Nura and Zohoy’s texts far exceeded the other
texts in terms of the number of words, whereas Nura used only two tables compared to
the other texts. Nura preferred to include most of her tables in the appendices, while pre-
senting and discussing her findings in the text. This may explain the reason for exceeding
the required number of words by 80 %. The five texts encompassed 37 tables and 11
graphs. The total number of tables and graphs in the two data sets was almost the same.
These semiotic visual modes achieve a number of functions, as presented in the SF-MDA.
SF-MDA findings and discussion

Table 3 shows thematic progression frequencies for each pattern and the percentages
equivalent to the total number of thematic units. It compares the use of thematic progres-
sion patterns in the students’ texts and the tutor’s two sample marketing plans, both in
the orthographic texts and the tables and graphs. The SF-MDA of thematic progression
patterns in the five marketing plan texts revealed the extensive use of Theme reiteration
pattern in the two data sets, followed by linear pattern. Multiple-Theme pattern was min-
imally used in the two data sets.
The SF-MDA showed that whereas Theme reiteration mostly occurred in the stu-
dents’ tables and graphs (55.80 %), its occurrence in the tutor’s orthographic texts
(46.45 %) was higher. This may be mainly ascribed to the fact that most of the numer-
ical values in the tutor’s tables were absent. As these semiotic resources were presented
to the students as a sample, it is natural to leave the value cells blank. The aim of the
tutor’s two sample texts is to present to the students the rhetorical structure of a
marketing plan and what is expected from them to write in each section. As a result,
the tutor’s total word count in the tables and the graphs was 65 % below the students
(Table 2). This may perhaps explain the significant difference between the students’
texts and the tutor’s in terms of the use of theme reiteration. Overall, Theme reiteration
comprised over 72 % of thematic progression patterns in the two data sets (Table 3).
This finding is in line with a number of studies (Alyousef 2015a, b; Ebrahimi and
Ebrahimi 2012; Li and Fan 2008) Examples of Theme reiteration pattern are shown in
Table 4 (reiterated experiential Themes are italicised).
Table 2 Key statistics of the three students’ texts and the tutor’s two sample texts
The students The tutor Sub-total
Nura Hind Zohoy Total Text 1 Text 2 Total Students & Tutor
Text word count 4866 2414 4353 11,633 2372 3893 6265 17,898
Visuals word count 553 609 869 2031 269 415 684 2715
Word count 5419 3023 5222 13,664 2641 4308 6949 20,613
Number of tables 2 9 9 20 9 8 17 37
Number of figures 3 1 2 6 2 3 5 11
Alyousef Functional Linguistics (2016) 3:3
Table 3 The frequency and percentage of thematic progression patterns in the three students’ texts and the tutor’s two sample texts
Thematic progression Semiotic Students Tutor Sub-total
type resource
Nura Hind Zohoy Total Text 1 Text 2 Total
Freq % Freq % Freq % Freq % Freq % Freq % Freq % Freq %
Reiteration (or parallel) Text 65 30.80 % 32 16.08 % 49 14.90 % 146 19.76 % 47 52.22 % 25 38.46 % 72 46.45 % 218 24.39 %
Visual 75 35.55 % 118 59.30 % 220 66.87 % 413 55.80 % 5 5.56 % 8 12.31 % 13 8.39 % 426 47.65 %
Text-Visual 140 66.35 % 150 75.38 % 269 81.77 % 559 75.64 % 52 57.78 % 33 50.77 % 85 54.84 % 644 72.04 %
Linear (or zig-zag) Text 64 30.33 % 47 23.62 % 58 17.63 % 169 22.87 % 35 38.88 % 23 35.38 % 58 37.42 % 227 25.39 %
Visual 0 0.00 % 0 0.00 % 0 0.00 % 0 0.00 % 0 0.00 % 0 0.00 % 0 0.00 % 0 0.00 %
Text-Visual 64 30.33 % 47 23.62 % 58 17.63 % 169 22.87 % 35 38.88 % 23 35.38 % 58 37.42 % 227 25.39 %
Multiple-Theme Text 7 3.32 % 2 1.00 % 2 0.60 % 11 1.49 % 3 3.34 % 9 13.85 % 12 7.74 % 23 2.57 %
Visual 0 0.00 % 0 0.00 % 0 0.00 % 0 0.00 % 0 0.00 % 0 0.00 % 0 0.00 % 0 0%
Text-Visual 7 3.32 % 2 1.00 % 2 0.60 % 11 1.49 % 3 3.34 % 9 13.85 % 12 7.74 % 23 2.57 %
Sub-total 211 100 % 199 100 % 329 100 % 739 100 % 90 100 % 65 100 % 155 100 % 894 100 %
Page 10 of 29
Table 4 Examples of theme reiteration pattern in the two data sets

Text Theme Rheme
No.
Textual Interpersonal Topical
Students Nura Alec price strategy is to produce high safety vehicles…
The other strategy is to counter balance with product
development.
Hind The company can research and develop our product
to increase the customer satisfaction.
Our company will develop the capacity of technology
in the style and safety features in our
vehicle.
Amazing car has strong competitive force in the
vehicle market,
Zohoy Currently our total is $164 million,
corporate advertising
budget
but the budget will increase in each period after our
company upgrade our cars.
Tutor Text 1 These active consumers represent a demographic group of
well-educated and successful
individuals;
they are single or married and raising
families.
Text 2 Canterbury Renovations is a powerful marketing tool
understand that the internet
so they are creating a website as an electronic
brochure.
Instead of repeating the clause “Alex price strategy” in the following one, Nura writes
“the other strategy”, using the reference word “other” in order to link this clause with the
previous one, thereby making her text more cohesive. Hind employs the topical Themes
‘Amazing Car’ and ‘company’ interchangeably to refer to the same entity. All the partici-
pants used inanimate nominal groups as topical themes. This finding is in line with a
number of studies (Alyousef 2013; Alyousef and Mickan in press; Iedema 2000; Thomas
1997). Nura and Zohoy used bullet points to list down, respectively, the marketing and
product objectives for Alec and customer groups, thereby making Theme reiterations and
other elements implicit, as shown below (implicit Themes are italicised and placed in
square brackets; implicit finites in Rheme position are placed in square brackets).
The marketing and product objectives for Alec next 4 years (Th) are: (Rh)
[The first objective] [is] Continuing to focus on consumer research.

[The second objective] [is] Aim to increase market share …
[The third objective] [is] Increase Alec technology capability …
[The fourth objective] [is] Increase net margins …
[The fifth objective] [is] Increase contribution margin …
[The sixth objective] [is] To achieve 72 % brand awareness in the economy market
…
[The last objective] [is] Increase inventory days … (Nura’s text)
Currently (Text Th), Efficient Motors vehicles (Th) are being purchased by many
customer groups (Rh); these (Th) predominately include (Rh):
[The first customer group] [are] Families (2F) and high income earners (4F) …
[The second customer group] [are] Singles (3U) and enterprisers (5U) …
[The last customer group] [are] Singles (3T) and value seekers (1T) … (Zohoy’s text)
The topical Theme “marketing and product objectives for Alec” is implicitly reiter-
ated seven times to draw readers’ attention to focal points. Bullet points grammatically
truncate (or encode) ensuing Theme reiterations and the relational identifying or at-
tributive clauses, which link the Rheme with the Theme through the use of verb be.
This finding is in line with Nathan’s (2013) genre-based study of a corpus of 53 market-
ing and marketing management case reports written by native and non-native post-
graduate students at a UK university. These tools could make recall easier and, in turn,
“facilitate the transition from prescription to action” (Chiapello and Fairclough 2002:
198). Although Hind used bullet points in this pattern, she did not employ structural
condensation to encode meanings in the most economical manner.
However, there are many different points also risk to competitors.
– Financial position
The Amazing Company has a greatest research and development expend…
The Amazing Company High sales value which is 24,236 (mill.) …
– Dealer and largest network
The Amazing Company has lowest firm preference percentage …
The Amazing Company has highest number of contribution dealer, …
– Vehicle Class
The Amazing Company is having a good market share value …
Hind did not avail from the powerful means of bullet points, as she preferred to re-
iterate the topical Theme “The Amazing Company” to provide additional information,
rather than focusing on the different pieces of information that are related to the com-
pany- i.e. financial position, dealer and largest network, and vehicle class. Zohoy used
bullet points to present the objectives of his marketing plan, in addition to the imple-
mentation plan for his company Efficient Motor. The tutor used bullet points and
numbered lists in Text 1 to present his marketing plan goals. The findings of the SF-
MDA revealed instances of imperatives in the section entitled “Corporate objectives
and goals”, which are typically treated as themes. The imperative is the only type of
clause in which the Predicator (the verb) is regularly found as unmarked theme (in-
stances of Theme reiteration are italicised).
Increase the sales/deals from $54.6 to $60 by period 8 in the north, east, west and
south. Increase budget for advertising from $210 million to $400 million and
promotion from $100 million to $200 million by period 9. (Hind’s Text)
Increase overall dealer ratings 7 to 75 % by period 8.

Increase firm preference by 4.1 to 22 % by period 9.
Increase MSRP from $20,199 to $20,899.
Increase $5 million in advertising (from $80 million to $85 million). (Zohoy’s Text)
Financial Goals
1. Obtain financing to expand manufacturing capabilities, increase distribution, and
introduce two new product lines.
2. Increase revenues by at least 50 % each year.
3. Donate at least $25,000 a year to conservation organizations. (Tutor’s Text 1)
Nonfinancial goals
4. Introduce two new product lines—customized logo clothing …
5. Enter new geographic markets, …
6. Develop a successful Internet site, …
7. Develop its own conservation program … (Tutor’s Text 1)
The tutor used diversified commands, such as “create a newsletter to go out monthly
to customers”, “offer business card magnets to real estate agents”, and “add a survey to
the website”, which in turn minimizes theme iteration instances. In addition, the tutor
minimally employed imperative clauses in Text 2.
All the two data sets included textual Themes that expand a proposition, con-
junctions that link two clauses through paratactic (equal status) relations or subor-
dinate two clauses through hypotactic (unequal status) interdependency nexus. A
paratactic relation is set up when two or more independent clauses are connected
by conjunctive linking devices (such as and, or, either, neither, but, yet, so, then),
while a hypotactic relation is set up when a dependent clause is connected to an
independent (dominant) clause by a conjunctive binding device (such as when,
while, before, after, until, because, if, although, unless). These two types of relations
are illustrated below (topical Themes are italicised, implicit topical Themes are ita-
licised and placed in square brackets, linking devices are underlined, and binding
devices are in bold):
1- The total sales (Th) were $39,547,000 in the last 5 years, and our current market
share (Th) is 64.3 % (Rh). (Nura’s Text)
2- The economy situation in the next years (Th) shows there is growth in the industry
expected and which (Th) will provide good environment for vehicle sales (Rh).
(Nura’s Text)
3- However, dealer discount (Th) will increase from 12 to 14 % by the end of period 9
as we (Th) believe (Rh) the increase in discount (Th) will increase dealer rating
(Rh). (Zohoy’s Text)
4- We (Th) are not going to increase too much on advertising budget (Rh) in order
[for us] (Th) to be profitable in 1T market (Rh). (Zohoy’s Text)
5- They might (Th) not be experts at the sports (Rh) they (Th) engage in (Rh), but
they (Th) enjoy themselves outdoors (Rh). (Tutor’s Text 1)
6- In just three years, (Th) Blue Sky has built some impressive strengths (Rh) while
[Blue Sky] (Th) looking forward to new opportunities (Rh). (Tutor’s Text 1)
7- Currently, they (Th) are offering bathroom renovations (Rh), but they (Th) plan on
extending into kitchen renovations and extensions as well (Rh). (Tutor’s Text 2)
8- West Homes (Th) is too far away to threaten Canterbury Renovations’ projected
customer base (Rh), but [they] (Th) need to be monitored in case they expand
eastwards (Rh). (Tutor’s Text 2)
Writers use conjunctions in writing to link clauses and to signpost the way for their
readers. Conjunctions tie a message to the immediate context of the preceding clause.
They are, therefore, crucial for expressing the logical relations between the clauses. For
example, the conjunctive devices in examples 1–8 develop on the meaning of another
preceding clause in terms of cause (‘in order to’, and ‘as’), manner (‘while’), addition
(‘and’), and variation (‘but’). The second topical theme ‘for us’ in Zohoy’s second ex-
ample is inserted between square brackets as it is ellipsed and was recovered from the
first one. Anaphoric ellipsed topical Themes occur in clauses that form a hypotactic re-
lation. The subject of the second clause is implicit and it is recovered from the first
sentence. The thematic complement in the tutor’s text (sentence 6) is marked because
the topical theme is not subject, a prepositional phrase functioning as an adjunct: “In
just 3 years, Blue Sky has built some impressive strengths”. It provides orientation or a
contextual frame (Davies 1997) for the message to follow. The marked Theme con-
flated with the prepositional phrase in a declarative clause to provide circumstantial de-
tails about time. As Halliday and Matthiessen (2014) argue, marked Themes have the
potentiality of being subjects since they are nominals but have not been selected sub-
jects; yet they are thematic since they are foregrounded as the Theme. The second top-
ical theme ‘Blue Sky’ in the tutor’s text is ellipsed.
Linear (or zig-zag) and multiple-Theme patterns were confined to the written
texts. The use of the former pattern was higher in the tutor’s text than in the stu-
dents, 37.42 and 22.87 % respectively. Overall linear Theme pattern was the second
most highly occurring pattern in the two data sets (>22 %), compared with
multiple-Theme pattern (<8 %). The high use of this pattern contrasts with Alyou-
sef ’s (2015a, 2015b) studies, <0.60 and <8.50 % respectively. Multiple-Theme pat-
tern was used by the participants to explain and provide further details related to
the market plan. The tutor’s text in the present study more often included in-
stances of this pattern than did the students, 1.49 and 7.74 % respectively. This
highlights the importance of this pattern in marketing plan texts. Although the
rare occurrence of multiple-Theme pattern in the students texts corresponds with
the results in Alyousef ’s (2015a) study of finance texts (<2.50 %), it contrasts with
Alyousef ’s (2015b) study of management accounting texts which lacked the occur-
rence of this pattern. All the two data sets employed non-defining relative clauses
with the pronouns ‘which’, ‘who’, ‘that’ ‘more’ and ‘this’ to give more information
about a proposition (a thing) or a person in the previous clause. These elements
serve two functions: as a marker of some special status of the clause (i.e. textual)
and as an element in the experiential structure (i.e. topical) (Alyousef 2015a). Ex-
amples of linear Theme pattern are shown in Table 5.
The tutor employs a non-defining relative clause with the pronoun ‘more’ to provide
information related to the proposition of adding new logos and slogans. Such clause
types are typically separated from the rest of the sentence by commas, semicolons, or
parentheses. The clause “At the beginning …” is a non-finite hypotactic enhancing
clause acting as a Circumstantial Adjunct.
Table 5 Examples of linear Theme pattern in the two data sets

Text Theme Rheme
No.
Textual Interpersonal Topical
Students Nura Our firm dealer is low compared to competitors as firm C,
rating of 63
which has the highest rate of 69,
Hind Alfa car has achieved some advantages in excess
of competitors
which are creating a different point
that relates to the company.
Zohoy Therefore, Estruck will focus on the 1T segment
which consists of Value Seekers.
Materials cost and are predicted to increase in period 6,
labour cost
which will increase the production cost
and decrease net income
Tutor Text 1 At the there would be ten new logos and five
beginning, new slogans;
more would be added later.
The firm has not would be beneficial to sell through a major
yet determined national chain such as REI or Bass Pro Shops,
whether it
as these outlets could be considered competitors.
Text 2 Canterbury will all wear a work shirt with the business
renovations staff logo on the left pocket.
This will identify them as part of Canterbury
Renovations
Canterbury provides materials such as timber, tiles,
Renovations paint and plaster as part of their
renovations services.
All of the will be of the best available quality
materials selected suitable for the purpose.
The use of multiple-Theme pattern by Nura and the tutor (Text 1) was almost the
same, 3.32 and 3.34 % respectively. The use of this pattern by Hind and Zohoy was 1 %
or below. It is not surprising to find Nura employing these cohesive resources as
she had received the highest mark. The use of multiple-Theme pattern in Nura
text is shown below (information derived from Rheme is underlined, Themes are
in italics).
1- Alec has some problems that need to be taken into account in making
decisions. One of these is the need to keep our high market share and stay
leader in the 2E market. In addition, low capacity production requires
consideration. (Nura’s Text)
2- The reason behind our high performance in the economy market is that the vehicle
satisfies 2E needs better than the competitor in terms of bigger size vehicle, higher
performance horsepower (HP), and high safety. Our safety exceeds competitor by 1
and the quality is the same. Alec is better in safety and the vehicle size is larger and
the performance HP is higher. (Nura’s Text)
3- The main target markets for Alec are 1E and 2E. In 2E market, Alec is doing well in
the market. … 1E is −5 % and 2E demand is 0 %, …. Moreover, 2E is the best
market for Alec to stay in the future, …. (Nura’s Text)
4- I decided to focus on advertising and promotion in order to enhance salience and
generate awareness. Another reason for my focus on advertising is to promote the
new product development and positive image of Alec. Promotions increased from
$30 to $35 million. (Nura’s Text)
5- There are two distinctive assets of Alec which help consumer easily identify offering
and build brand salience. Finally, safety attribute are higher than the competitor. …
Secondly, bigger size vehicle and high HP than competitor. (Nura’s Text)
The prepositional phrase “In 2E market” in Nura’s text (Sentence 3 above) is a

marked Theme, and it provides a contextual frame. Eggins (ibid: 320) argues that “skill-
ful writers and speakers choose marked Themes to add coherence and emphasis to
their text” through the use of Theme Predication, which includes thematic and infor-
mational choices. The tutor used multiple-Theme pattern in order to describe the qual-
ifications of Blue Sky Clothing’s entrepreneurs, Lucy Neuman and Nick Russell, and
their experiences (Excerpt 1). He also used this pattern to discuss the reasons under-
lying his positive outlook for the industry in general and Blue Sky in particular (Excerpt
2), and to present the key features of customized clothing items, which Blue Sky in-
tends to introduce over the next 5 years (Excerpts 3–4).
1- Blue Sky Clothing was founded 3 years ago by entrepreneurs Lucy Neuman and
Nick Russell, Neuman has an undergraduate degree in marketing and worked
for several years in the retail clothing industry. Russell operated an adventure
business called Go West!, which arranges group trips to locations in Wyoming,
Montana, and Idaho, before selling the enterprise to a partner. Neuman and
Russell, who have been friends since college, decided to develop and market a
line of clothing with a unique—yet universal—appeal to outdoor enthusiasts.
(Tutor’s text 1)
2- The outlook for the industry in general—and Blue Sky in particular—is positive for
several reasons. First, consumers are participating in and investing in recreational
activities that are near their homes. Second, consumers are looking for ways to enjoy
their leisure time with friends and family without overspending. Third, consumers
are gaining more confidence in the economy and are willing and able to spend
more. (Tutor’s text 1)
3- While all of the companies listed earlier can be considered competitors, none offers
the kind of trendy, yet practical products provided by Blue Sky—and none carries
the customized logos and slogans that Blue Sky plans to offer in the near future. In
addition, most of these competitors sell performance apparel in high-tech manufac-
tured fabrics. With the exception of the fleece vests and jackets, Blue Sky’s clothing
is made of strictly the highest quality cotton, so it may be worn both on the hiking
trail and around town. Finally, Blue Sky products are offered at moderate prices,
making them affordable in multiple quantities. For instance, a Blue Sky T-shirt
sells for $15.99, compared with a competing high-performance T-shirt that sells
for $29.99. Consumers can easily replace a set of shirts from one season to the
next, picking up the newest colors, without having to think about the purchase.
(Tutor’s text 1)
4- Over the next 5 years, Blue Sky plans to expand the product line to include
customized clothing items. Customers may select a logo that represents their
sport—say rock climbing. Then they can add a slogan to match the logo, such as
“Get over It.” A baseball cap with a bicyclist might bear the slogan, “Take a Spin.”
(Tutor’s text 1)
The tutor used the temporal conjunctive adjuncts (‘first’, ‘second’, and ‘third’) to ex-
pand (or enhance) the meaning in a preceding clause in terms of time. The tutor also
employed multiple-Theme pattern in Text 2 (Appendix) in order to list down, in bullet
points, the assumptions through which the plan was based on, the objectives, and the
recommendations, which included strengths, weaknesses, opportunities, and threats
(SWOT) of Canterbury Renovations Company. This pattern was also used to list key is-
sues from the SWOT analysis and the main findings of competitor analysis. Canterbury
Renovations is employed as the topical Theme in four instances (Appendix) to mention
aspects related to marketing strengths, improvements, ways to overcome threats, and
the main competitors.
Canterbury Renovations (Th) have a number of marketing strengths (Rh):
[The first strength] [is] technical competence of the proprietors

[The second strength] [is] dedication of the staff
[The third strength] [is] total agreement between proprietors on their objectives
[The fourth strength] [is] financial resources
[The fifth strength] [is] good network of contacts for potential clients, suppliers and
tradespersons
[The sixth strength] [is] ability to respond to the needs of the market
[The last strength] [is that] business is located within the target market.
(Tutor’s text 2)
The use of bullet points in multiple-Theme pattern grammatically truncate the

ensuing Themes and the implicit relational identifying or attributive clauses,
which link the Rheme with the Theme through the use of some form of verb be,
such as “[The first point] [is the] lack of management skills” (Appendix, Excerpt
2). The function of the dot points is to avoid repetition of the same or very simi-
lar Themes. Second, the bullet points allow for all listed key features of an aspect
to be understood as Thematic, so each one is foregrounded in Theme position. If
we write the implicit Themes in full then each of these bullet points are back-
grounded (or moved to Rheme position). Finally, the tutor extensively employed
bullet points to encode structural information and relational identifying processes
in the most economical manner. In other words, focusing only on key features of
an aspect “while dispensing with peripheral information where possible” (O’Hal-
loran 1999: 10). Baldry and Thibault (2006: 19) attributed this to the “meaning-
compression principle”, which refers to “the effect of the interaction of smaller-
scale semiotic resources on higher-scalar levels where meaning is observed and
interpreted”. Although multiple-Theme pattern was minimally used in the two
data sets, the tutor’s text more often included instances of this pattern than did
the students, 7.74 and 1.49 % respectively. This indicates that the use of this pat-
tern is one of the key linguistic features of marketing plan texts. Multiple-Theme
pattern is the most difficult one for academic writing students, as they need pack
or list a number of different pieces of information in Rheme position, each of
which are then picked up and used as the Themes in subsequent clauses. Hind
and Zohoy’s minimal use of this pattern in the present study may indicate their
limited knowledge of these cohesive resources and probably their limited writing
opportunities. These factors, however, did not seem to have hindered their
lexico-grammatical choices and the multimodal meaning-making processes in
their assignments, as shown below from their use of this pattern and the good
results they have achieved in this assignment.
1- High-technology has two advantages, one is enhancing our capacities; strength is

reducing some costs, including labor and material costs. (Hind’s Text)
2- The Table 2 is defining the different ≤sic≥ between Alfa and Defy cars which are
both family type car in different company on the market. Defy is the best car for
the 2F customer … Alfa is the second highest vehicle in the 2F customer, … (Hind’s
Text)
3- Two main types of market segment purchase Estruck. They are 1T (Value Seekers)
and 3T (Singles). However, 1T segment occupied most of unit of sales, which was
59 % in period 4 … The 3T segment is a niche market as we only had 8 % unit share
… (Zohoy’s Text)
4- Because Estruck only have two types of customers: 1T (Value Seekers) and 3T
(Singles), we consider focus on the main customers … The Value Seekers Segment is
a large market with 520,000 unit sales … On the other hand, the Single segment has
a smaller market with 377,000 units of sales … (Zohoy’s Text)
The Rheme in the second excerpt “The Table 2 is defining…” in Hind’s text intro-
duced two family car types (‘Alfa’and ‘Defy’), each of which was made Theme in subse-
quent clauses. Similarly, the two Themes “1T segment” and “the 3T segment” in
Zohoy’s text were created from the Rheme in the preceding clause: 1T (Value Seekers)
and 3T (Singles). Students’ language proficiency and their previous academic experi-
ences seemed to have facilitated their use of linear and multiple-Theme patterns
(Jalilifar 2010).
The findings of the SF-MDA also revealed that all the participants employed antici-
patory ‘it’ in Subject position with (be to+) infinitive to give their viewpoints or sug-
gest certain attitude regarding the marketing plan, while remaining in the background
through the use of projecting clauses. Projecting clauses are treated as interpersonal
Themes since they “express a comment on a proposition” or “constitute the propos-
ition itself” (North 2005: 439). Some examples of thematised comment from the cor-
pus are given in Table 6.
The function of projection in the marketing plan texts is to present viewpoints
(“it is expected that the increase in profits and market share will continue for the
next 4 years.”) and to suggest attitudes (“it is important to focus on corporate ad-
vertising to maintain and then increase our preferences”). The comment adjuncts
Table 6 Examples of it-clauses in the two data sets

Text Theme Rheme
Theme Rheme Theme Rheme
Textual Topical Struct. Topical
Students Nura It is predicted that the will be stable
economy
Also, it is required to increase which is low compared
training and support to the competitor’s spending.
budget,
It is expected that the will be bigger size
vehicle with high hot buttons
in period 9.
Also, it ’s important to that theme does not negatively
make sure change impact 1E market.
It is expected that the will continue for the
increase next 4 years.
in profits
and
market
share
Hind and it is important to point
our path from current
period and in the future.
because it is important to achieve
the objectives of Alfa in
the end of periods 9.
Zohoy It can be seen that our stock continued to decrease.
price
According it is predicted that the will increase in
to the material period 6,
Table 3.2.1, cost and
labour
cost
It is important to focus on
corporate advertising to
maintain
and then increase our
preferences
It is easy to link the red
color for our passion
and vision,
Tutor Text 1 even if it is just because they like the new colors.
Text 2 However projections indicate it will be profitable in
subsequent years.
‘predicted’, ‘required’, ‘important’, ‘easy’, ‘just’, and “expected” express comment, and
are in Rheme position since they are part of the verbal groups. This finding con-
trasts with Hyland’s (2005: 14) claim that since academic assessment genres are
“influenced by the dominant ideologies of the genre they are employing”, they lack
the writer-reader equality found in peer-oriented research papers. The participants’
use of grammatical metaphors in it-clauses with extraposed subjects is in line with
Hewings and Hewings’ (2002) findings of similar structures in business studies re-
search articles and students’ essays. What follows is the SF-MDA of the tables and
the graphs.
The SF-MDA of marketing plan tables and graphs and the accompanying text
aim to unpack the elements and the processes (Monteiro and Ainley 2006)
through which students constructed knowledge of Theme, Given-New/Ideal-Real
compositions of information, and the logico-semantic relations that exist between
these multisemiotic resources. Figure 4 includes four instances of Theme reiter-
ation pattern in the graph, as the Theme “revenues” in the clause underlying the
interpretation of the first bar is taken as the Theme of the ensuing ones, thereby
constituting an implicit relational identifying clauses expressed by some form of
the verb be that links the Rheme with the Theme. For example, the first bar can
be interpreted as “Revenues for the period 1 [were] $8000 million”. The Theme
‘revenues’ is repeated four times in the interpretation of periods 2–5.
The Rheme conflates with the Given information instead of New, since the revenues
represent known information. The 5 bars function intrasemiotically (within the graph)
to make meaning. Unlike orthographic texts, the reading path of this graph proceeds
from right to left, rather than the opposite. The values in the vertical axis represent
Given information since they were stated in the task sheet. In addition, the numerical
values that correspond with each period along the horizontal axis do not represent
New information. The graph does not only function intrasemiotically but also interse-
miotically through the interaction of the graph and the accompanying text, as it
prompted Nura in drawing conclusions in the orthographic text. In other words, New
information was inferred from the graph: e.g. in period 4 and 5 “revenue increased
which is a good predictor that Alec is performing well due to the changes and actions
that were taken in periods 4 and 5”. Nura expresses her authorial interpersonal stance,
arguing that the increase is ‘a good predictor’. The graph-text logico intersemiotic se-
mantic relation is that of enhancement since it aided Nura in arriving at New related
information that is qualified circumstantially by the propositional clause: “due to the
changes and actions that were taken in periods 4 and 5”. This clause provides reason
(or purpose) for Nura’s viewpoint. Image-text enhancement relation provides related
temporal (when, how), spatial (where) or causal (Why) information. Liu & O’Halloran
(2009) argue that image–text relations are metafunctionally integrated across experien-
tial, textual and logical meanings at the discourse stratum.
Similarly, the percentage values in Zohoy’s graph (Fig. 5) do not represent New infor-
mation, as they were stated in the task sheet. Besides, the reading path of this graph
proceeds from right to left. The data in the graph aided Zohoy not only in describing
the market share of his truck company Estrruck and the competitor Detonka during
Fig. 4 Nura’s 5 years revenues graph

Fig. 5 Zohoy’s Competitive Analysis (Detonka, Estruck’s competitor)
the past 5 periods, but also in spotting trends and providing interpretive analytical ana-
lyses in the accompanying text that assist in arriving at a conclusion. This conclusion
represents New information that was inferred from the graph (“Hence, we need to in-
crease establishing dealers in these regions to improve our sales and enhance the dealer
relationship”). As O’Halloran (1999: 27) states, “verbal discourse functions to describe
commonsense reality, visual display connects our physiological perceptions to this real-
ity and in combination with metaphorical shifts, creates new entities which are intui-
tively accessible.” This finding exemplifies one of the three ideational complementarity
relations, augmentation, mentioned by Unsworth (2006) and Daly and Unsworth
(2011), where each of the semiotic modes (image/text) provides ideational content that
is additional to and consistent with those provided in the other. The other two relations
are distribution, where the two modes jointly construct similar content, and divergence,
where the ideational content is opposed in the two.
Zohoy established rapport with the readers and showed consideration for his fellow
members through the use of the first person plural pronoun ‘we’ in Theme position.
This finding contrasts with Yeung’s (2007, p. 177) claim that the use of first person pro-
nouns “does not seem to be a defining characteristic of business reports as claimed”. It
also contradicts with Hyland’s (2005: 14) argument that while ‘expert writers’ use per-
sonal pronouns and interjections to claim affinity with audience, students tend to
underuse these features.
All the students applied their conceptual knowledge of marketing in order to con-
struct statistical graphs that encompassed a complex array of elements and processes
(Monteiro and Ainley 2006). The graph in Fig. 6 shows Hind’s analysis of Alfa’s adver-
tising and promotion from period 6 to period 9. Advertising and promotion slopes rep-
resent relational attributive processes that are interpreted in terms of their degree of
steepness (or vertical movement), whether steep upward or steep downward to repre-
sent an increase or decrease respectively. The slopes can be interpreted in natural lan-
guage as “advertising and promotion [Carrier] will be [Pr: Rel, Attrib] gradually
growing [Attribute] from periods 6 to 9”.
The graph includes six instances of Theme reiterations, as the Themes “Alfa’s adver-
tising in period 6” and “Alfa’s promotion in period 6” are taken as the Themes of the
Fig. 6 Hind’s analysis of Alfa’s advertising and promotion from period 6 to period 9
ensuing ones for periods 7–9. Kress and van Leeuwen’s (1996) Ideal-Real compositions
in images and text-images can be applied to the semiotics of marketing plan tables.
The table in the tutor’s text 2 shows the calculations of hourly labour rates (Table 7).
The categories on the left represent the ‘Ideal’ since they are concerned with ab-
stract technical entities- while the numerical values represent the ‘Real’ since they
are concerned with practical or factual details. The tutor comments on the text
following the table: “the hourly rate for estimating work will be charged at $28.00
+ 25 % mark up = $36.50 per hour”. Whereas the clause “the hourly rate for esti-
mating work” represents the Given information, “will be charged at $28.00 + 25 %
mark up = $36.50 per hour” is the New information. The topical Given Themes in
Table 7 are to the left side while the numerical values in the first ten rows repre-
sent Given information. The sum of these values in the last row, “$28.03”, repre-
sents New. The numerical values in marketing plan tables include not only New
information but also historical Given information. The analysis of informational
choices in tables and graphs extends Kress and van Leeuwen’s (1996) approach to
the analysis of visual artefacts in terms of compositional zones. The SF-MDA of
informational content in tables and graphs showed that left-hand and right-hand
English language spatial dimensions do not necessarily correspond with the Given-
New/ Ideal-Real compositions. This finding cannot be generalisable given the fact
Table 7 The tutor’s calculations of hourly labour rates, Text 2

Costs $
Hourly rate (according to award or workplace agreement) $18.00
Add on costs (e.g. WorkCover 5 %) $0.90
Medical costs $1.00
Superannuation 6 % $1.08
Payroll tax (N/A)
Sick pay allowance $2.00
Annual leave allowance ($18×52/48)−$18 $1.50
Tools allowance $1.00
Total $25.48
Allowance for non-productive time, e.g. industrial action, maintenance, travelling, tool maintenance, $2.55
training, injuries etc., e.g. 10 %
Total $28.03
that a few students participated in the present study. They are, however, in line
with Alyousef ’s (2015a, 2015b) and Jones’ (2006) argument that an image or text
occurring on the right side does not necessarily present New visual or verbal infor-
mation. Table 8 includes 32 instances of Theme reiteration pattern, as each Theme
in column 1 is implicitly reiterated four times (periods 6–9).
For example, “sales” in the first column is taken as the Theme of the ensuing ones:
“sales for period 5 will be $8,585,000,000. Sales for period 6 will be $9,014,250,000” and
so on. Therefore, the topical Themes in column 1 function intrasemiotically with the
values in the other columns to facilitate comparability.
The Themes in Table 8, however, do not only function intrasemiotically but also
intersemiotically through the interaction of the table and the accompanying text, which
included two types of information: Given information elicited directly from the table
(i.e. context) and the other representing New related information. In the latter type,
New information is inferred from the table, thereby constituting intersemiotic enhance-
ment relation. In other words, the table helped Nura in drawing conclusions in the
orthographic text, as shown below (Given information in italics, New information in
bold).
our margin is increasing in the next 4 years (Th), reflect efficient operations and
management (Rh). The expenses (Th) are lower than forecasted sales revenues for
coming periods (Rh), which (Th) reflect increase in our income from operation
(Rh). The income increases in the next years, the most obvious cause of that increase
(Th) is lowering operating expenses (Rh).
Rheme conflates with New information in the text above. One notices that the
italicized text clarified the information in the table. This is called an elaborating re-
lation, and it refers to the same participants, processes and circumstances in the
table. The reiteration of a theme in the text accompanying the graphs and the ta-
bles not only provides a strong topical focus by presenting additional Given infor-
mation (extension), but also achieves other functions, including making further
explanations (elaborating), providing interpretive analytical analysis and taking deci-
sions by exhibiting viewpoints that assist in drawing conclusions (enhancing)
Table 8 Nura’s 4 year projections for Amazing Cars Alec Company

Column 1 Period 5 Period 6 Period 7 Period 8 Period 9
Sales $8,585,000,000 $9,014,250,000 $9,464,962,500 $9,938,210,625 $10,435,121,156
COGS $6,334,000,000 $6,524,000,000 $2,442,000,000 $2,442,000,000 $2,442,000,000
Margin $2,251,000,000 $2,490,250,000 $7,022,962,500 $7,496,210,625 $7,993,121,156
Expenses
Marketing $95,000,000 $145,000,000 $145,000,000 $145,000,000 $145,000,000
R&D $1,392,000,000 $1,392,000,000 – – –
General & administrative $988,000,000 $988,000,000 $988,000,000 $988,000,000 $988,000,000
Overhead $639,000,000 $639,000,000 $639,000,000 $639,000,000 $639,000,000
Depreciation $639,000,000 $639,000,000 $639,000,000 $639,000,000 $639,000,000
$3,753,000,000 $3,803,000,000 $2,411,000,000 $2,411,000,000 $2,411,000,000
Income from operation $6,004,000,000 $6,293,250,000 $9,433,962,500 $9,907,210,625 $10,404,121,156
inferred from the graphs and the tables, and which presented meanings that are
entirely new (Alyousef 2015a). As the accompanying text added New information
based on the findings, a logico-semantic relation of extension exist between the
two multimodal discourses and the text accompanying them.
Having presented and discussed the findings of the SF-MDA of the participants’ mar-
keting plan, what follows is a conclusion and the theoretical and the pedagogical impli-
cations of the findings.
Conclusion and implications

The findings suggest that the students have managed professional workplace practices
by showing their understandings of marketing as part of overall business management.
This was achieved by engaging in analyses and planning, linked to what one wants to
do with the company, which in turn could affect them in and beyond the university. All
the three participants successfully completed their Business of Management (Market-
ing) undergraduate program.
The SF-MDA of marketing plan texts showed that they intertwine different thematic
progression patterns. Marketing plan texts include the following key linguistic features
Extensive use of Theme reiteration pattern, followed by the linear Theme

pattern
The use of imperative Themes in the marketing plan objectives and goals
Extensive use of textual Themes that expand a proposition, conjunctions that link
two clauses through paratactic or hypotactic relations
The use of bulleted and numbered lists to facilitate recall and transition from
prescription to action. Bullet points foreground key features of an aspect by
truncating Themes and relational identifying processes, thereby encoding structural
information in the most economical manner
The use of anticipatory it in Subject position to present viewpoints and to
express their attitudes towards particular points through the use of projecting
clauses.
The use of multiple Theme pattern is one of the characteristic features of this genre,
as writers need to persuade their readers to the viability of their plans. This pattern is
the most difficult one for academic writing students because they need to list a number
of different pieces of information. Students’ language proficiency and their previous
academic experiences may have influenced their use of linear and multiple-Theme pat-
terns. Multiple-Theme pattern was mainly used to
Discuss key features of items

Describe the qualifications of entrepreneurs and their experiences
Discuss the reasons underlying positive outlook for the industry in general and the
researched company in particular
List the assumptions through which the plan was based on, objectives, and the
recommendations,
List strengths, weaknesses, opportunities, and threats (SWOT) of the researched
company.
List key issues from SWOT and the main findings of competitor
analysis.
The SF-MDA of the visual semiotics showed marketing plan texts extensively employ
structural condensation in the graphs and tables to encode numerical data in the most
economical manner. Unlike orthographic texts, rhematic statuses in tables and graphs
are contingent on the material value of the message, whether it is known before or not.
Meanings are created in visual semiotic resources not only intrasemiotically but also
intersemiotically through the interaction of the graph and the tables and the text sur-
rounding them.
The participants made meaning through the intersemiotic shifts (or resemiotisation
processes) from diagrammatic and tabular forms to textual. A logico intersemiotic se-
mantic relation of extension exists between the tables and graphs and the text accom-
panying them since the latter provides New information based on the content of the
visual semiotic mode.
This paper contributes to our understanding of thematic progression, the compos-
ition of information value, and the logico-semantic relations in marketing plans. A
number of pedagogical and theoretical implications can be suggested as an outcome of
this research study. The SF-MDA of informational choices in the multimodal busi-
ness artefacts extends Kress and van Leeuwen’s (1996) functional interpretations of
visual artefacts in terms of compositional zones. The SF-MDA of these artefacts in-
dicates that text-based analyses is facilitated by the participants’ intuitive interpret-
ation (or reading path) of the meaning-making processes. As thematic progression
patterning plays a vital role in comprehending a text and in providing a naturally
flowing text, it can be emphasised in the teaching and learning of writing, particu-
larly in ESP/EBP courses. Diversification of the thematic choices can lead to a well
constructed text. ESP/EBP instructors need to give more attention to the process
of writing rather than the product. Tutors can draw students’ awareness to the dif-
ferent thematic choices available for them. This can be followed by class activities
that include identifying and analysing thematic progression patterns. Marketing
texts include a large number of implicit linguistic resources which can be made ex-
plicit in ESP/EBP classes. Students can, through practice, easily control the flow of
their texts. As Butt et al. (2009: 154) argue that students can consciously organize
their texts more effectively once they “explore how patterns of Theme and Rheme
enable the progression of different types of texts”. Moreover, it is essential for
ESP/EBP students to understand and interpret the logical relations between tables
and graphs and the accompanying text. Tutors can introduce the ways of expand-
ing the meaning-making potential in these visual semiotic resources by presenting
samples of elaboration, extension, and enhancement relations. They can start first
with the simplest one, extension, and then move to the others. If these implica-
tions were made explicit, student’s learning experiences and their understanding of
the meaning making resources may be greatly enhanced and, in turn, affect them
in and beyond university.
Finally, the study is limited to five individual marketing plan reports and, there-
fore, the findings are not based on a representative sample of the discipline’s
academia.
Appendix
The tutor’s use of bullet point in multiple-Theme pattern (Text 2)
Information derived from Rheme is underlined, implicit Themes and implicit finites
and other recovered elements are in square brackets:
1. Canterbury Renovations (Th) have a number of marketing strengths

(Rh):
• [The first strength] [is] technical competence of the proprietors
• [The second strength] [is] dedication of the staff
• [The third strength] [is] total agreement between proprietors on theirobjectives
• [The fourth strength] [is] financial resources
• [The fifth strength] [is] good network of contacts for potential clients,suppliers and tradespersons
• [The sixth strength] [is] ability to respond to the needs of the market
• [The last strength] [is that] business is located within the target market.
2. Canterbury Renovations (Th) has identified areas where improvements are required, including (Rh):
• [The first point] [is the] lack of management skills
• [The second point] [is the] small size of showroom premises
• [The third point] [is that there is] no track record in business
• [The fourth point] [is that there is] no current plan for management succession in the short term
• [The fifth point] [is the] inefficient equipment
• [The sixth point] [is the] lack of research and development facilities
• [The seventh point] [is that the] proprietors have limited security with which to raise finance for
future growth and development
• [The seventh point] [is that] business is principally dependent on one person during the formative stages.
3. Canterbury Renovations (Th) will need to be aware of (Rh) and (Textual Th) try to overcome the following
threats (Rh):
• [The first threat] [is] poor reputation of the industry in the areas of quality and reliability
• [The second threat] [is] potential for economic downturns
• [The third threat] [is] existence of competitors within the industry
• [The fourth threat] [is] increasing cost of materials, equipment and subcontractors
• [The fifth threat] [is] difficulties of getting and retaining suitable staff and reliable subcontractors
• [The sixth threat] [is] possible government regulation
• [The seventh threat] [is] geographically diverse market
• [The eighth threat] [is] increased number of DIYs and supported through home TV shows and magazines
• [The last threat] [is] increased interest in travelling and owning new cars rather than spending money on
home improvements.
4. Canterbury Renovations’ main competitors (Th) are West Homes and Balwyn Kitchens (Rh). West Homes
(Th) is too far away to threaten Canterbury Renovations’ projected customer base (Rh), but (Text Th)
needs to be monitored in case (Rh) they (Th) expand eastwards (Rh). Balwyn Kitchens (Th) only renovates
kitchens at this stage (Rh) and (Text Th) have the following strengths and weakness (Rh):
Balwyn Kitchens Strengths (Th):
• [The first strength] [is] established & well known business (Rh)
• [The second strength] [is] featured in Home Beautiful magazine
• [The last strength] [is] two apprentices.
Weaknesses
• [The first weakness] [is] only does kitchen projects
• [The second weakness] [is] not answering emails and minimal contact withstaff during working hours
• [The third weakness] [is] • no website
• [The fourth weakness] [is] • lack of fully qualified staff
• [The fifth weakness] [is] • high overheads with large showroom on a main road
5. From the SWOT (Th), the key opportunities include (Rh):
• [The first key opportunity] [is] providing good customer service that will put Canterbury Renovations
ahead of market perceptions and earn word of mouth advertising
• [The second key opportunity] [is] popularity of renovations in the area
• [The third key opportunity] [is] utilising technology to promote business and keep in contact with staff
• [The last key opportunity] [is] good networks allow good service and joint promotion activities.
6. This plan (Th) is based on the following assumptions (Rh):
• [The first assumption] [is] Balwyn Kitchens only offer kitchen renovations
• [The second assumption] [is] interest rates remain fairly low and steady
• [The third assumption] [is that] the website won’t require much maintenance once it is created
• [The fourth assumption] [is] West Homes won’t expand into the East
• [The last assumption] [is that] materials and equipment costs don’t change much.
Competing interests
The author declares that he has no financial or non-financial competing interests.
Authors’ information
Hesham Suleiman Alyousef is an assistant professor at the Department of English Language and Literature at King
Saud University in Riyadh, Saudi Arabia. He completed his Master’s degree in applied linguistics in 2007 at King Saud
University. He worked as an MA thesis examiner during his Ph.D. candidature and as an academic editor for the Asian
ESP Journal. He has published a number of papers in refereed academic journals, as well as in journals listed in
Thomson’s Institute for Scientific Information (ISI) database. His research interests include systemic functional
linguistics, academic literacies, metadiscourse, multimodal discourse analysis, reading comprehension, and the use of
Web 2.0 technology in higher education. He has more than 25 years of experience in teaching ESL/E FL students.
Acknowledgements
The author expresses his appreciation to the Deanship of Scientific Research at King Saud University and to the
Research Center at the Faculty of Arts for funding the current research study.
Received: 13 November 2015 Accepted: 12 January 2016
References
AlHuthali, Mohammed. 2007. The construction of mechanical engineering literacies: autonomous or social practices?
Australia: University of Adelaide.
Alshammari, Bandar. 2011. A cohesion analysis of scientific papers written by Saudi students in an Australian University.
Adelaide, Australia: University of Adelaide.
Alyousef, Hesham. 2013. An investigation of postgraduate Business students’ multimodal literacy and numeracy
practices in Finance: a multidimensional exploration. Social Semiotics 23(1): 18–46. http://dx.doi.org/10.1080/
10350330.2012.740204.
Alyousef, Hesham. 2015a. A multimodal discourse analysis of international postgraduate business students’ finance
texts: an investigation of theme and information value. Social Semiotics 1–46. http://dx.doi.org/10.1080/10350330.
2015.1124518.
Alyousef, Hesham. 2015b. A study of theme and information structure in postgraduate Business students' multimodal
written texts: a SF-MDA of management accounting texts. In 2015 Asian Conference on Language Learning (ACLL
2015): Proceedings of the International Academic Forum. Kobe, Japan: IAFOR.
Alyousef, Hesham, and Suliman Alnasser. 2015a. A study of cohesion in international postgraduate Business students’
multimodal written texts: an SF-MDA of a key topic in finance. The Buckingham Journal of Language and Linguistics
8: 56–78. http://ubplj.org/index.php/bjll/article/view/1047.
Alyousef, Hesham, and Suliman Alnasser. 2015b. A study of cohesion in international postgraduate students’
multimodal management accounting texts. Arab World English Journal (AWEJ) 6(3): 30–46.
Alyousef, Hesham, and Peter Mickan. in press. Literacy and Numeracy Practices in Postgraduate Management
Accounting. In Multimodality in Higher Education, eds. Arlene Archer, and Esther Breuer. Leiden: Brill Publishing.
Ardley, Barry Charles, and Lee Quinn. 2014. Practitioner accounts and knowledge production: An analysis of three
marketing discourses. Marketing Theory 14(1): 97–118.
Baldry, Anthony, and Paul J. Thibault. 2006. Multimodal transcription and text analysis: a multimedia toolkit and
coursebook. London/Oakville: Equinox.
Bargiela-Chiappini, Francesca. 2009. The handbook of business discourse. Edinburgh: Edinburgh University Press.
Bateman, John. 2011. Multimodality and genre: a foundation for the systematic analysis of multimodal documents. New
York: Palgrave Macmillan.
Butt, David, F. Rhondda, S. Spinks, and C. Yallop. 2009. Using functional grammar: an explorer’s guide, 2nd ed. Australia:
Macmillan.
Camiciottoli, Belinda Crawford. 2010. Discourse connectives in genres of financial disclosure: earnings presentations vs.
earnings releases. Journal of Pragmatics 42(3): 650–663. http://dx.doi.org/10.1016/j.pragma.2009.07.007.
Chiapello, Eve, and Norman Fairclough. 2002. Understanding the new management ideology: a transdisciplinary
contribution from critical discourse analysis and new sociology of capitalism. Discourse & Society 13(2): 185–208.
http://dx.doi.org/10.1177/0957926502013002406.
Clatworthy, Mark, and Michael J. Jones. 2001. The effect of thematic structure on the variability of annual report
readability. Accounting, Auditing & Accountability Journal 14(3): 311–326. http://dx.doi.org/10.1108/
09513570110399890.
Copley, Paul. 2010. Through a discourse analysis lens less darkly: illuminating how SME principals and support agency
practitioners see marketing in SMEs. The Marketing Review 10(4): 353–368.
Daly, Ann, and Len Unsworth. 2011. Analysis and comprehension of multimodal texts. Australian Journal of Language
and Literacy 34(1): 61–80.
Davies, Florence. 1997. Marked Theme as a heuristic for analysing text-type, text and genre. Applied linguistics: theory and
practice in ESP, 45–79.
de Oliveira, Luciana, and Dazhi Cheng. 2011. Language and the multisemiotic nature of mathematics. Reading Matrix:
An International Online Journal 11(3): 255–268.
Drury, Helen, P. O’Carroll, and T. Langrish. 2006. Online approach to teaching report writing in chemical engineering:
implementation and evaluation. International Journal Engineering Education 22(4): 858–867.
Ebrahimi, Seyed Foad, and S.J. Ebrahimi. 2012. Information development in EFL students composition writing. Advances
in Asian Social Science 1(2): 212–217.
Eggins, Suzanne. 2007. An introduction to systemic functional linguistics, 2nd ed. London & New York: Continuum.
Fitchett, James, and R. Caruana. 2015. Exploring the role of discourse in marketing and consumer research. Journal of
Consumer Behaviour 14: 1–12. doi:10.1002/cb.1497.
Forey, Gail. 2002. Aspects of theme and their role in workplace texts. University of Glasgow.
Garzone, Giuliana. 2009. Multimodal analysis. In The handbook of business discourse, ed. F. Bargiela-Chiappini, 155–165.
Edinburgh: Edinburgh University Press.
Guo, Libo. 2004. Multimodality in a biology textbook. In Multimodal discourse analysis: systemic-functional perspectives,
ed. K. O’Halloran, 196–219. London/ New York: Continuum.
Halliday, M.A.K. 1978. Language as social semiotic: the social interpretation of language and meaning. London: Edward Arnold.
Halliday, M.A.K. 1994. An introduction to functional grammar. London/Beijing: Edward Arnold/Foreign Language
Teaching & Research Press.
Halliday, M.A.K., and R. Hasan. 1976. Cohesion in english. London: Longman.
Halliday, M.A.K., and C. Matthiessen. 2014. An introduction to functional grammar. 4th revised edition of Halliday’s
Introduction to Functional Grammar. Aufl: Routledge.
Hawes, Thomas. 2015. Thematic progression in the writing of students and professionals. Ampersand 2: 93–100.
http://dx.doi.org/10.1016/j.amper.2015.06.002.
Hewings, Martin, and Ann Hewings. 2002. “It is interesting to note that…”: a comparative study of anticipatory ‘it’in
student and published writing. English for Specific Purposes 21(4): 367–383.
Hsu, Pei-Ling, and W.-G. Yang. 2007. Print and image integration of science texts and reading comprehension: a
systemic functional linguistics perspective. International Journal of Science and Mathematics Education 5(4): 639–659.
http://dx.doi.org/10.1007/s10763-007-9091-x.
Hyland, Ken. 2005. Patterns of engagement: dialogic features and L2 undergraduate writing. In Analysing academic
writing: contextualized frameworks, ed. L. Ravelli and R. Ellis, 5–23. London: Continuum.
Iedema, Rick. 2000. Bureaucratic planning and resemiotisation. In Discourse and community: doing functional linguistics,
ed. Eija Ventola, 47–69. Tubingen: Gunter Narr Verlag Tubingen.
Jalilifar, Alireza. 2010. Thematization in EFL students’ composition writing and its relation to academic experience. RELC
Journal 41(1): 31–45.
Jones, Janet. 2006. Multiliteracies for academic purposes: a metafunctional exploration of intersemiosis and multimodality
in university textbook and computer-based learning resources in science. University of Sydney.
Kim, Kyongseok, J. Hayes, J. Adam Avant, and L. Reid. 2014. Trends in advertising research: a longitudinal analysis of
leading advertising, marketing, and communication journals, 1980 to 2010. Journal of Advertising 43(3): 296–316.
http://dx.doi.org/10.1080/00913367.2013.857620.
Kress, Gunther, and Theo van Leeuwen. 1996. Reading images: the grammar of visual design. London: Routledge.
Lea, Mary, and Brian Street. 2006. The “academic literacies” model: theory and applications. Theory Into Practice 45(4): 368–377.
Lee, Hye‐Kyung. 2005. Rethinking arts marketing in a changing cultural policy context. International Journal of Nonprofit
and Voluntary Sector Marketing 10(3): 151–164.
Li, Jian, and Xiang-Tao Fan. 2008. Application of patterns of thematic progression to literary text analysis. Journal of
Dalian University 29(4): 59–62.
Liu, Yu, and Kay O'Halloran. 2009. Intersemiotic texture: Analyzing cohesive devices between language and images.
Social Semiotics 19(4): 367-388. http://dx.doi.org/10.1080/10350330903361059.
Martinec, Radan. 1998. Cohesion in action. Semiotica 120(1–2): 161–180.
Martinec, Radan, and Andrew Salway. 2005. A system for image–text relations in new (and old) media. Visual
Communication 4(3): 337–371.
Medve, Vesna Bagarić, and Višnja Pavičić Takač. 2013. The influence of cohesion and coherence on text quality: a cross-
linguistic study of foreign language learners’ written production. In Language in cognition and affect. Springer, 111–131.
Mellos, Vickie. 2011. Coherence in English as a second language undergraduate writing: a theme-rheme analysis. San
Diego: San Diego State University.
Monteiro, Carlos, and Janet Ainley. 2006. Student teachers interpreting media graphs. In Proceedings of the Seventh
International Conference on Teaching Statistics. International Statistical Institute and International Association for
Statistical Education.
Nathan, Philip. 2013. Academic writing in the business school: the genre of the business case report. Journal of English
for Academic Purposes 12(1): 57–68. http://dx.doi.org/10.1016/j.jeap.2012.11.003.
North, Sarah. 2005. Disciplinary variation in the use of theme in undergraduate essays. Applied Linguistics 26(3): 431–452.
http://dx.doi.org/10.1093/applin/ami023.
O’Halloran, Kay. 1996. The discourses of secondary school mathematics. Western Australia: Murdoch University.
O’Halloran, Kay. 1999. Towards a systemic functional analysis of multisemiotic mathematics texts. Semiotica 124(1/2): 1–30.
O’Halloran, Kay. 2000. Classroom discourse in mathematics: a multisemiotic analysis. Linguistics and Education 10(3): 359–388.
O’Halloran, Kay. 2004. On the effectiveness of mathematics. In Perspectives on multimodality, ed. E. Ventola, C. Charles,
and M. Kaltenbacher, 91–118. Amsterdam: John Benjamins Publishing Company.
O’Halloran, Kay. 2005. Mathematical discourse: language, symbolism and visual images. London: Continuum.
O’Halloran, Kay. 2008a. Mathematical and scientific forms of knowledge: a systemic functional multimodal grammatical
approach. In Language, knowledge and pedagogy: functional linguistic and sociological perspectives, ed. F. Christie
and J.R. Martin, 205–236. London: Continuum.
O’Halloran, Kay. 2008b. Systemic functional-multimodal discourse analysis (SF-MDA): constructing ideational meaning
using language and visual imagery. Visual Communication 7(4): 443–475.
O’Halloran, Kay. 2009. Systemic functional multimodal discourse analysis (SF-MDA) approach to mathematics, grammar
and literacy. In Advances in language and education, ed. A. McCabe, M. O’Donnell, and R. Whittaker, 77–102.
London & New York: Continuum.
O’Halloran, Kay. 2011. Multimodal discourse analysis. In Continuum companion to discourse analysis, ed. K. Hyland and B.
Paltridge, 120–137. London & New York: Continuum.
O’Toole, Michael. 1994. The language of displayed art. London: Leicester University Press.
Oakes, Helen, and Steve Oakes. 2012. Accounting and marketing communications in arts engagement: a discourse
analysis. Accounting Forum 36(3): 209–222. doi10.1016/j.accfor.2012.02.002.
Okawa, Toshikazu. 2008. Academic literacies in the discipline of nursing: grammar as a resource for producing texts.
Adelaide: University of Adelaide.
Pulendran, Sue, R. Speed, and R.E. Widing. 2003. Marketing planning, market orientation and business performance.
European Journal of Marketing 37(3/4): 476–497. http://dx.doi.org/10.1108/03090560310459050.
Thomas, Jane. 1997. Discourse in the marketplace: the making of meaning in annual reports. Journal of Business
Communication 34(1): 47–66.
Unsworth, Len. 2006. Towards a metalanguage for multiliteracies education: describing the menaing making resources
of language-image interaction. English Teaching: Practice and Critique 5(1): 55–76.
van Leeuwen, Theo. 2005a. Introducing social semiotics. London: Routledge.
van Leeuwen, Theo. 2005b. Multimodality, genre and design. In Discourse in action: introducing mediated discourse
analysis, ed. S. Norris and R. Jones, 73–94. London: Routledge.
Yeung, L. 2007. In search of commonalities: Some linguistic and rhetorical features of business reports as a genre.
English for specific purposes 26(2): 156-179.
Yu, Liu, and K. O’Halloran. 2009. Intersemiotic texture: analyzing cohesive devices between language and images. Social
Semiotics 19(4): 367–388. http://dx.doi.org/10.1080/10350330903361059.
Submit your manuscript to a

journal and benefit from:
7 Convenient online submission
7 Rigorous peer review
7 Immediate publication on acceptance
7 Open access: articles freely available online
7 High visibility within the field
7 Retaining the copyright to your article
Submit your next manuscript at 7 springeropen.com

Journal of Arts & Humanities
Volume 06, Issue 03, 2017, 59-69
Article Received: 27-02-2017
Accepted: 14-03-2017
Available Online: 21-03-2017
ISSN: 2167-9045 (Print), 2167-9053 (Online)
A Multimodal Discourse Analysis of Advertisements-Based on

Visual Grammar1
Fang Guo2, Xiaowen Feng3
ABSTRACT
In addition to words, the symbols, colors, sculptures, photographs, music, etc. are also frequently
employed by participants to express themselves in communication. Advertising is closely related to
sounds, colors, picture animations and other symbols. This paper aims to present how semiotics acts
effectively to realize the real business purpose to reflect the unique significance of the multimodal
discourse analysis. Based on Visual Grammar, this paper analyzes the 2014 Brazil World Cup
advertisements from the perspective of representational meaning, interactive meaning and
compositional meaning, this research means to prove that different modes within an advertisement
depend on each other and have an interdependent relationship. And these relationships have
different roles in different contexts.
Keywords: Advertisement, Multimodal Discourse Analysis, Visual Grammar, World Cup.

This is an open access article under Creative Commons Attribution 4.0 License.
1. Introduction
With the development of science and media, more and more advertisements come to people’s life. In
order to attract the attention of the viewers, the business companies choose simpler signs to express
their notion of the brand. And other semiotic resources besides language begin to be used more
frequently and widely. The World Cup belongs to people all over the world which is held every four year.
There is no doubt that it can attract the attention of the whole world, and then its language of
advertisements will also be special.
Based on Halliday’s systemic functional grammar, Kress and Van Leeuwen propose the theory of Visual
Grammar believing that every semiotic system has the ability to project the social relations between the
receivers and the speakers (1996). Lim (2004) proposes that the society is full of various semiotic
1 This paper is funded by “Teaching Reform Project of NCEPU (No. 2014JG103) and “Scheme for Master Teacher Cultivation
(No. GJ2014004).
2 School of Foreign Languages, North China Electric Power University.
3 Linxia Tuqiao High School, Gansu Province.
Journal of Arts and Humanities (JAH) 59

Guo and Feng, JAH (2017), Vol. 06, No. 03: 59-69
resources, which create the significance by the cooperation of signs. Badry (2000) discusses the
attributes and functions of scientific technologies, such as individual computer, mobile and so on. Kress
and Van (2001) propose that “The so-called modes include visuals, gestures and actions, form the social
resource modal in the use of process just like the language, therefore, the semiotic resources always
have social meaning. Royce (2001) studies the complementary of different semiotics in multimodal
discourse and the cooperation in the second language teaching. O’Halloran (2004) also studies the
modes in a multimodal discourse, and interprets how these modes act together to reflect its
significance according to systemic functional grammar. Nouri and Shahidy (2004) pay their attention to
the study of the effect of PPT, in which they discuss the relevance of the theory of professor and the
memory of students, and draw a conclusion that the PPT class can help the students improve their
memory and have a good grade. Lim Fei (2007) analyzes the interactions of verbal parts or static parts
between their corresponding culture and context. Chinese linguist Li Zhanzi is the first person who
introduces this theory into China in 2003.
Those studies above have made a great contribution to the knowledge of the multimodal discourse
analysis and have a further enrichment to multimodal discourse analysis theory. However, a multimodal
analysis of World Cup advertisements has never been done previously. This paper will study World Cup
advertisements based on Visual Grammar to explain how representative, interactive and compositional
meanings are realized in World Cup advertisements by proposing a framework of multimodal discourse
analysis of advertisements.
2. An analysis of the world cup advertisements
2.1 Imagination and representational meaning
There are six core ideas containing in Systemic-Functional Grammar, and meta-functional idea is one of
them. It consists of three parts as follows: ideational function, interpersonal function and textual
function. The representational function looks like the ideational function in visual grammar, and it is
transformed from the systemic functional linguistics. As we all know, experience function and logic
function consist of ideational function, the language between them is the expression of subject things
and object things. According to Kress and van Leeuween, representational meaning refers to the
relationship acted between participants, things, animation settings and viewers. Seeing the picture will
inspire the speaker’s imagination.
See the picture below. When people feel happy or excited, they will buy easily this drink to celebrate.
From the back of the participant, we can find that the designer chooses the young people as the center
of this picture, which will easily arouse the people’ desire to join in this activity. The representation
meaning can be expressed by the action process of the young participant in the picture, and its location
is usually the most prominent, which can be shown by the size, contrast, color, and distance. From the
location of the participants, whose back gives us a feeling of imagination, at least we can know they are
happy at this moment. From the color and the spilled coca, we can analyze that the coca drink gives
them a feeling of happiness and sharing the success of the victory. When the actions are linked to the
spilled drinks it can stimulate the desire of drinking.
And thus, the speaker speaks out its endorsement of the

product to achieve a coherent discourse, form a logical
relationship to promote the product sales volume. All of
these are achieved by two processes: narrative process
and conceptual process to realize the meaning
representation.
Figure 1: Coca Cola Advertisement in World Cup.
Narrative process is a statement of actions and things shown in the unstable settings, and it is shown
by speech mode. The action process involves the notion of action, “persons who are moving” and “the

A multimodal discourse analysis of advertisements …
object of action”. Animation shows action mode. They are unstable and easily changed in the
background of careful arrangement.
From the above picture, the action process contains the spilled drink, the participants who are moving
and the coca cola. These elements show the action process. The participants who are moving are the
two young men, and their actions testify the energy and happiness they have. Their back to the viewers
and their actions build a vector. Superficially, it seems like there are no viewers to make a reaction to
them, and their gestures have nothing to do with this advertisement. However, every action must have
the receivers. In this picture, the potential audiences mean the receivers, even if they didn’t take
actions to the viewers. That is to say, the viewers receive this advertisement by a potential aspect. For
example, look at this image as follows:
There is a reaction process. The number 9 is still the

participant who is playing football. He will turn his eyes
into the right front of the football, which is not far from
him. The represented participant in the center of this
advertisement put his eyes into the right direction of the
football. If the process of his behavior is to reflect the Figure 2: “Nick”Advertisement of World Cup
direction of the football in front, his reaction process
located accurately with his attention. That is to say, the analysis of the reaction process is helpful to the
participants in the course of the action, which are the main body and the two kinds of reproduction
process.
Simply speaking, it is more stable for the conceptual process which is a concrete meaning, and it shows
its classification, construction and significance by text mode. Conceptual process contains classification
process and analytical process, and from the two processes, viewers start their association to
experience by themselves with the work of Kress and Van (1996). The difference between them is
whether the vector exists in them or not. Narrative process contains it but the other is not. Therefore,
their true distinction depends on the direction of the viewers.
In that picture, participant in analytical process belongs to the part and overall relationship. Logically,
the relationship of the part and the overall is another description, which means that a group of
participants are subject to the other participants.
In this picture, another representational meaning embodies in the environment. We can judge that it is
a regional environment by the contrast or distance of the foreground and background. The represented
participant is playing the football. The represented participant is closed to the viewers, and it belongs
to the foreground. And if the represented participant is very far from the audiences, it belongs to the
background. Therefore, the represented participant, object, size and the clarity are the main reference
factors. From this picture, we can get that Number 9 is very close to the viewers, and his figure is very
big and clear in this picture, so the represented participant belongs to foreground. And other audiences
behind him belong to background.
In contrast, the outstanding participant in the picture is in the center position and in the front row.
What’s more, the area that he is standing is larger than other factors. Furthermore, this picture almost
shows the whole face or figure of the represented participant who locates in the center position. As for
the other participants, seats, and field is very far from the audiences. And these factors reflect the
background because of their clarity and location. Thus, the foreground and background of the regional
environment are combined together to reflect the representational environment. In general, the
representational meaning is just a functional description of the product. In order to leave a good
impression to the viewers, the representational meaning plays a very important role to propagate its
product. If the advertisements we can see in the TV just have a statement but no pictures, no voice, and
no style, all of us maybe ignore them easily.

Take this advertisement for example. This

advertisement is a good combination of football
and coca, and its screen design surrounded by
many cocas and players. In this advertisement, we
can see that the designer put the football as a
high position, and put the players in the middle,
the below is the drink of coca. It means that if
football player wants to beat the other team, they Figure 3: Coca Cola Advertisement
need drink a bottle of coca, which gives the player more power and more confidence. Football match in
the World Cup is very important to all over the world, and this advertisement must make a prominence
of them. Therefore, they put the football into the top position, which shows the key word of this
advertisement. Furthermore, they put the players in the heart of this advertisements, it shows that
players in the field plays a very important role who determines the failure or success in this match.
Therefore, they put their coca into the low position to propagate their products, which is the basic
material thing. And this is their true destination. All of them together show the basic theme of
expression, and this advertisement of settings, coca, football, and sky are acted together to express its
representational meaning. All of these pictures and current situation can evoke the viewer’s attention
to join the passion of the fans cheering, and the desire of drinking. By this, it increases the value of the
product, and shows its representational meaning.
As shown above, the ‘play’ behind the players is the excitement of football. Based on the experience of
human activities, the coca caused the audience's desire, and led the audience's imagination to buy.
Look at this picture as follows:
From the comprehensive effect, there are several

characteristics embodied in this picture. Firstly, the
objects as being the main factors is significant in the
picture, which is embodied by the foreground, size,
brightness, color, and signs; the broad football field
leaves the viewers a kind of imagination. The cup uses
yellow to show the honor and the playground uses green
to show cool, they act to evoke the players to use this
space to seek the honor. And the seats imply that the Figure 4: China Mobile Advertisements
audiences are looking forward to their success.
Secondly, the overall environment is outstanding; it contains the foreground and the background. In
this advertisement, the dark horse and the World Cup belong to the foreground, which embodies the
theme of this picture. And the seats and field belongs to the background, which are very far from the
viewers. These factors combined to stand out the significance of this picture.
Thirdly, the symbolic meanings are associated with the traditional ideas, which is easy to understand by
people. In this picture, the text “Who is the biggest dark horse in the World Cup” is outstanding in the
whole advertisement. “Who is the biggest dark horse in the World Cup”--- Traditionally, “dark horse”
originally refers to a horse that is unwelcome by all the people in the match, but it makes a great
achievement in the final match as unexpected winner. From then on, the term “dark horse” gradually
refers to a young man or a company will stand out in the near future. In this advertisement, the
designer chooses the dark horse as a marked sign in order to show its potential power in future. What’s
more, the dark horse is also a metaphor of match. It represents the power, energy and vigor of the
competitors. In this picture, the text “Who is the biggest dark horse in the World Cup” is located in the
center of the whole image. “China mobile communications” at the top-left corner helps the reader to
understand the potential meaning of this text. That is to say, “China mobile communications” and
“dark horse” combined together to reflect the potential power of this company. We can get its
symbolic significance by analytical process.

In fact, these features suggest the symbolic significance in many aspects. Although it appears in the
image to show the power of the company, it is always free from the plot. It connected the theme of
World Cup together to express its unique competence by the brightness and the location of the
foreground, which is associated with representational meaning. Take this advertisement for example:
From the perspective of narrative process, this picture

adopts players’ hands as a theme of this advertisement. It
implies a strong strength of people, and expresses the
players in the World Cup cooperated together to seek
victory. And players will pull themselves together to
overcome every difficulty in their battle. When the
audiences get this information by sound, they can sense,
memorize and imagine their experience, and this process
is the results of the audiences observe the outside world Figure 5: Cooperate Advertising in World Cup
and experience the life.
In terms of analytical process, in this advertisement, players create a shape of football by hands,
making the audiences to imagine something. The brain is the basic product of producing opinion.
The two processes shows the action modality by utterances and text, the audiences get the outside
information by listening the utterances, and process the physical information by the brain to contract
the useful things.
And then, by the process of classification, the audiences will easily start from their experience.
Generally, they can make the knowledgeable information become conceptual information by
processing and generalizing the information so that make the concept into semiotics. Thus, the
representational meaning in advertisement can be realized by the process of narrative, analytical and
classification.
2.2 Position and interactive meaning
The interactive meaning is similar to the interpersonal function in visual grammar, and it is transformed
from the systemic functional linguistics. According to the interpersonal function, it holds that the
language is the action of social people doing some meaningful things. It contains three aspects as
follows: contact, social distance, and attitudes.
The designer must put the shared information into the advertisements, it is necessary for the speaker
and the listener to communicate in the voiceless, if so, the boss will easily get his profit and persuade
people to believe his products is better than other products in the same level. Contact includes the
direct contact and indirect contact. And direct contact just tells information to the audience by eye
contact, while the latter means tells information by other ways.
In this picture, eight football stars stand together to

cheer with their victory, who are surrounded by the Pepsi.
The digital letter ”8” is a fortunate letter in China, it
means “a wish for making a fortune”, therefore, the
designer choose 8 football stars to participate this
advertisement. The viewers are attracted by their
cheering or the eye contact and guided them to think it
over this drink, which is a representative drink of victory,
evoked the viewers’ desire to buy it. It is a very important Figure 6: Advertising of Pepsi
meaning to show the value of the product. Thanks to the objects need represented by some semiotics,
so the designer must put the shared information into the advertisements, it is necessary for the speaker
and the listener to communicate in the voiceless.

Utterances belong to the certain text, which are the sentences that can be listened and seen. In this
advertisement, the audiences observe from the participant’s eyes, they will be easy to gain their
identity, just like sup a heart-beating time by themselves. The participants and the audiences can face to
face by eye communication, which can put the audience and the participants together so that it can
strengthen the power of this advertisement. And in this advertisement, the designer put the public
person as the presented participant, it strengthens the attraction of the audiences on the one hand,
and on the other hand, it has a certain effect to the world people.
This picture treats the Pepsi drink as the setting, which means that passion needs drink, needs Pepsi. If
you want to celebrate the victory, the drink of Pepsi is necessary for all the people in the world.
Superficially, it strengthens the effect of World Cup. But it implies the potential products by other ways
naturally. Simply speaking, it looks like an advertisement instantaneously but it belongs to a carefully
purpose of the business. And by eye contact, the viewers are attracted by these settings, and they are
brought by another field to think it over this product. So the business just tells the potential
information by other ways to show the indirect contact.
As for the social distance, according to the position of taking photos of the represented participants, it
contains two parts as follows: far and nearly.
Take this advertisement for example. We can apply the

two parts into the analysis of the advertisements. To
analyze the advertisement from two parts is a very
important method. If the represented participants are far
from the viewers, there always must have other things
together will show to the viewers. Generally speaking, if
the represented participant is near to the viewers, he or Figure 7: Advertising of Cola in World Cup
she must be a representative in a certain area.
If the designers put the represented person away from the audiences or the location of the subjective
participant is very far from the audience, it will form different effect.
In this advertisement, the designers put the nine public players into one advertisement, and put them
into a high position, it implies the participants have a strong background, and arouse the public to
image. Their unique gestures, sweet smiles and rich facial expression can draw the consumer’s
attention so that the designers can get their real destination. Look at this advertisement as follows:
Thanks to the effect of the visual communication, the

audience can easily get the effect from image. From the
perspective of the audiences, this advertisement looks
very close to the audiences, but on the other hand, it
seems that they are very far from the audiences. By
studying the advertisements carefully, it can help to
create a relation between the audiences and the
speakers.
Figure 8: Advertising of “Nick”
In the first place, the public participants give the audience their back, which makes the audience far
from the super football star. And then the audience will pay their attention to other place in this
advertisement, which is another participant who is faced by us.
His serious expression gives the audiences a kind of brave spirit, and next procession he will try his best
to beat every enemy to seek success. Therefore, different perspectives from the audience will produce
different results. All in all, the social distance can be realized by these two parts.

Therefore, different perspectives give the audiences different feelings. The pictures in advertisements
are always linked to the human beings by their mind and psychological activities. It is necessary to
analyze the potential meaning in an advertisement, so that we can study the images and text by a
scientific way.
Attitude contains participant, picture and color. If the

designer chooses the participants who are very famous in
the heart of people, and he or she is a representative
person who is positive in some fields, the boss will easily
get his profit and persuade people to believe that his
products are better than other products in the same level.
In this advertisement, on the behalf of Messi who is a
world football champion joined the team to express the
Figure 9: Brand Advertising in World Cup
spirit of team. Messi who is a representative football
champion stands in the middle of others, and they are
cheering for victory. They can use their rich facial
expressions, confidence and sweet smiles to draw the
viewer’s attention to achieve the boss’s purpose.
Look at this picture as follows: the unique characteristic

of the super football star will be much better than other
person. They represent a new power and positive spirit to
the public. Therefore, it is easier to persuade the
audiences to join them.
The brightness of the picture is the main factor. The Figure 10: Advertising of Cup
designer should choose the attracting picture to grasp
the audience. The color of the advertisement belongs to subjectivity, which is also important to the
whole effect. The designer applies all the sense to show the true part or the potential part in an
advertisement.
Let’s take this picture for example, what we can see in the picture, and what we think there are in the
picture and what’s the meaning of this picture that the designer wants to express. This is the study that
we can explore. If the audiences look at them from the perspective of the white color, it looks like a cup.
However, if the audiences look at them from the perspective of the black color, it shows that two
people are looking at each other face to face. By comparing, we can find the different design in this
picture.
In this picture the striking slogan is “To tank Pepsi, You

have to beat me!” This slogan means “to Pepsi cans, you
have to win me!” The wording contains a provocation but
they are specific to the winner of a taste of it, the use of
attempting to really urge people to buy Pepsi in various
activities to celebrate the victory of the current in order
to achieve real business intentions.
Figure 11: Advertising of Pepsi Cola
The screen symbols modal effective integration of
resources generated by the pictures, colors, text, and two male action stars, handsome screen layout
design to express implied meaning through efficient allocation of resources into a unified symbolic
resources --- Pepsi, to issue a common multimodal communication behaviors, stimulate language exists
implication, in order to achieve a strong influence in the world, with a certain appeal. Participant’s
behavior by verbal action effectively stimulates the viewer feedback, so to achieve the interactive
meaning of words through interactive force.

2.3 Signs and compositional meaning
The compositional meaning is similar to the textual function in systemic functional grammar, and it is
transformed from the systemic functional linguistics. It studies the language in use, and it put the
language and the context together to make the speaker say only the words that are related to the field
of the context in order to get its purpose.
Compositional meaning is a whole expression of advertising discourse, which can be shown in the
screen of TV or anything else. Its meaning is combined by other resources to express the whole
significance. In order to get the connections between the speakers and the audiences, it is necessary to
realize the whole effect of producing meaning. Kress & van Leeuwen proposed that the compositional
meaning contains information value, salience and framing. In their framework, as for the information
value, which contains settings, theme and semiotic. It is a value of the various factors in the picture to
get their unique value. The role of any element in the overall role is determined by if it is placed on the
left or right, middle or edge, or image space above or below.
From the background music and picture, we can imagine the related product. And in terms of the
theme, we can judge from its prominence in the scene. As for the semiotic, we can understand its
meaning in a certain area. Take this advertisement for example, its analysis as follows:
As we all know, any element in the overall role is determined by the place of the left or right, middle or
edge, or image space above or below.
From the perspective of information value, which includes

background, subject content and sign. This advertisement
is structured by the vertical angle, in which the main
elements are the represented participant, football and the
sign“√”.
Figure 12: Advertising of Some Brand Clothes
In this advertisement, the designer chooses black screen as the background to highlight the colorful
one. Furthermore, in terms of its subject content, the top part shows the real purpose of the
designer---Nike, which is famous for their clothes. In this picture, the role of the represented participant
who wears these clothes in the whole role plays a key role by the location of the middle. At last, the
sign “√” of this advertisement offer people a feeling of impression to buy the same clothes. And the
verbal discourse “Final competition” is also a potential origin and competence, with the sign“√” linked
together as a top part offers the theme of this advertisement.
Actually, in this picture, football as the lower part plays

the ideal role, while the true part “√” shows the real
information. In other ways, we can also analyze the
advertisement by horizontal angle. The different
presented participants stand in different gestures and
they have different height, which offers new information
in the advertisement---people can wear their brand
clothes regardless of being fat, short and thin. That is to
say, their clothes are fit for every person who has Figure 13: Clothes Advertising in World Cup
different sizes. The picture on the top part offers the new sign“√”, which belongs to the main theme in
this advertisement. Players with different figures in the middle and the football below are shown in this
picture, which are presented and have a certain information value in the advertisement. Look at this
picture as follows:
In the perspective of its settings, a large number of audiences are important element in this
advertisement. In the perspective of its theme, Nike “Kick out the legend”, and the word “legend” on
behalf of the future, energy and spirit. It gives the audiences an illusion to the future, great hope for

tomorrow.
In the perspective of its semiotic, the outstanding sign of this “√”means Nike, and their clothes are
marked by this semiotic, which means a victory, people who are wearing their clothing means that
every day is a new beginning, every day is a new self. The sign “√” shows the person who is seeking for
success, happy and confidence.
In order to attract the audience’s different attention to a different extent, the term salience is used.
The salience of this advertisement is the represented participant C Luo, who is a best player in
European, his gestures key information stands in the center of this picture offers the audience
imagination. It shows that the participant will make a great action at a crucial moment; His facial
expression, animation and gestures give a feeling of outbreak to audience, and the expected eyes with
the feeling of hope to success. All of these can cause the viewers to have the desire to buy the same
clothes.
Meanwhile, compared with the verbal action of the presented participant, the static background of this
picture also gives the audience a feeling of holding their breath to wait the moment of victory. The
verbal action and the static setting enhance the visual impression of the audiences. In order to get its
intentional purpose, in this advertisement, the text of “Kick out the legend” on the left can enhance its
meaning, and draw the attention of the audiences.
It is shown by segmentation lines for framing to link to the factors of picture. Take this advertisement
for example as follows.
Every country in this picture has the same size, which

shows that every country in the world has equal statue.
This designer promotes the friendship and cooperation of
various countries on the one hand, and on the other hand,
it highlights the 2014 World Cup's prospective. All of
these flags connects together to show the big unity of
the world.
Figure 14: Flags of Advertising in World Cup
Furthermore, the order of the flags is classified in alphabetical order, which reflects the fairness of the
countries. Look at this advertisement as follows:
In terms of the background of this advertisement, there

are many players in this picture, but we cannot know
them because of the distance of the audience and the
participants. However, they also play a key role in this
picture.
From the point of the clothes that the participants wear, Figure 15: Teamwork Advertising in World Cup
we can know they belong to a football team. On behalf of the energy, power, and confidence, they give
the audiences a feeling of the positive energy to seek success.
From the players’ gestures, we can judge that they are very happy and cheer in this picture, and they
are celebrating the success of this victory. The viewers can easily join in this atmosphere, and share the
happiness with the participants together.
From the notable sign, that is the back of the represented participant, we can know the notable sign is
the number 10. Furthermore, we know that Meixi is a very famous football player in all over the world,
who was born in Argentina and Won the championship in many official matches. He is the pride of the
country of Argentina, and nearly every fan likes him because of his extraordinary football skill. Not only
does he win the world championship for Argentina, but also win the honor for himself as the best

player in the world. His creativity and brave can give the people an unbelievable energy and always full
of the powerful and the number 10 always represent this famous football star. When the football fun
sees the number 10, they will easily think of this famous football star Meixi. The designer put carefully
the football players and the unique number 10 together just want to express the cooperation of a team
or a country. The represented participant face to face with other football players, in which it shows the
represented person have a great value in the football players’ mind. So the notable sign gives the
viewers a certain information value by the number 10 player. Therefore, their gestures and the unique
number 10 player are outstanding in this picture, which has a crucial significance.
Superficially, this advertisement has nothing to do with the viewers. However, when we analyze this
picture carefully by multimodal discourse analysis, we can find it contains more information, and all the
elements at least plays very important role in expressing the significance of the modes.
At the same time, in terms of the verbal gesture of the represented participant, we can judge that it is a
reaction to the football players. And the represented participant reaction represents a modest and
gratitude to the football players cheering. The public football star as a key person stands a key role in
people’s heart, and the designer chooses him can convince the viewers.
All in all, the verbal gestures of the players and the reaction of the represented participant represent a
kind of vitality. The dynamic background of this advertisement shows a sense of honor, victory and
confidence, and these elements delegate a country’s image. It will strengthen the visual impression of a
country and attract the interest of this country.
From the above we have analyzed, the information value and the notable sign in this advertisement
connect all the factors together to show certain significance, which has a compositional meaning in this
advertisement.
Take this advertisement as an example, every country in

this picture has the same sign “football”, which is the
notable sign in this picture. Generally speaking, the
notable sign always imply a key theme in the
advertisement. Therefore, there is no doubt that this key
elements imply the great value of this picture, and the
notable sign “football” shows the theme of the World
Cup advertising, which represents the information value Figure 16: Signs of Advertising in World Cup
through key factors.
In the first place, the designer puts Brasil into the first position to show the place of World Cup is in
Brasil in 2014. And then other countries behind Brasil have the same size to express the cooperation of
various countries. In the second place, every country has its unique style, such as color, brightness, and
the marketable signs. On the one hand, it shows that every country is seeking common grounds while
remaining differences. On the other hand, it shows different countries have the same interest while
seeking friendship first. In this advertisement, the information value is expressed by the marketable
signs. And thus, this picture enhances the key point of this World Cup.
As we have analyzed above, the shared elements connected together to form the frame of the
advertisement, it is carefully listed by the designer.
3. Conclusion
Guided by the theory of multimodal discourse analysis and influenced by the systemic functional
grammar of Halliday, the writer has set up a new framework of analyzing the classic advertisement.
According to the proposed framework, the writer relates the meaning of the modes and their relations
together to analyze the classic advertisement in 2014 successfully. The main findings can be

summarized as follows: (1) As for the imagination affected by the representational meaning, the
relations of participants and objects can be reflected by analyzing the picture. And the settings can be
shown by the size, contrast, color and distance. An advertisement can be expressed by the speech
mode and text mode, which embodies the respective process to get the association of the
representational meaning. Through analysis, the writer can help the audience have a better
understanding of the advertisement. Furthermore, the writer has testified the feasibility of this new
analytical framework. (2) In terms of the position depended by the interactive meaning, it includes
three parts to finish its whole meaning as follows: location, touch and feelings. According to the
distance of far or near, we can easily judge the importance of the objects. When the listeners meet the
related objects, it will express the information by many ways. Thus, it can be expressed by the direct
and indirect aspect. Thus, it can greatly improve the understanding of the classic advertisement. (3)
When it comes to the signs created by the compositional meaning, we can easily find the important
information by seeing the notable signs in the picture. The writer puts the background, theme and
symbol carefully into an advertisement in order to express the value of the business product. Only if the
people see the signs, will they quickly know what they want to express.
In conclusion, this research is meaningful and it is a new potential field to have a further explore and to
make a systemic analysis for advertisements.
References
Baldy, A. (ed.). (2000). Multimodality and Multimediality in the Distance Learning Age. Italy: Palladino
Editore.
Halliday, Michael Alexander Kirkwood (1994). An Introduction to Functional Grammar (2nd) . London:
Edward Arnold.
Kress G, Van Leeuwen T. (2001).Multimodal Discourse--The Modes and Media of Contemporary
Communication. London: Arnold.
Kress, G. and T. Van Leeuwen (1996).Reading images: The Grammar of Visual Design London: Routledge
Press, 69-160.
Li, Zhanzi. (2003). Analysis of Multimodality Discourse of Social Semiotics. Beijing: Foreign Language
Researches（5）,1-8.
Lim, V. F. (2004). Developing an integrative multisemiotic model. In O’ Halloran, K. L. (ed.) Multimodal
Discourse Analysis: Systemic- Functional Perspectives. London Continuurn. 220- 246.
Lim Fei, Victor. (2007). The visual semantics stratum: making meaning in sequential images. In: Royce,
T.D., Bowcher, W.L. (Eds.), New Directions in the Analysis of Multimodal Discourse. Lawrence
Erlbaum, Mahwah, pp. 195–213.
Nouri, H. & Shahidy. (2005). The effect of PowerPoint presentations on student learning and attitudes.
Global Perspectives on Accounting Education (2) 1.
O'Halloran, K. L. (ed.). (2004). Multimodal Discourse Analysis: Systemic 一 Functional Perspectives.
London: Continuum.
Royce, T. (2002). Multimodality in the TESOL classroom: Exploring visual-verbal synergy. TESOL
QUARTERLY, 36(2): 191-205.

Yearbook of the Poznań Linguistic Meeting 2 (2016), pp. 141–165
DOI: 10.1515/yplm-2016-0007
Concepts in multimodal discourse analysis

with examples from video conferencing
Sigrid Norris
Auckland University of Technology
sigrid.norris@aut.ac.nz
Abstract
This article presents theoretical concepts and methodological tools from multimodal (in-
ter)action analysis that allow the reader to gain new insight into the study of discourse
and interaction. The data for this article comes from a video ethnographic study (with
emphasis on the video data) of 17 New Zealand families (inter)acting with family mem-
bers via skype or facetime across the globe. In all, 84 social actors participated in the
study, ranging in age from infant to 84 years old. The analysis part of the project, with
data collected between December 2014 and December 2015, is ongoing. The data pre-
sented here was collected in December 2014 and has gone through various stages of
analysis, ranging from general, intermediate to micro analysis.
Using the various methodological tools and emphasising the notion of mediation,
the article demonstrates how a New Zealand participant first pays focused attention to
his engagement in the research project. He then performs a semantic/pragmatic means,
indicating a shift in his focused attention. Here, it is demonstrated that a new focus
builds up incrementally: As the participant begins to focus on the skype (inter)action
with his sister and nieces, modal density increases and he establishes an emotive close-
ness. At this point, the technology that mediates the interaction is only a mundane as-
pect, taken for granted by the participants.
Keywords: human–computer interaction; language and interaction; mediation; multi-

modal discourse analysis; multimodal (inter)action analysis.
1. Introduction1
Multimodal (inter)action analysis (Norris 2004, 2009, 2011a, 2011b, 2013) is a

theory of human communication with an abundance of methodological tools to
1
This project is conducted by the Multimodal Research Centre at Auckland University of Tech-
nology, New Zealand with Sigrid Norris as PI; Jarret Geenen, Madeline Henry, Keely Kidner, Ewa
Kusmierczyk, and Jesse Pirini as Researchers. Data collection and partial analysis has been co-
142 S. Norris
empirically investigate interaction. Growing out of applied linguistics, anthropo-

*
logical linguistics, sociolinguistics, discourse analysis, and socio-cultural psy-

chology (Goffman 1963, 1974; Gumperz 1982; Tannen 1984; Schiffrin 1987;
Hamilton 1998; Scollon 1997; 1998; 2001; van Lier 1996; Wertsch 1998;
Wodak 1989) and strongly influenced by social semiotic thought (van Leeuwen
1999; Kress 2000; Kress and Van Leeuwen 1998, 2001), multimodal (in-
ter)action analysis (Norris 2004, 2011a) is a multimodal discourse approach.
Whereas some scholars in applied linguistics (Shohamy and Gorter 2008),
pragmatics (Herring et al. 2013) and sociolinguistics (Bucholtz and Hall, forth-
coming) are calling for or are incorporating multimodal data, this article offers a
novel framework (Norris 2004, 2011) that opens up the study of discourse and
interaction in vastly different ways than does the mere inclusion of multimodal
data into a linguistic study.
Multimodal (inter)action analysis (Norris 2004, 2011a) differs in substan-
tial ways from most other discursive approaches as well as from other multi-
modal approaches: In multimodal (inter)action analysis, language and other
modes are not viewed as phenomena that exist outside of the individual to be
studied as entities in and by themselves. Rather, multimodal (inter)action anal-
ysis champions to investigate language and other modes as part of the individ-
uals in the world and thus, more accurately, as part of the action that the indi-
viduals perform with others, the environment, and objects within. Certainly, no
one will disagree with the fact that language and other modes are part of indi-
viduals or disagree with the fact that humans are a part of their socio-cultural
world acting in and with it. But linguistic theories as well as other multimodal
theories fall short of explanatory tools that allow for the analysis of exactly
how social actors, world and objects connect. Too often we read that language
constructs the social at the same time as language is constructed by the social
(Schiffrin 1994) and while this is certainly true, the question remains: How do
we analyse this fact in detail?
Multimodal (inter)action analysis, based in an understanding of mediation
as advocated by Wertsch (1998) and Scollon (1998, 2001), builds a framework
for such detailed analysis. In this framework, every action is taken to be mediat-
ed in multiple ways.
*
funded by The Faculty of Design and Creative Technologies, The School of Communication Stud-
ies, and The Digital Mobility Research Group of the New Zealand Work Research Institute, Auck-
land University of Technology, New Zealand. We would like to thank all of the participants for
their involvement in this research project.
Concepts in multimodal discourse analysis 143
A mediated action focuses on two elements: the agent and the media-
tional means, emphasizing an inherent irreducible tension between the
two.
(Norris and Jones, 2005: 17)
All actions are thus mediated because social actor(s) always act with or through
mediational means/cultural tools (Wertsch 1998; Scollon 1998). The notion of
mediated action makes the concept of mediation, psychological as well as phys-
ically embodied and physically through objects and the environment, a highly
important concept. Through the underlying concept of mediation in all respects
of action, the framework allows for the simultaneous theoretical inclusion of so-
cial actors and their psychological make-up, objects, and the environment. The
notion of mediation in this framework facilitates the resolution of differences
between human actors, the things they use, and the world that they inhabit (Nor-
ris 2013). Thus, in multimodal (inter)action analysis, the notion of mediation is
a theoretical concept that allows for the theoretically comprehensively bringing
together of cognitive and socio-psychological, embodied physical, object physi-
cal, and environmental physical aspects into one framework. Through the inclu-
sion of all of these facets, the theoretical framework embraces the complexity of
interaction. In order to analyse this complexity in practical terms, various meth-
odological tools have been developed (Norris 2004, 2009, 2011a, 2014, forth-
coming; Geenen 2013; Makboon 2015; Pirini 2016), taking the study of interac-
tion and language in use to a deeper level.
This article explicates some key concepts and methodological tools, by il-
lustrating these through examples from a large-scale study of 17 New Zealand
families (84 individuals in age from infant to 84 years old) interacting via video-
conferencing technology with family members across the globe, using either
skype or facetime. During the research sessions, New Zealand families used a
researcher-provided laptop that recorded the online interactions. A stationary
video camera positioned in the New Zealand participants’ home recorded the
video conferencing interactions from an external point of view, and one to three
researchers (depending on availability) were present, observing the interactions
and/or taking fieldnotes. The data was/is then logged according to the steps out-
lined in Norris (forthcoming) and is currently being analysed using multimodal
(inter)action analysis (Norris 2004, 2011a), building upon general philosophical
and theoretical concepts as exemplified below. Data analysis is still ongoing, but
the data for this article, one of the first interactions recorded, has gone through
all of these steps of analysis.
144 S. Norris
1.1. General philosophical and theoretical concepts
The usefulness of Merleau-Ponty’s (1962, 1963) philosophical point of view,

which states that the human being is a part of the world acting in and with it,
erasing the internal/external duality is particularly evident when examining hu-
man-computer interactions. Figure 1, for example, shows a moment where Mic,
a New Zealand participant in our study, (inter)acts with his environment and the
objects within. Here, the left side of the image shows the larger part of Mic’s
computer screen, the top right illustrates a different part of his computer screen
(where he will later see his own image), and the bottom right shows Mic from a
video camera positioned on a tripod in his home. Mic’s right hand is placed on
the touchpad of the computer and his right middle finger has just pushed onto it
as he is attempting to re-connect with family members in Australia.
Figure 1. (Inter)acting with an object.
Without his (inter)action with the objects, the computer and touchpad, he would
not be able to establish a new connection in order to then (inter)act with his sis-
ter and her children. But besides handling the object, he also (inter)acts with his
environment in other important ways. Figure 2 illustrates the very next moment,
when the connection is being established.
Figure 2. (Inter)acting with the environment and objects within.
In Figure 2, we see Mic gazing at the screen. Here, we observe him on the left
in Figure 2 (circled in red) as he sees himself on screen and to the right, we see
him sitting at the desk from the in-room camera view. He is sitting similarly as
in Figure 1, but here his body shows a slightly more relaxed position with his
right hand now placed on his right leg. His proxemics to the computer screen
are about the same as in Figure 1, which is close enough for him to easily ma-
nipulate the computer mouse and keypad, and also far enough away to leisurely
watch and be seen on screen. His facial expression that is visible on screen (left
in Figure 2 circled in light green), is happy and relaxed. All of his embodied
modes express his waiting and anticipation of the new connection to be estab-
lished at the same time as the computer makes a ringing sound indicating the
call to Australia and showing a waiting signal as droplets are moving towards
the name of the call recipient, both of which Mic appears to be watching.
Soon, the receiver has taken the call (Figure 3), the ringing stops and an im-
age appears in its place. Here, in Figure 3, we see the connection being made on
the left of the image, the screenshot of the participant as he sees himself is now
visible top right, and the in-room camera view of the participant is again located
at the bottom right. However, the connection is not quite established yet, and we
see Mic’s face has changed from a full smile a moment earlier (Figure 2) to a
slight worry as the connection might fail at this point.
146 S. Norris
Figure 3: A possible point of failure.
Here in Figure 3, it becomes highly evident that human beings, as Bateson

(1972) pointed out, are ecologically interdependent with as well as dependent
upon the environment. Only if the connection becomes established, will an (in-
ter)action between brother and sister (or uncle, nieces and nephew) unfold. The
awareness of his dependency on technology that goes beyond computer and
software, which is taken for-granted and is largely ubiquitous as soon as a work-
ing connection is established, is here in Figure 3 present and visible in the par-
ticipant’s facial expression.
These examples illustrate the notion of social actors as part of the world,
acting in and with it. In multimodal (inter)action analysis social actors are, oth-
er than in actor network theory (Latour 2005), always and only humans. The
computer in the above examples is a cultural tool/mediational means, and so are
the software and the many hidden technologies that make the connection be-
tween social actors possible.
Mediation is a term that is often used in regards to technology as computer
mediated communication regards any kind of communication mediated by one
or more technological devices. In multimodal (inter)action analysis however,
technology in the above example is only one aspect of mediation: For example,
as Mic operates the touchpad (Figure 1), he utilises the cultural tools (laptop,
skype, broadband connection, and other ubiquitous technologies) in order to

connect to his family in Australia. Clearly, this is the kind of mediation that
many researchers have in mind when speaking of computer mediated interac-
tion. However, as we will see below, multimodal (inter)action analysis with its
roots in mediated discourse analysis, takes mediation as theoretically much
more important than other frameworks.
2. Multimodal (inter)action analysis: An interdisciplinary

approach
Multimodal (inter)action analysis (Norris 2004, 2009, 2011a, 2011b, 2013,

2014, 2015) originating from mediated discourse analysis (Scollon 1998, 2001)
is based in the sociological interest of humans acting in the world that we find in
the work of Goffman (1963); incorporates the interest in intercultural interaction
that we find in the work of Gumperz (1982); includes an interest in power in in-
teraction that we find in the work of Wodak (1989); delves into the micro-
analysis of interaction that we find in the work of Tannen (1984), Schiffrin
(1987), or Hamilton (1998); has a strong interest in applied linguistics that we
find in the work of van Lier (1996); is strongly influenced by socio-cultural
psychology as we find in the work of Wertsch (1998); and is grounded in social
semiotic thought that we find in the writings of van Leeuwen and Kress (van
Leeuwen 1999; Kress 2000; Kress and van Leeuwen 1998, 2001). With these
foundations, multimodal (inter)action analysis (Norris 2004, 2011) has devel-
oped into a strong theoretical framework with an abundance of methodological
tools (Norris 2004, 2009, 2011, 2013a, 2013b, 2014, forthcoming; Geenen
2013; Makboon 2015; Pirini 2015, 2016) that make the analysis of (always)
multimodal (inter)action possible, opening up research into new and promising
directions.
As mentioned above, a main theoretical notion in this framework is the con-
cept of mediation. The importance of mediation finds itself in the unit of analy-
sis, the mediated action, which has been adopted from Wertsch (1998) (who de-
veloped it from Vygotsky) and Scollon (1998) (who developed it from Wertsch),
and is further developed and thereby delineated into three methodological tools
by Norris (2004). Theoretically, the mediated action is defined as social actor(s)
acting with or through cultural tools/mediational means (Wertsch 1998; Scollon
1998). The mediated action as unit of analysis incorporates the social actor(s)
and the (always multiple) cultural tools/mediational means. Thus human(s) +
148 S. Norris
cultural tools with their always present inherent tension build the unit of analy-
sis. The terms cultural tools and mediational means are used interchangeably as
mediational means are cultural and cultural tools mediate action. This theoreti-
cal concept of mediation is embraced in the conception of the three methodolog-
ical units of analysis, the lower-level mediated action, the higher-level mediated
action, and the frozen mediated action.
2.1. The concepts lower-level, higher-level and frozen mediated

actions: Units of analysis
Multimodal (inter)action analysis conceives of all actions as mediated actions.

Therefore, as soon as we speak of lower-level, higher-level, or frozen actions,
we speak of mediated actions (even if it is not always stated explicitly). The
lower-level mediated action is defined as the smallest pragmatic meaning unit of
a mode (Norris, 2004). For example, an utterance is the smallest meaning unit
of the mode of spoken language. An utterance is a lower-level mediated action
as it is produced by a social actor + multiple socio-cultural and psychological,
embodied and physical, and semiotic mediational means/cultural tools as an ut-
terance is mediated by mediational means/cultural tools such as the larynx, lips,
teeth, tongue, out-breath, a language system, knowledge, and socio-cultural rel-
evance. By theorizing that every lower-level action, no matter what it entails, is
mediated in multiple ways, we can see that computer mediation in human-
computer interaction is not so very different from the mediation involved in the
production of an utterance. Revisiting Figure 1, where Mic pushes the touchpad,
this lower-level action (or smallest pragmatic meaning unit of the mode of com-
puter use) is also mediated by multiple socio-cultural and psychological, em-
bodied, physical, and semiotic mediational means/cultural tools. Here, the ac-
tion of pressing onto the touchpad is mediated by the finger, the hand/arm/ body
posture (to allow for the finger movement), the laptop and its touchpad, the
ubiquitous technological tools effecting a change through this finger movement,
the knowledge about the device and the result of this action, and so on. While in
practical terms, the mediation in the production of an utterance is vastly differ-
ent from the mediation involved in the pushing onto a touchpad, theoretically
speaking, we clearly can see that there exist great similarities as well; as each
lower-level action performed by a social actor is mediated by multiple socio-
cultural and psychological, embodied and physical, and semiotic mediational
means/cultural tools.
In line with this, the term mode in multimodal (inter)action analysis, is de-
fined as a system of mediated action (Norris 2013), incorporating a psychologi-
cal, physical, socio-cultural and with it a historical dimension to the concept and
adhering to the theoretical notion of mediated action. Conceived of as systems
of mediated action (Norris 2013), modes are learned by social actors in and
through contact with other social actors, the environment and objects within. In
this definition, the complexity of modal use in interaction is embraced at the
very same time as the always multiple mediation and the inherent tension be-
tween social actor(s), environment and objects within are contained.
Lower-level mediated actions are methodological tools that allow research-
ers to delineate micro actions that are (almost) never delineated by social actors
in their everyday lives. We may, of course, find the deliberate performance of a
blinking of the eyes or a loud outbreath or the push of a touchpad, but such in-
stances of individual lower-level actions are still always performed together
with other lower-level actions, some in and some out of synchrony, within the
performance of higher-level actions. For example, the lower-level action of
pushing the touchpad in Figure 1 is performed intentionally, but this action is
performed together with other lower-level actions such as a smile and gaze as il-
lustrated on Figure 4 (Figure 1 revisited).
Figure 4. Interconnection of lower-level actions.

150 S. Norris
Higher-level mediated actions are those actions that social actors usually intend
to perform and/or, as explained in more detail below, are aware of and/or pay at-
tention to. Higher-level actions come about through the coming together of
many chains of lower-level actions (such as several utterances chained together
by speakers, gaze shifts, postural shifts and so on) at the same time as the high-
er-level actions constitute these lower-level actions. Thus, lower-level and high-
er-level mediated actions always constitute each other. Figure 5 illustrates this
point.
Figure 5. Lower- and higher-level mediated actions constitute each other.
As we see in Figure 5, the connection has been established and the uncle’s wor-
ried expression from just a moment earlier turns into a smile at the same time as
he begins to wave to his niece in Australia and the niece in Australia simultane-
ously smiles at her uncle. All of these lower-level mediated actions, each one of
which is mediated in multiple ways, are part of the higher-level mediated action
of these participants interacting via skype. Here, it becomes apparent that medi-
ation of this higher-level action, the skype interaction, is anything but simple.
Rather, we find that a higher-level action such as this skype interaction is medi-
ated in vastly complex ways. While in much research on computer mediated

communication, technological mediation is discussed or referred to, multimodal
(inter)action analysis demonstrates that mediation on the one hand goes far be-
yond technological mediation, opening up the study of technology-mediated in-
teraction in new directions; and on the other hand, illustrates that technology-
mediated interaction is theoretically not all that different from other kinds of in-
teraction because all interaction is complexly mediated, opening up the study of
interaction in new directions.
In multimodal (inter)action analysis, we can dissect a higher-level action
and the multitude of mediation; or we can dissect a higher-level action and illus-
trate how it is made up and simultaneously produces a multitude of chained
lower-level mediated actions, that a social actor may or may not be focused up-
on. The more focused upon a higher-level action a social actor is, the stronger is
the higher-level action’s modal make-up. Strength of a higher-level action’s
modal make-up is represented through the concept of modal density which is
discussed in the next section.
But, briefly revisiting Figure 5, it is important to note that neither the wave
nor the smiles or the evolving utterances are separated from each other by the
participants in interaction; it is exactly their coming together that makes this
video-conferencing session just that: a video conferencing-session. Besides the
lower-level and the higher-level mediated actions, the third unit of analysis is
the frozen mediated action in multimodal (inter)action analysis. This concept al-
lows for the analysis of relevant actions that have been performed by a social
actor at an earlier time, which become frozen in objects or the environment. As
a quick example, when we have a look at Figure 5 once more, we see a beer bot-
tle standing on the desk (in the lower right image of the screen grab). This bottle
tells of the action of Mic drinking a beer and having positioned it where it is
standing now. Even if we had not witnessed him at points in the video having a
sip of beer now and again, we would read the action of him drinking beer off of
the object itself. As discussed elsewhere (Norris 2004), usually social actors
read those actions off of objects that are closest in time and space to the object
and the individual. These read-off actions may or may not be correct and are in
interaction often confirmed or rejected and corrected. As we will see in section
2.4 below, the concept of frozen action, just as the concept of lower-level action
and the concept of higher-level action, is highly relevant when analysing inter-
action.
152 S. Norris
2.2. The concepts modal density and foreground-background

levels of attention
Modal density = lower-level action density within a higher-level action (Norris

2004, 2008, 2009, 2011). The concept of modal density allows to analyse inter-
actions beyond the focus; and the concept of a foreground-background continu-
um allows to visually represent the various levels of attention that an individual
is simultaneously engaged in. Revisiting the example given in Figure 3, more
information is necessary to allow for the analysis of Mic’s attention levels at
that very moment as shown in Figure 6.
Figure 6 illustrates that Mic is engaged in three simultaneous higher-level
actions: (1) He is skyping with family members in Australia; (2) He is engaged
in a research project; and (3) He is interacting with his girlfriend. The first high-
er-level action, the moment of reconnecting with his sister and nieces in Austral-
ia has briefly been discussed above (Figure 3). Mic’s skype call, as mentioned in
the Introduction, is part of a research session, in which Mic is using a research
laptop that records his online interaction, an external camera that records him
from an in-room point of view, and two researchers, who are observing him
from the back of the room. Simultaneously, and from before the time when the
researchers arrived at his house, Mic’s girlfriend is present. Mic, no doubt is
aware of all of this as he is sitting in front of the laptop trying to reconnect with
his sister in Australia. However, Mic is not aware of or paying attention to all of
these higher-level actions to the same degree. Here, as Mic is waiting for the
connection to be established, he is highly aware of the research session. When
looking at Figure 6, we see that at this very moment the research session modal-
ly dominates: Mic takes up close proxemics to the research laptop and he is well
aware of being recorded; he is aware of his proxemics to the stationary camera
and of the fact that this camera too records him; and he is aware of the presence
of the researchers due to his proxemics to them and having spoken with them
just a moment before. Taking part in a research project and the many mediated
actions that this entails (which are now frozen in the objects: laptop, tripod,
camera, researchers’ notebooks, etc.) as well as Mic’s embodied modes of pos-
ture and his bodily proxemics to the objects that entail the frozen actions and to
the researchers present in the room, cumulate in high modal density as illustrat-
ed in Figure 6, demonstrating that he is focusing on his engagement in the re-
search session at this moment. At the same time, and as mentioned previously,
Mic is paying attention to skype as he is waiting for the connection to be made.
His lower-level actions of a worried facial expression, direct gaze at the com-
puter screen, posture (positioned to easily see and be seen), relaxed arms/hands
all cumulate in medium modal density as illustrated in Figure 6, demonstrating
that he is engaged in the skype call in the mid-ground of his attention. Still sim-
ultaneously, but to a much lesser degree, Mic is aware of the presence of his
girlfriend and his interaction with her. For example, he turns to her later and re-
quests her to join him in his skype interaction. However, at this very moment, it
is her proxemics to him and her presence in the room that cumulate in a low
modal density as illustrated in Figure 6, demonstrating that Mic is paying least
attention to the interaction with her at this time.
Figure 6. The various interactions that Mic is engaged in at a particular point in time.
Mic’s focused attention/awareness of taking part in the research project persists

for some time. But at almost 4 minutes into the skype session, Mic indicates a
154 S. Norris
change in focus, which is analysable through the concept of semantic/pragmatic

means outlined in the next section. As he refocuses, Mic becomes more engaged
in the skype interaction as the modal density of this higher-level action rises.
2.3. Concept semantic/pragmatic means
Semantic/pragmatic means are pronounced lower-level actions that indicate a

change in focus by the one producing them (Norris 2004, 2011). These means
are semantic in that they produce a change in meaning of the higher-level ac-
tions in the attention levels of the performer; and they are pragmatic as their use
produces a knowledge of that change in attention to a different higher-level ac-
tion for others engaged in interaction. Semantic/pragmatic means are always
pronounced and have a structuring function. As such, they sit somewhat outside
of the higher-level actions themselves. When Mic produces the semantic/prag-
matic means of bowing forward (Figure 7), he indicates a shift from paying fo-
cused attention to his engagement in the research project to paying focused at-
tention to the interaction with his sister and nieces in Australia. Here, bowing
down low (a pronounced lower-level action) does not convey meaning as a part
of the higher-level action of engaging in the research project, nor does it convey
meaning that connects to the higher-level action of interacting with his sister
and nieces via skype.
Figure 7. Semantic/pragmatic means: Bowing forward.
Social actors, who are engaged in multiple higher-level actions, quite frequently
shift their focused attention from one to another higher-level action that they are
involved in. Refocusing is always structured by semantic means, as the social
actor is restructuring not only the attention that they are paying but also the
meaning that they are constructing by focusing on a particular higher-level ac-
tion. As the means that structure attention and meaning in the mind of the social
actor producing them is always visible or audible, these means also function
pragmatically in interaction so that others are often aware of what someone else
is focusing on (Norris 2004, 2006, 2011a).
As is visible in the brief transcript in Figure 7 image 1 (reproduced larger in
Figure 8), Mic’s sister is prompting 3-year old Sophie indirectly to show Mic
her tooth when she says did you show Mic your tooth? (see Geenen, forthcom-
ing for a detailed analysis of Sophie’s (inter)action). Mic, however, is still
laughing at something that occurred earlier in the skype conversation, and he is
still focused upon the research session. However, as he continues to laugh, he
now bows his head low (Figure 7 image 2) in a semantic/pragmatic means, and
when his 5-year old niece Isla directs him to look at her tooth (to look at So-
phie’s tooth) Mic’s facial expression changes and illustrates that he is now fo-
cused upon the skype interaction with his sister and nieces in Australia as visible
in the transcript (Figure 9) discussed in the next section.
3. How do these concepts work together?: A shift in focus
As discussed in Section 2.2, Mic is first focused upon the research session, he
mid-grounds the skype interaction, and backgrounds the interaction with his
girlfriend (Figure 6). This analysis was conducted through the concepts of low-
er-level, higher-level and frozen mediated actions, modal density, and the fore-
ground-background continuum of attention/awareness. Utilizing the concept of
semantic/pragmatic means, it was then illustrated in Section 2.3 that it is possi-
ble to delineate the exact point at which Mic changes his focus from being en-
gaged in a research project to interacting via skype with his sister and nieces
due to the analysis of a semantic/pragmatic means (Figure 7). In Figure 9 below,
Mic’s new focus becomes apparent as we again utilise the concepts of lower-
level, higher-level and frozen mediated actions as well as modal density and the
foreground-background continuum of attention/awareness.
The multimodal transcript (Figure 9) follows the transcription conventions
described in Norris (2002, 2004, 2011) with a reading path from left to right and
top to bottom. Each individual screengrab is numbered top right and the exact
time in the video recording is presented top left of each screen grab; utterances
by individual participants are colour coded, overlaid on top of the screengrabs to
illustrate the coming together of spoken language and other modes and high-
lighting the rising and falling of intonation as produced by the speaker as illus-
trated in Ladefoged (1975). In the following transcript, we see Mic’s sister’s
156 S. Norris
(the girls’ mother’s) utterances in red. She is not visible in the images. Then, we
find 5-year old Isla’s utterances in white and she is only visible in the first and
last two images of the transcript, but her hand is clearly visible in images 7–10.
Sophie is visible in all screen grabs but she does not speak in this excerpt; and
Mic is clearly visible and his utterances are produced in yellow as shown in
Figure 8.
Figure 8. Social actors and their colour-coded utterances in the transcript in Figure 9:
Mother’s utterances in red; uncle’s (Mic’s) utterances in yellow;
and Isla’s utterances in white.
The first three images in Figure 9 repeat the images in Figure 7 as they illustrate
on the one hand that a new topic is broached by the Sophie’s mother (Mic’s sis-
ter) and that Mic is not immediately responding to this topic as he is still fo-
cused upon the research session. As he refocuses, Mic becomes visibly more
engaged in the skype interaction demonstrating that modal density of this high-
er-level action rises.
In the first three images in Figure 9, Mic performs his sematic/pragmatic
means and in image 4 we see how modal density begins to rise. Social actors of-
ten lag once they have performed a semantic/pragmatic means (Norris 2004,
2011a) before they are fully engaged in the newly focused upon higher-level ac-
tion. What this shows is that social actors often take some time before they build
up the modal density and when examining these changes in great detail, we can
Figure 9. Mic is now fully focused upon the skype interaction (images 5–10).
This same excerpt is analysed in Geenen (forthcoming), detailing Sophie’s learning
of making a relevant interactive contribution in family interaction.
158 S. Norris
see how modal density is built up incrementally. In image 4 of Figure 9, only a

little over a second after the indicated shift in focus, we see a small change in
Mic’s facial expression and head movement: his previous smile turns into a se-
rious expression and his head moves forward and down a bit. Then, in image 5,
another second later, Mic has moved his posture and with it his head further
forward, is now gazing intently at Sophie’s teeth displayed on his screen, and
speaks, ending quite loudly, saying oh my God where. As Sophie pulls down her
lower lip, Mic continues to look intently, beginning to tilt his head and saying
you look like you’ve got all your teeth (image 6). However, his facial expression
displays that he is unsure as he tilts his head further and continues to intently
gaze at the teeth and Isla’s hand makes her way to Sophie’s tooth (image 7). In
image 8, Isla is pointing at a specific tooth in Sophie’s mouth; her mother says
no, and Isla latches this no of her mother saying that one. As they are producing
the utterances and Isla is pointing, we can see in Mic’s facial expression the pain
that he is feeling by the mere thought of Sophie having knocked out a tooth.
Mic moves his head and posture back a little as if to move away from a blow;
his head is still tilted and the facial expression is expressing even more pain
now with his mouth showing his teeth, the edges of his lips pulled downward,
and his eyes squinted (image 9). Mic continues to move back slightly and con-
tinues to produce the facial expression when his sister says it was (image 10)
and he exclaims oh really and she continues with it was horizontal (image 11);
and Mic questions Sophie how’d you do that.
By analysing the lower-level actions produced, we can demonstrate that
Mic’s change in focused higher-level action comes about after the production of
a semantic/pragmatic means which re-structures the amount of attention that he
pays to the simultaneous higher-level actions that he is engaged in and which
indicates this restructuring to others. However, rather than occurring immediate-
ly, a shift is produced incrementally (see also Norris 2008) with modal density
building up through (inter)action. In the above example, Mic begins building up
modal density of the higher-level action of (inter)acting with his sister and niec-
es via skype through embodied lower-level actions (and chains thereof) such as
facial expression, head movement, postural change, gaze, and proxemics to the
laptop screen as well as the production of utterances. As modal density of this
higher-level action increases, modal density of the higher-level action of engag-
ing in a research project decreases in tandem. Resultantly, Mic’s attention levels
can now be visualised with the concept of the modal density foreground-
background continuum (Figure 10).
Figure 10. New distribution of higher-level actions in Mic’s attention levels.
The graph in Figure 10 visualises the new distribution of attention/awareness of

the higher-level actions that he is involved in. As illustrated in Figure 9 previ-
ously, Mic begins to focus more and more on the higher-level action of skyping
with family members as he is concurrently paying less attention to the higher-
level action of engaging in a research project. As lower-level mediated action
density to produce the higher-level action of skyping with family increases, the
lower-level action density for the higher-level action of engaging in the research
project decreases. With a shift in focus, Mic’s gazing at the laptop screen is re-
lated more to the looking at the damaged tooth and his awareness of being rec-
orded diminishes. Correspondingly, with modal density increasing to produce
the higher-level action of skyping with family members through the many em-
bodied modes that Mic uses, the modal density produced by the frozen actions
embedded in the tripod and camera as well as the physical presence and proxe-
mics to the researchers diminish with his paying less attention to them (indicat-
ed by dotted lines in Figure 10), thereby pushing the higher-level action of en-
gaging in a research project to the mid-ground of Mic’s attention/awareness.
The modal density foreground-background continuum, although a two-
dimensional and relatively simplistic visualisation, allows us to map the very
160 S. Norris
Figure 11. Modal density before and after Mic’s performance

of the semantic/pragmatic means.
complexly performed change in Mic’s attention/awareness in order to clearly

demonstrate the shift that has taken place (Figure 11).
4. How do the concepts work together?: Mediation
In this section, the above example (Figure 9) is revisited with an emphasis on

mediation: Each lower-level action performed by a social actor is mediated by
multiple socio-cultural, cognitive and psychological, embodied, physical, and
semiotic mediational means/cultural tools. The sematic/pragmatic means that
Mic performs in the first three images of Figure 9 is mediated psychologically
as he appears to feel more comfortable to change his focus away from the re-
search session onto the actual skype interaction; it is mediated cognitively, as
the means itself indicates a cognitive re-structuring of Mic’s focused attention;
the semantic/pragmatic means (the bowing of his head) is mediated socio-
culturally as it is learned through social and cultural development; the means is
mediated by his physical body, particularly his head; and it is mediated semioti-
cally as the bowing of the head at this moment in interaction is meaningfully
produced as a structuring device and can be read by others as a shift in his fo-
cus.
In image 4 of Figure 9, where Mic’s previous smile turns into a serious ex-
pression and his head moves forward and down a bit, he reacts to the utterance
and the serious tone of his sister’s voice when asking Sophie did you show Mic
your tooth? (image 1) and then explaining to Mic ‘she knocked her tooth out
(image 4). This producing of a serious expression is again mediated in multiple
ways from cognitive/psychological as he realises that his sister is sharing a seri-
ous matter; it is mediated socio-culturally as a serious matter and tone of voice
by one social actor in interaction is to be responded to in a serious way by the
other; it is mediated embodied physically as he changes the tension in his facial
muscles; and it is mediated semiotically as the facial expression displays his
knowledge of these semiotic systems.
Similarly, one can work through each of the lower-level actions that Mic
performs and establish the multiple ways that they are mediated. However, an
intensity of modal density is also developed by the interplay of several lower-
level actions and their mediation. In image 5 of Figure 9 for example, Mic con-
tinues to move forward and he gazes intently at Sophie’s teeth as he says oh my
God where, emphasizing the where with intensity of voice. These lower-level
actions not only are each mediated in multiple ways, they also mediate each
other: Mic’s embodied physical postural shift forward mediates his intent gaze
at Sophie’s teeth; Mic’s newly established closeness to Sophie’s teeth and his in-
tent gaze in turn mediate his emphasising the word where. As all of these lower-
level actions come together, they demonstrate Mic’s focus.
Then, even though Mic says you look like you’ve got all your teeth (image
6) and continues with I can’t see any missing (image 7) in a re-assuring tone of
voice, Mic’s facial expression, proxemics to the screen and intensity of gaze
suggest worry. Here, we see dual socio-cultural mediation of an intertwined
multimodal moment, linking reassurance with worry in embodied complex
ways. The physical embodied mediation allows for a skilful realisation of semi-
162 S. Norris
otic dual expression of contradictory meaning, whereby the semiotic systems of

course also mediate the interactive moment.
As Mic’s sister produces her no, and then explains that the tooth was hori-
zontal (images 8-11) Mic’s facial expression mediates his empathy, the pain that
he is feeling for Sophie having damaged her tooth. His empathy is further medi-
ated as Mic moves his head and posture back and he squints his eyes in apparent
pain. Of course, each of these lower-level actions is not only mediated psycho-
logically by his feeling of empathy, but are also mediated in embodied physical,
socio-cultural, and semiotic ways.
During this time of high modal density and complex cognitive, psychologi-
cal, socio-cultural, and semiotic mediation of the interaction with his sister and
nieces, the computer technological mediation, which was apparent in Mic’s ear-
lier facial expression (Figure 3) is here taken for-granted and ubiquitous.
5. Conclusion
This article has explicated some key concepts of multimodal (inter)action analy-
sis (Norris 2004, 2011a, 2015, forthcoming) using examples from a family vid-
eo conferencing interaction. Multimodal (inter)action analysis is a framework
with strong theoretical foundations (Wertsch 1998; Scollon 1998; 2001) and
theoretically linked methodological tools that situate human social actors with
their cognitive, psychological, and bodily physical dimension as always linked
to their physical and socio-cultural environment. Taking the mediated action as
its unit of analysis, the framework embraces the complexity and constant inher-
ent tensions that exist in the unit of social actor(s) plus mediational
means/cultural tools. Through this unit of analysis, and more so through the
methodological tools derived from it (the lower-level, higher-level, and frozen
mediated actions) the framework allows for an inclusion of all of the various
multimodal dimensions. Thus it becomes possible to incorporate all modes into
a discourse study; analyse the interaction as linked to the relevant settings and
objects within; and to analyse the (almost) always multiple actions that social
actors engage in on various levels of their attention. After having explicated
some of the key concepts of this framework in the first sections, the article
turned to the analysis of a brief family interaction via skype in which it was first
shown that that the New Zealand participant Mic payed more attention to his
engagement in the research project than to the unfolding skype interaction. This
analysis is only possible because of the multiplicity of data collected: the online
recording, the stationary camera recording, and the observations made by the re-
searchers. Such an analysis, for example, would not be possible for any of the
overseas participants because for all overseas participants we only have the
online data.
Next, the article showed Mic’s semantic/pragmatic means (his bowing his
head), which indicated a shift in focused attention. A close analysis of the
emerging interaction then illustrated how Mic’s modal density for the higher-
level action of interacting with his sister and nieces via skype incrementally in-
creased as the modal density for the higher-level action of being engaged in a
research project decreased. Through increasing multimodal interactional com-
plexity mediated in multifaceted ways, Mic increased modal density of the in-
teraction with his sister and nieces and established an emotive closeness. At this
time, the sharing about Sophie’s damaged tooth and Mic’s displayed empathy
takes on great importance, while the technology that mediates the interaction is
only a mundane aspect, which is taken for granted by the participants. Whereas
the technology is not taken for granted at a possible point of failure (Figure 3), it
here becomes ubiquitous as it functions correctly.
References
Bateson, G. 1972. Steps to an ecology of mind. New York: Ballantine.

Bucholtz, M. and K. Hall. Forthcoming. “Embodied sociolinguistics”. In: Coupland, N.
(ed.), Sociolinguistics: Theoretical debates. Cambridge: Cambridge University
Press.
Geenen, J. 2013. Kitesurfing: Action, (inter)action and mediation. (PhD thesis, Auck-
land University of Technology.)
Geenen, J. Forthcoming. Multimodal acquisition of interactive aptitudes: A microgenet-
ic case study.
Goffman, E. 1963. Behaviour in public places. New York: Free Press.
Goffman, E. 1974. Frame analysis. New York: Harper.
Gumperz, J. 1982. Discourse strategies. Cambridge: Cambridge University Press.
Hamilton, H. 1998. “Reported speech in survivor identity in on-line bone marrow trans-
plantation narratives”. Journal of Sociolinguistics 2(1). 53–67.
Herring, S., D. Stein and T. Virtanen (eds). 2013. Pragmatics of computer-mediated
communication. Berlin, New York: de Gruyter Mouton.
Kress, G. 2000. “Design and transformation: New theories of meaning”. In: Cope, B.
and M. Kalantzis (eds.), Multiliteracies: Literacy learning and the design of social
futures. Psychology Press. 153–203.
Kress, G. and T. Van Leeuwen. 1998. Reading images: A grammar of visual design.
London: Routledge.
164 S. Norris
Kress, G. and T. Van Leeuwen. 2001. Multimodal discourse: The modes and media of
contemporary communication. London: Edward Arnold.
Ladefoged, P. 1975. A course in phonetics. Orlando: Harcourt Brace.
Latour, B. 2005. reassembling the social: an introduction to actor-network-theory. Ox-
ford: Oxford University Press.
Makboon, B. 2015. Spiritual vegetarianism: Identity in everyday life of Thai non-
traditional religious cult members. (PhD thesis, Auckland University of Technolo-
gy.)
Norris, S. 2004. Analyzing multimodal interaction: A methodological framework. Lon-
don: Routledge.
Norris, S. 2006. “Multiparty interaction: A multimodal perspective on relevance”. Dis-
course Studies 8(3). 401–421.
Norris, S. 2008. “Personal identity construction: A multimodal perspective”. In: Bhatia,
V., J. Flowerdew and R.H. Jones (eds), New directions in discourse. London:
Routledge. 132–149.
Norris, S. 2009. “Modal density and modal configurations: Multimodal actions”. In:
Jewit, C. (ed.), Routledge handbook for multimodal discourse analysis. London:
Routledge.
Norris, S. 2011a. Identity in (inter)action: Introducing multimodal (inter)action analy-
sis. Berlin: de Gruyter Mouton.
Norris, S. 2011b. “Three hierarchical positions of deictic gesture in relation to spoken
language: A multimodal interaction analysis”. Visual Communication 10(2). 1–19.
Norris, S. 2013a. “What is a mode? Smell, olfactory perception, and the notion of mode
in multimodal mediated theory”. Multimodal Communication 2(2). 155–169.
Norris, S. 2013 b. “Multimodal (inter)action analysis: An integrative methodology”. In:
Müller, C., E. Fricke, A. Cienki and D. McNeill (eds.), Body – language – commu-
nication. Berlin/New York: de Gruyter Mouton.
Norris, S. 2014. “The impact of literacy based schooling on learning a creative practice:
Modal configurations, practices and discourses”. Multimodal Communication 3(2).
181–196.
Norris, S. (ed). 2015. Multimodality: Critical concepts in linguistics. (Vols. I–IV.) Ab-
ingdon: Routledge.
Norris, S. Forthcoming. Working with multimodal data: Research methods in multimod-
al discourse analysis. Hoboken, NJ: John Wiley and Sons.
Norris, S. and R.H. Jones. 2005. “Introducing mediated action”. In: Norris, S. and R.H.
Jones (eds.), Discourse in action: Introducing Mediated Discourse Analysis. Lon-
don, New York: Routledge. 17–19.
Pietikäinen, S., P. Lane, H. Salo and S. Laihiala-Kankainen. 2011. “Frozen actions in
the Arctic linguistic landscape: A nexus analysis of language processes in visual
space”. International Journal of Multilingualism 8(4). 277–298.
Pirini, J. 2015. Research into tutoring: Exploring agency and intersubjectivity. (PhD the-
sis, Auckland University of Technology.)
Pirini, J. 2016. “Intersubjectivity and materiality: A multimodal perspective”. Multi-
modal Communication 5(1). 1–14.
Schiffrin, D. 1987. Discourse markers. Cambridge: Cambridge University Press.
Schiffrin, D. 1994. Approaches to discourse. Oxford: Blackwell.
Scollon, R. 1997. “Handbills, tissues, and condoms: A site of engagement for the con-
struction of identity in public discourse”. Journal of Sociolinguistics 1(1). 39–61.
Scollon, R. 1998. Mediated discourse as social interaction. London: Longman.
Scollon, R. 2001. Mediated discourse: The nexus of practice. London: Routledge.
Shohamy, E. and D. Gorter (eds.). 2008. Linguistic landscape: Expanding the scenery.
London: Routledge.
Tannen, D. 1984. Conversational style: Analyzing talk among friends. Norwood, NJ:
Ablex.
van Leeuwen, T. 1999. Speech, music, sound. London: Macmillan.
Van Lier, L. 1996. Interaction in the language curriculum: Awareness, autonomy and
authenticity. London: Longman.
Wertsch, J. 1998. Voices of the mind: A sociocultural approach to mediated action.
Cambridge, MA: Harvard University Press.
Wodak, R. 1989. Language, power and ideology: Studies in political discourse. Am-
sterdam, Philadelphia: John Benjamins.
Address for correspondence:

Sigrid Norris
Multimodal Research Centre
School of Communication Studies
Auckland University of Technology
Private Bag 92006
Auckland 1142
New Zealand
sigrid.norris@aut.ac.nz
1032194
research-article20212021
SGOXXX10.1177/21582440211032194SAGE OpenAlyousef
Original Research
SAGE Open
Text Cohesion in English Scientific

July-September 2021: 1–13
© The Author(s) 2021
DOI: 10.1177/21582440211032194
https://doi.org/10.1177/21582440211032194
Texts Written by Saudi Undergraduate journals.sagepub.com/home/sgo
Dentistry Students: A Multimodal

Discourse Analysis of Textual and
Logical Relations in Oral Biology Texts
Hesham Suleiman Alyousef1
Abstract
This qualitative study examined multimodal cohesive devices in English oral biology texts by eight high-achieving Saudi
English-as-a-foreign-language students enrolled in a Bachelor of Science Dentistry program. A Systemic Functional Multimodal
Discourse Analysis (SF-MDA) of the textual and logical cohesive devices in oral biology texts was conducted, employing
Halliday and Hasan’s cohesion analysis scheme. The findings showed that students used varied cohesive devices: lexical
cohesion, followed by reference and conjunctions. Although ellipsis was minimally employed in the oral biology texts, its
discipline-specific uses emerged: the use of bullet points and numbered lists that facilitate recall. The SF-MDA of cohesion in
multimodal semiotic resources highlighted the processes underlying construction of conceptual and linguistic knowledge of
cohesive devices in oral biology texts. The results indicate that oral biology discourse is interdisciplinary, including a number
of subfields in biology. The SF-MDA of pictorial oral biology representations indicates that they include instances of cohesive
devices that illustrate and complement verbal texts. The results indicate that undergraduate students need to be provided
with a variety of multimodal high-cohesion texts so that they can successfully extend underlying conceptual and logical
meaning-making relations.
Keywords
oral biology discourse, textual cohesion, conceptual knowledge, logical relations, Systemic Functional Linguistics (SFL),
Multimodal Discourse Analysis (MDA)
In academic texts, use of cohesive devices contributes to Analysis (SF-MDA) of cohesive devices have previously
text coherence between sentences, thereby facilitating been confined to business studies (Alyousef, 2013, 2015,
comprehension and learning (Hall et al., 2014; Ozuru 2016, 2017, 2018), mathematics (O’Halloran, 1998, 1999,
et al., 2009; Yuniartiah et al., 2018). Because English sci- 2005, 2008), nursing (Okawa, 2008), history (North,
entific texts are multimodal, encompassing orthographic 2005), journalism and media (Hawes, 2015), and science
texts and images (i.e., multisemiotic), a study of salient and computing (Drury et al., 2006; Jones, 2006). To the
multimodal textual and logical cohesive relations is neces- best of my knowledge, text-based investigations exploring
sary, as is investigating how students use these multimodal the use of cohesive devices in multimodal oral biology
texts to extend conceptual and logical meaning-making texts are lacking.
relations—in other words, how pictorial representations This study’s objective is to conduct SF-MDA of cohesive
are integrated with verbal texts to construct what Lemke devices in oral biology texts written by eight high-achieving
(2005) calls “a system of organizational relations defining
wholes and parts of those wholes” (p. 94). Halliday and 1
King Saud University, Riyadh, Saudi Arabia
Hasan (2014) define cohesive relation as “the semantic
relation between an element in the text and some other ele- Corresponding Author:
Hesham Suleiman Alyousef, Department of English Language and
ment that is crucial to the interpretation of it” (p. 8). In the Literature, Faculty of Arts, King Saud University, Riyadh 12482-6683,
study of textual and logical features in tertiary contexts, Saudi Arabia.
applications of Systemic Functional Multimodal Discourse Email: hesham@ksu.edu.sa
Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License
(https://creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of
the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages
(https://us.sagepub.com/en-us/nam/open-access-at-sage).
2 SAGE Open
Saudi English-as-a-foreign-language (EFL) students in a multimodal SFL-based framework for science classroom
Bachelor of Science Dentistry program. The study is perti- discourse to investigate its potential to provide insights into
nent as in Saudi Arabia, the number of Saudi students how a biology teacher selected, sequenced, and modified
enrolled in dentistry undergraduate programs has increased semiotic modalities to help Grade 11 students construct sci-
dramatically over the past 10 years. In 2016, for example, the entific meaning of the “chemosynthesis” concept. The
number of Saudi students enrolled in the Bachelor of Science framework aimed to capture multiple dimensions of mean-
Dentistry program increased by 22.34%, from 9,883 to ing during teacher explanations, that is, semiotic and epis-
12,091 (Saudi Ministry of Education, 2016). Because the temological functions of multiple modalities and learning
Saudi government intends to localize the dental profession, outcomes. These included epistemological, presentational,
currently occupied mostly by foreign expatriates, unsurpris- orientational, and organizational meanings. Organizational
ingly, many Saudi students are attracted to this field. This meanings, however, did not include construal (or realiza-
study might provide insights for science tutors and under- tion) of cohesive and logical relations in students’ biology
graduateEnglish-as-a-Foreign-Language/English-as-a-Second- discourse. The framework also drew key features of modal-
Language (EFL/ESL) science students because it could shed ities that extend meaning-making possibilities during a
light on practices and discourses that constitute well-con- teacher’s explanations. The researcher, however, did not
structed multimodal cohesive oral biology texts. To the best investigate biology students’ use of semiotic modalities to
of my knowledge, this study is the first to explore how Saudi extend conceptual and logical meaning-making relations.
undergraduate dentistry students construct multimodal cohe- Data included field notes, lesson transcripts, and informal
sive oral biology texts. interviews with the teacher. The findings revealed the use-
fulness of multiple modalities for understanding teachers’
explanations and for supporting, scaffolding, extending,
Literature Review and reinforcing the learning of a new science concept.
Investigations of the language of scientific discourse aim to Jaipal (2010) also identified aspects related to genre as pre-
reveal creative effects of language through Systemic Functional sented by the teacher, such as the structure and sequencing
Linguistics (SFL) analyses and descriptions of its distinct fea- of topics, lessons, concepts, modalities, and words. In
tures. Halliday (2004), for example, studied how scientific lan- visual diagrams, typographical (e.g., figures, lines, arrows)
guage has been used throughout history and showed how it and compositional tools (e.g., texture, color) were identi-
uses grammatical nominalizations and favors relational, mate- fied. Whereas Jaipal (2010) investigated a biology teach-
rial, or mental verbal processes. A number of studies (Hall er’s discourse, Guo (2004) studied a tertiary second-year
et al., 2014; Ozuru et al., 2009; Yuniartiah et al., 2018) demon- cell biology textbook. Guo employed SFL in his study of
strated that reading highly cohesive scientific texts improved systems and functions on which schematic drawings and
comprehension and learning. In an introductory biology course, statistical graphs were created and drawn to make meaning
for example, Ozuru et al. (2009) found positive correlation in the textbook. Although M. O’Toole’s (1994) framework
between high-cohesion texts and undergraduate students’ com- for analysis of schematic drawings was employed,
prehension and learning of text-based information. This indi- O’Halloran’s (1996) framework was used to analyze statis-
cates that students with sufficient reading skills can learn new tical graphs. Guo (2004) argued that each visual modality
concepts with greater efficiency when their text is more cohe- has specific conventions for making meaning. The results
sive. Therefore, highly cohesive texts play a vital role in maxi- showed that resources employed for textual (or composi-
mizing the efficacy of students’ learning. tional) meaning include reference through language and/or
As mentioned, SFL applications to multimodal textual symbolism, labeling, framing, parallel/contrast in shape
and logical features have previously been done in business and color, and geometry. Labeling of conceptual terms
studies, mathematics, science and computing, journalism appeared frequently in the cell biology textbook because it
and media, history, and nursing. However, text-based inves- represents a pedagogical aim of pictorial representations.
tigations of the use of cohesive devices in multimodal oral Furthermore, Hannus and Hyönä (1999) investigated use of
biology texts are lacking. The majority of research on multi- pictures in elementary-level biology textbooks by using
modal texts has focused on primary or secondary school con- eye-tracking to trace students’ “visual” trajectory with pre-
texts (Hsu & Yang, 2007; Jaipal, 2010; Korani, 2012). cision. The findings showed that high-ability children per-
Because multimodality is inherent in every biology text, formed better at integrating relevant passages containing
interest has grown in studying the perception of and interac- text and pictures. Bordet (2014) stated that variations
tion with these artifacts. between general scientific terms and domain-specific terms
Investigations of multiple semiotic modalities in biology appeared as a dominant pattern across texts, although they
textbooks have contributed to our understanding of their could not be found in parts of texts composed by L2 writ-
underlying meaning-making processes (Baldry & Thibault, ers. More specifically, Bordet conducted a comparative
2005; Guo, 2004; Hannus & Hyönä, 1999; Jaipal, 2010; study of 60 doctoral abstracts to evaluate contributions of
Kress, 2003). Jaipal (2010), for example, developed a collocational regularities and variations in construction of
Alyousef 3
Table 1. Textual and Logical Cohesive Resources.
Structural Cohesive
Textual statuses THEME: Theme ^ Rheme; INFORMATION: REFERENCE (cataphoric, anaphoric, definite article,
Given + New demonstrative, pronoun, possessive, comparative); ELLIPSIS
and SUBSTITUTION
LEXICAL COHESION (synonymy, antonymy, repetition,
Textual transitions (clause complex ⇒ logical) hyponymy, meronymy, hypernymy)
CONJUNCTIONS:
Elaboration, extension, and enhancement
Source. Adapted from Halliday (2014).
credible scientific discourse in four subcorpora: materials tenor refers to roles and relationships taken up by the writer
science and didactics of mathematics abstracts written by or the speaker. The textual metafunction realizing the mode
native and nonnative speakers. The combinations seemed of discourse is concerned with how multimodal semiotic
tightly connected with the discipline’s epistemological val- forms are organized and presented through successful use of
ues. A wide range of general scientific terms was employed thematic choices and cohesive devices. Due to spatial limita-
to express a particular concept’s changing view and to tions, I examined only representation of cohesive and logical
guide the reader’s attention across these changes. Use of structures. Cohesive structures are represented by four cohe-
collocational chains reinforced the author’s legitimacy in sive devices: lexical cohesion, reference, ellipsis, and substi-
claiming membership in the target discourse community. tution. These devices are concerned with textual statuses that
This indicates the need to attract scientific and academic L2 form textual cohesion. On the contrary, logical structures are
writers’ attention not only to the use of general and special- construed through conjunctions concerned with textual rhe-
ized terms but also to their combinations’ variety and to the torical transitions (or relations) between clauses: elaboration,
rhetorical impact of their textual distribution. Bordet (2014) extension, and enhancement (Table 1).
did not study the use of other cohesive discourse structures Lexical cohesion is defined through semantic relations and
in scientific texts, and Guo (2004) did not study the con- achieved through the use of synonyms, antonyms, repetition
strual of cohesive and logical relations in biology discourse. of the same word, hyponyms, meronyms, and hypernyms.
Finally, Lovejoy (1991) investigated cohesion in introduc- Reference elements include anaphora (referring to a preced-
tory sections of three articles from three disciplines: coun- ing element), cataphora (referring forward to an element), the
seling psychology, biology, and history. The number of definite article, demonstratives, pronouns, possessives, and
lexical cohesive ties was higher than other types. Given the comparatives. These elements are grammatical rather than
small data size, Lovejoy (1991) argues, “no claims are lexical because they include closed systems, such as person,
made about the ‘typicality’ of texts in the disciplines repre- number, and proximity. Substitution and ellipsis are also
sented” (p. 317). expressed through grammar because the substitute or elided
The study aimed to contribute to this line of research by element might be a verb, a noun, or a clause. Substitution
investigating a key topic in tertiary multimodal oral biology involves using a substitute word or clause such as “some” and
texts. The literature review revealed a lack of research inves- “none.” The elided element is recoverable from the preceding
tigating cohesive devices in tertiary oral biology discourse, text (e.g., question tags, “ . . . isn’t he?”). On the contrary,
thus indicating a need to explore and analyze multimodal conjunction includes both grammatical and lexical ties
textual and logical cohesive devices. (logico-semantic relations) because these elements involve
lexical selection and are interpreted in terms of grammatical
systems. As Halliday and Hasan (2014) state, it is “mainly
Theoretical Framework grammatical, but with a lexical component in it” (p. 6).
According to the SFL approach, language serves three kinds
of meaning (or metafunctions): to construe (or realize) mean-
Method
ing ideationally, by representing and expressing our experi-
ences of the world and basic logical relations; interpersonally, A qualitative case study research design was adopted
by enacting social relationships in discourse; and textually, because, as stated, the study aimed to explore how Saudi
by construing deployment of textual resources that organize undergraduate dentistry students construct multimodal cohe-
the first two metafunctions into a coherent text (Halliday & sive oral biology texts. Halliday’s (2014) SFL approach was
Hasan, 2014). These three metafunctions correlate, respec- relevant to the context of the study because the aim was to
tively, with the three register variables of field, tenor, and investigate the functional uses of multimodal cohesive
mode. Whereas field refers to experiential content in a text, devices in tertiary dentistry texts. The SFL approach
4 SAGE Open
considers functions of language, which is viewed as a social a non-random purposive sampling method was used. In purpo-
semiotic resource for meaning-making. The acronym sive sampling, students were deliberately sought based on
SF-MDA was used because oral biology texts typically gender (four from each gender) and a high level of achieve-
include pictorial representation. SF-MDA was framed by ment (A and A+) during previous semesters. The assignments
Halliday’s (2014) and Halliday and Hasan’s (2014) cohesion were comparable because the main topics were similar: devel-
analysis scheme. SF-MDA also utilized participants’ intui- opmental abnormalities (or defects of the face and oral cavity).
tive verbal interpretations (or intended reading path) (van Participants were not constrained by a word limit. The instruc-
Leeuwen, 2005) of oral biology pictorial representations to tor did not present this topic in class but asked students to con-
investigate how students used multiple semiotic modalities duct a search and to write about a minimum of two facial/oral
to extend conceptual and logical meaning-making relations. cavity defects. As a result, the researcher did not conduct class
These interpretations aimed to reveal processes underlying observation. Allotted 10 marks, this individual assignment
construction of conceptual knowledge and text–image rela- aimed to make students cognizant of oral and dental tissues’
tions. Each verbal interpretation was audio-recorded, tran- structure and composition as well as of the oral cavity’s con-
scribed next to its relevant image, and analyzed. genital and acquired anomalies.
Although SF-MDA of cohesive devices’ use was primar-
ily qualitative, numerical/quantitative data were employed to
make comparisons (e.g., “more,” “most,” and “equal”) more Results and Discussion
accurate. Each cohesive device type and its subcomponents
The results of textual cohesion analyses (Table 2) showed
were manually identified and annotated to calculate its fre-
that the students used a variety of cohesive devices (14.02
quency of occurrence per 100 words. To ensure the results’
devices per 100 words) in the eight oral biology texts.
validity, I calculated the frequency and percentage of the
Lexical cohesion was the most common category (6.33
occurrence of each cohesive device type per total number of
cohesive devices per 100 words). Extensive use of lexical
words in that text by dividing the total number of occur-
cohesion emerged from reiteration of the same lexical
rences of each type by the total number of words and multi-
items, a subcategory more frequently employed (3.91
plying the result by 100. Two procedures were followed to
cohesive devices per 100 words) than other subcategories.
ensure reliability in annotating cohesive devices. Students’
This finding aligns with a number of studies (Abusharkh,
verbal interpretations of oral biology pictorial representa-
2012; Hessamy & Hamedi, 2013; Liu & Braine, 2005;
tions were checked for accuracy by a practicing dentist. The
Mohamed-Sayidina, 2010; Wahid & Wahid, 2020). For
annotation codes were double-checked for accuracy and then
example, Wahid and Wahid (2020) found that undergradu-
revised by a fellow linguist. The text analysis web tool ate EFL students extensively employ reiteration in their
Textalyser (2004) was used to determine lexical items’ fre- essays
quency and ranking in texts. To rule out pronouns, articles, In this study, Textalyser’s (2004) findings showed that the
and prepositions, Textalyser was set to include only words of most frequently reiterated lexical items in the eight texts
more than three letters. were tongue, cleft(s), branchial, deformity, and process.
Such reiterations were expected because the assignment was
Participants and Data Selection on developmental abnormalities of the face and oral cavity.
Lexical reiterations and sense relations of meronymy, hypo-
Data were composed of individual assignments (6,085 words) nymy, and hypernym are all instances of reiterations because
written by eight male and female Saudi undergraduate den- they all have one lexical item referring to another (Halliday
tistry students enrolled in an Oral Biology course at a Saudi & Hasan, 2014). Meronymy was the second most frequently
university in Riyadh: Zahra, Sara, Yara, Noura, Ibrahim, used lexical cohesive device (Table 2).
Khalid, Sultan, and Ahmed. Participants’ ages ranged from 19 Paradigmatic lexical sense relations of hyponymy (super-
to 20 years. The number of participants seemed suitable for the ordination or inclusion) refer to a general class with its sub-
purpose of the present qualitative study because it aimed to class, and meronymy refers to part–whole relations.
provide an understanding of the construction of conceptual Examples 1 to 6 illustrate these relations. For example, the
and linguistic oral biology knowledge. However, as the num- more general (or superordinate) noun phrase “oral cavity” in
ber of participants cannot be claimed as representative, no Extract (2) is a hyponym of the four semantically correlated
attempt is made to generalize or replicate the findings but subclasses “anterior tongue,” “tongue-base anlagen,” “thyro-
rather to understand a specific context as it is. Each student
glossal duct,” and “foramen cecum,” which are meronyms
signed a consent form after reading a student information
(or part) of their general class:
sheet which provides brief information about the researcher,
the aims of the research study, and the information that will be (1)
ENLARGED TONGUE [L: Hyponym] also called as
collected. Oral biology is a second-year undergraduate Macroglossia [L: Meronym] or tongue hypertrophy [L:
required course for dentistry majors. As the study aimed to Meronym] or prolapses of the tongue. [L: Meronym]
examine the representation of cohesive and logical structures, (Sara)
Alyousef 5
Table 2. Frequency and Percentages of Cohesive Ties in the (5) During the initial stages, five facial processes (one frontal,
Eight Oral Biology Texts. [L: Meronym] two maxillary, [L: Meronym] two
mandibular) [L: Meronym] form and subsequently fuse
Subtotal (by the sixth week of gestation) to form the human face. [L:
Subcategory
Hyponym]
Type of tie
Frequency
Category
Classification of clefts: [L: Hyponym]
%
Number 0 cleft [L: Meronym]
Number 3 cleft (oro-naso-ocular) [L: Meronym]
Numbers 6 to 8 clefts [L: Meronym] (Zahra)
Elaboration Appositive 11 0.18
Clarification 3 0.06 (6) TONGUE ANOMALIES [L: Hyponym]
Extension Additive 63 1.04 Congenital lesions of the tongue include cysts [L:
Adversative 8 0.13 Meronym], mucoceles, [L: Meronym] macroglossia, [L:
Variation 21 0.34 Meronym] ankyloglossia [L: Meronym] lingual thyroid,
Enhancement Temporal 5 0.08 [L: Meronym], and median rhomboid glossitis. [L:
Conjunctives
Manner 23 0.38 Meronym]. (Noura)

Causal 19 0.31
Conditional 16 0.26 Similarly, the noun phrase “human face” in Extract (5) is a
Total 169 2.78 hyponym of the processes that lead to fetal craniofacial
Substitution 24 0.39 development (or composition of the human face): “one
Ellipsis 68 1.12 frontal” [L: Meronym], two maxillary [L: Meronym], two
Total 92 1.51 mandibular [L. Meronym].” As Halliday (2004) states,
Repetition 238 3.91 “technical terms cannot be defined in isolation; each one
Synonym 41 0.67 has to be understood as part of a larger framework, and
Lexical cohesion
Antonym 19 0.31
each one is defined by reference to all the others” (p. 162).
Hypernym 1 0.02
Tight semantic unity (or hierarchical networked structure)
Hyponym 23 0.38
in oral biology texts leads to well-formed taxonomic lexical
Meronym 63 1.04
relations that bind separate lexical strings. Halliday adds
Total 385 6.33
Demonstrative 45 0.74
that two prime semantic relationships constitute technical
Definite 47 0.77 taxonomies: superordination (a is a kind of x) and composi-
Comparative 34 0.56 tion (b is a part of y). Lexical relations between general
Pronouns 55 0.90 scientific terms and technical terms are organized into a
Possessive 10 0.17 network. The participants’ interpretations showed that these
Reference
Anaphoric 0 0 relations are related to a number of subfields in biology:

Cataphoric 16 0.26 study of the formation and early development of organisms
Total 207 3.40 (embryology), manifestations of an anomaly through a
Subtotal 853 14.02 microscopic examination (histopathology), and its causes
(etiology). This indicates that oral biology discourse is
interdisciplinary: It spans the spectrum of oral biology to
(2) ORAL CAVITY [L: Hyponym] ANOMALIES include a number of subfields in biology such as embryol-
Malformations of the oral cavity may result from errors in ogy, histopathology, and etiology. The students also used
the embryonic fusion of the anterior tongue, [L: Meronym] other sense relations of synonymy and antonymy, mostly to
tongue-base anlagen [L: Meronym], and explain scientific terminology, as in “ankyloglossia
the origin of the thyroglossal duct [L: Meronym] at the (tongue-tie)” [L: Syn.] (Sara); “symptomatic/asymptom-
area of the foramen cecum. [L: Meronym] (Khalid) atic” [L: Ant.] (Yara); “abnormalities/anomalies” [L: Syn.]
(Sultan); “formation/malformation” [L: Ant.] (Khalid);
(3) BENIGN TUMORS OF THE ORAL CAVITY [L: “anterior/posterior” [L: Ant.] (Noura). These taxonomic
Hyponym]
classifications contribute to the texts’ cohesiveness through
These small tumors . . . occur mainly on the palate, [L:
Meronym] fauces, [L: Meronym] and gingivae [L:
hierarchical lexical strings that are “particularly important
Meronym] in children and young adults. (Sultan) for organizing scientific fields as taxonomies” (Dreyfus
et al., 2016, p. 116).
(4) LIP ABNORMALITIES [L: Hyponym] The second most frequently used cohesive type was refer-
Clefting. [L: Meronym] ence (Table 1). Although occurrence of the reference item
Microstomia. [L: Meronym] “definite article” was rare (0.77 instances per 100 words), it
Lip pits. [L Meronym] was used homophorically (e.g., the teeth, the gum, the mouth).
Labial frenula. [L Meronym] (Yara) Here, the entity specified (or identified) is considered
6 SAGE Open
representative of the whole class, as in “often the frenulum adjective) in the first instance and as a Head in the second.
will stretch and correct itself” (Sara). However, homophoric The word “abnormality” in the first instance is called a shell
reference does not create grammatical cohesion within the noun, and it takes the pattern of demonstrative adjective
text. As the definite article “the” was not used as a signal of a “this” + optional premodifier+ shell noun (N). Shell nouns
specific identity in the text, or rather of identifiability, the ref- link nominal concepts with clauses containing complex
erent is not retrievable from a specific situation. Most stu- chunks of information. Ibrahim used the singular demonstra-
dents employed demonstrative singular and plural reference tive in Extract (8) as a Head, which refers anaphorically to
items “this” and “these.” These items include an elaboration the preceding noun phrase “hearing loss.” The demonstrative
relation because they refer to the location of a proposition in “these” in Extract (9) acts as a Head that refers back to a
the previous clause that is participating in the process through whole set of nouns related to types of congenital lesions of
the use of substitution. Unlike substitution, as reference con- the tongue. The demonstrative “this” in Extract (11) is
structs semantic relation and not grammatical one, it does not employed twice to refer to “distraction osteogenesis of the
necessarily have to be in the same grammatical class as the mandible, a technique used for the correction of deformity
referent. This linguistic resource achieves semantic unity, with minimal morbidity of the mandible.” The second
making a text more cohesive: instance of the personal pronoun “it” in Extract (12) refers to
the process when the “lingual thyroid tissue . . . causes air-
(7) In some cases, normal sized tongues appear larger because way obstruction,” whereas the first occurrence refers only to
there is underdevelopment of the mandible. This [R: Dem.] the nominal group “lingual thyroid tissue.” Halliday and
abnormality may lead to difficulties in speech and swallowing . Hasan (2014) argue that only this reference type has the
. . Why this [R: Dem.] happens is largely unknown. (Sara) property of extended reference. None of the participants
employed the first-person plural pronoun “we” to connect
(8) If left untreated, ear infections may lead to hearing loss. To with their readers and to maintain their attentiveness by
prevent this [R: Dem.] from occurring, special tubes are placed
engaging them in the argument. This could be caused by the
in children’s eardrums to aid fluid drainage, and their [Ref:
Poss.] hearing has to be checked every year. (Ibrahim)
instruction students typically receive during their preuniver-
sity schooling, which prohibits the use of the first-person
(9) Congenital lesions of the tongue include cysts, mucoceles, pronouns “I” and “we.” This finding contrasts with Martı́nez’s
macroglossia, ankyloglossia, lingual thyroid, and median (2003) study of theme in discussion sections of biology
rhomboid glossitis . . . The associated lack of tonsillar and research articles in English. Martı́nez argues that given the
adenoid tissue in patients with these [R: Dem.] anomalies argumentative nature of the discussion section, occurrence of
suggests that . . . (Khalid) first-person pronouns in thematic position is natural. Use of
these pronouns is one academic strategy writers employ to
(10) The median nasal process grows downward between the engage readers in their arguments and, thereby, to establish
maxillary processes to form the globular process. This [R: Dem.] rapport with them. As Hyland (2005) states, “while many
will form the philtrum. (Ahmed)
students are taught to shun the use of first person, it plays a
crucial interactional role in mediating the relationship
(11) At other pediatric institutions, mandibular distraction between writers’ arguments and their discourse communi-
osteogenesis is performed during the neonatal period in an effort ties” (p. 57).
to avoid a tracheotomy. This [R: Dem.] involves bilateral
Possessive pronouns were scarcely used in the eight texts.
division of the mandibular body and gradual advancement with
an adjustable fixation device. The tongue is progressively
Only Sara, Ibrahim, Ahmed, and Yara employed this refer-
moved forward with the mandible, improving the airway. This ence type to make anaphoric semantic connections to previ-
[R: Dem.] technique has been successful in experienced hands. ously introduced nominals. Although other personals require
(Yara) only one referent for their interpretation, possessive pro-
nouns demand two recognizable participants: a person or
(12) Excision of lingual thyroid tissue should be avoided unless object, called a possessor, and a possessed. Use of this refer-
it [R: Pro.] causes airway obstruction because it [R: Pro.] may ence item creates a sense of belonging or ownership. The
represent the only functional thyroid tissue. If this [R: Dem.] is students, however, used a higher number of comparative ref-
the case, the mass . . . (Noura) erence elements than possessives, in particular “more,” as in
the following:
The nominal demonstratives “this,” “these,” and “those”
have the experiential function of deictic expressions, and (13) Children with clefts are more [R: Comp.] vulnerable to a
they function either as modifiers or Head at the logical struc- larger than average number of cavities. (Ibrahim)
ture level. For example, the demonstrative “this” in Extract
(7) refers anaphorically to whole sets of nouns in the preced- (14) Deformities can be broadly subdivided into craniofacial
ing clauses, and it acts as a modifier (or demonstrative anomalies, clefting anomalies, and dentofacial anomalies. There is
Alyousef 7
often a crossover into other [R: Comp.] categories, for example, (18) The patches are often striated, forming a lace-like pattern,
most [R: Comp.] children with cleft lips and palates . . . (Sultan) but [C: Extension: Advers.] [Ellipsis: N.] can also [C: Extension:
Add.] be papular and confluent. Thus [C: Enhancement: Caus.]
(15) Clefting anomalies of the upper lip are more [R: Comp.] deformity of the cranium may also [C: Extension: Add.] be seen
common and more [R: Comp.] varied than clefting anomalies of as a facial deformity. (Sultan)
the lower lip because fusion of the components of the upper lip
occurs later in embryogenesis and is more [R: Comp.] complex (19) Ankyloglossia can affect eating, speech, and oral hygiene
than fusion of the lower lip. (Ibrahim) as well as [C: Extension: Add.] have mechanical/social effects.
Although [C: Enhancement: Man.] the exact incidence of facial
These reference items establish a relation of contrast clefts is unknown, they are estimated . . . (Zahra)
between two or more entities, and this reflects one linguistic
feature of oral biology texts. Whereas the texts lacked The additive conjunctive relation is cohesive when it
instances of anaphoric reference, cataphoric reference was connects two propositions in a series of events and struc-
rarely employed to refer readers to information outside the tural when it connects two or more entities within the same
text (exophoric). Exophoric reference does not contribute to clause (e.g., “lymphatic malformations and hemangio-
the text’s cohesion because it does not tie two elements mas”). Extension devices add or vary a clause message at
together within the text. The cataphoric reference items the pragmatic level. On the contrary, enhancement con-
“colon” (:), “follow,” and “below” were used to refer read- junctive devices provide reason (e.g., “so,” “because,”
ers to a following text or image, as in “the symptoms of “thus”), arrange events’ sequential structure (e.g., “first,”
macroglossia may be as follows:” (Sara) and “see the image “second”), and clarify how an action occurs (e.g., “as,”
below [R: Cat.]” (Ibrahim). Halliday states that the colon “although,” “though,” “while”). The eight texts expanded
does not indicate any structural relationship, but only sig- propositions by using causal conjunctive devices:
nals cataphora; therefore, it does not contribute to the tex- “because,” “thus,” “therefore,” “since,” “hence,” “in order
ture of the text. to,” and “so.”
The third most frequently occurring cohesive device was Whereas elaboration cohesive devices expand an utterance
conjunctions (Table 2). Conjunctions are “cohesive not in by redeveloping the message to provide focus on the content
themselves but indirectly, by virtue of their specific mean- (e.g., “for example,” “indeed”), enhancement devices do this
ings” because “they express certain meanings which presup- through provision of circumstantial details related to time,
pose the presence of other components in the discourse” place, manner, cause, or condition, as in the following:
(Halliday & Hasan, 2014, p. 226). Conjunctions were primar-
ily used to signal extension (1.51 devices per 100 words) and (20) “Thus, [C: Enhancement: Caus.] deformity of the cranium
enhancement (1.03 devices per 100 words) relationships. This may also [C: Extension: Add.] be seen as a facial deformity;
finding converges with Alyousef’s (2016) study of under- indeed [C: Elaboration: Clari.], this may be more obvious than
graduate business students’ marketing texts and Mohammed’s the skull deformity.” (Sultan).
(2015) study of L2 students’ texts, which indicate the use of
cohesive devices to primarily signal extension and enhance- Martin (1992) argues that logical relationships in scientific
discourse register are expressed not only through conjunctions
ment relationships. While extension devices are employed to
but can also be expressed through metaphorical forms of these
provide further related information or contrasting views, conjunctions, which involve nouns (the effect, the cause, the
enhancement conjunctive devices are used to enhance the consequence), verbs (cause, lead to, result in) and prepositions
meaning of another by qualifying it. Additive conjunctive ele- (due to, because of):
ments had the highest frequency in the eight texts (1.04 addi-
tive elements per 100 words), compared with other subtypes (21) Tongue enlargement is caused by lymphatic malformations
of elaboration and enhancement, whose occurrences were (Noura). Micrognathia results in a tongue that is
below 0.50 devices per 100 words (Table 2): disproportionately large for the oral cavity (Yara). Clefting
anomalies of the upper lip are more common and more varied
(16) Tongue-tie can affect a baby’s oral development, as well as than clefting anomalies of the lower lip, because fusion of the
[C: Extension: Add.] the way he or she eats, speaks, and components of the upper lip occurs later in embryogenesis and
swallows. In some cases, normal sized tongues appear larger is more complex than fusion of the lower lip (Khalid). Most
because [C: Enhancement: Caus.] there is underdevelopment of scientists believe clefts are due to a combination of genetic and
the mandible. (Sara) environmental factors (Ibrahim). Tongue-tie can also lead to
the formation of a gap or space between the two bottom front
(17) Congenital lip pits may also [C: Extension: Add.] be teeth (Sara). Ankyloglossia, also known as tongue-tie, is a
observed near the midline of the vermilion border. Some patients congenital oral anomaly that may decrease mobility of the
may have no adverse effects, while [C: Enhancement: Man.] tongue tip and is caused by an unusually short, thick lingual
others may have difficulty with speech. (Ahmed) frenulum (Zahra).
8 SAGE Open
In the excerpts above, the lexico-grammatical realizations of (24) There are problems associated with the cleft lip and cleft
causal relationships aim to explain causes and consequences palate, for
of oral cavity anomalies. Humphrey and Hao (2011) argue,
“cause and effect play a more central role in connecting ideas example: [C: Elaboration: Appos.]
and developing explanations” (p. 45). Such logical meta-
[Ellipsis: Claus.] Eating problems . . .
phors package information (or experience) underlying con-
cepts abstractly. As in all academic discourse, oral biology [Ellipsis: Claus.] Ear infections/hearing loss . . .
texts included few forms of grammatical metaphor repre-
sented by incongruent embedded causal logical relations: [Ellipsis: Claus.] Speech problems . . .
(22) Children with cleft palate are at increased risk of ear [Ellipsis: Claus.] Dental Problems. (Ibrahim)
infections since they are more prone to fluid build-up in the
middle ear (Ibrahim). During the third week, ectoderm infolds (25) They divide the frontal process into three parts
to form the stomodeum, the primitive oral cavity (Ahmed). In
infancy, the maxillary (upper) labial frenulum typically extends [Ellipsis: Claus.] The median nasal process
over the alveolar ridge to form a raphe that reaches the palatal
papilla . . . Mucoceles are formed when salivary gland secretions [Ellipsis: Claus.] The right lateral nasal process
dissect into the soft tissues surrounding the gland (Yara).
[Ellipsis: Claus.] The left lateral nasal process. (Ahmed)
Dreyfus et al. (2016) use the term “hidden” because such
forms are not lexically related to conjunction forms and (26) Classification of clefts: [L: Hyponym]
because the terms carry, besides their logical meaning, a
technically heavy experiential load. Clarification devices 1. [Ellipsis: Claus.] Number 0 cleft . . .
deepen the context by refocusing readers’ attention on cer-
2. [Ellipsis: Claus.] Number 3 cleft [L: Rep.] (oro-naso-ocular) . . .
tain proposition(s). Clarification and appositive devices
rarely occurred in the oral biology texts. Only Yara and 3. [Ellipsis: Claus.] Number 6 to 8 clefts. (Zahra)
Ibrahim minimally employed the temporal conjunctives
“until” and “when.” Zahra used a numbered list to classify types of clefts, whereas
Substitution and ellipsis rarely occurred in the students’ Ahmed employed bullet points to classify different parts of the
texts (Table 1). This finding converges with a number of frontal process. The other six students used bullet points to
studies (Abusharkh, 2012; Hessamy & Hamedi, 2013; Liu & present consequences of oral cavity anomalies (cleft lip or
Braine, 2005; Mohamed-Sayidina, 2010; Wahid & Wahid, tongue-tie). Bullet points and numbered lists avoid redundancy,
2020) that attributed these devices’ rare occurrence to par- facilitate comprehension, and can make recall easier. Karreman
ticipants’ limited knowledge. Because the tutor highly rated and Loorbach (2007) found that participants who used a web-
the eight texts, all the students successfully employed clausal site with text structured as lists performed better than those
ellipsis by availing themselves of the powerful means of two who used it with text structured as paragraphs. Ellipsis is used
rejoinders (Martin, 2001), bullet points and numbered lists, in oral biology texts as a means of avoiding redundancy.
to encode structural information most economically. These Instances of substitution are shown in the following excerpts
resources help students avoid repetition of the same or very from the students’ texts. The substitute typically has the same
similar constituents and to provide strong focus on concep- structural function of the item it replaces: a noun, a verb, or a
tual information. As a result, temporal devices (e.g., “first,” clause. The substitute also has the same grammatical class as
“second”) were elided. Bullet points are used to elide infor- the presupposed one (e.g., “cases,” “abnormality,” and “macro-
mation grammatically that is known or at least recoverable glossia” function as Head nouns in nominal groups):
by the reader and to emphasize unknown (or new) informa-
tion (Alyousef, 2020). Hence, elision is a form of presuppo- (27) Macroglossia, an unusually large tongue, is very
sition. The students’ texts were succinct because only key uncommon. In some cases [Substitution: N.] normal sized
features of an aspect were presented, thereby allowing all tongues appear large because there is underdevelopment of the
features to be understood in relation to each other: mandible. This [R: Dem.] abnormality [Substitution: N.] may
lead to . . . (Sara)
(23) For example, [C: Elaboration: Appos.] tongue-tie can lead to:
(28) Leukoplakia refers to a white patch that cannot be
[Ellipsis: Claus.] Breast-feeding problems . . . characterized clinically or pathologically as any other condition.
This [R: Dem.] definition [Substitution: Claus.] does not imply
[Ellipsis: Claus.] Speech difficulties. any specific histological changes. GRANULAR CELL
MYOBLASTOMA. This [R: Dem.] uncommon benign tumor
[Ellipsis: Claus.] Poor oral hygiene. (Sara) [Substitution: N.] usually forms . . . (Sultan)
Alyousef 9
Figure 1. Ibrahim’s illustration of lip anomalies.
“Some” and “this” in Sara’s text act as defining modifiers of generalized and more or less stable and timeless essence, in
Head nouns. Most participants used general words like “abnor- terms of class, or structure or meaning” (Kress & van
mality” and “anomalies” as substitutes for repudiated defects Leeuwen, 2006, p. 79). Visuals aid students in building their
of the face and oral cavity, such as “macroglossia.” As Halliday taxonomy of oral biology terms. Furthermore, more than
and Hasan (2014) state, “there is a borderline where substitu- 25% of pictures included adjacent pictures (Figure 1).
tion shades into lexical cohesion, involving the use of general In addition, the pictures’ most frequently reiterated con-
words such as a thing in a cohesive function” (p. 91). cepts (or themes) were clefting, ankyloglossia, thyroid, and
Macroglossia is repudiated (i.e., not carried over) by the plural macroglossia. Surprisingly, Yara did not include any visuals in
numerator “some” and the deictic “this.” Similarly, Sultan her text; this might indicate that her learning style is verbal.
substituted the nominal group “granular cell myoblastoma” The situational context is formed through a combination of
with the general word “tumor” although it was modified by the natural language and other pictorial representations through
deictic “this” and the epithets “uncommon” and “benign.” which students make meaning. Concepts, such as “ankylo-
Bordet (2014) argues that variations in the combination of glossia” and “dyspnea,” are transformed into another format.
general scientific and specialized technical terms are critical in The meaning-making processes of these representations
making scientific discourse credible and more cohesive. All encompass conceptual explanations. A logico interdependent
participants employed clausal substitution in which an entire intersemiotic (across different semiotic resources) relation of
clause was presupposed. For example, Sultan used the word elaboration exists between each pictorial representation and
“definition” to refer to a whole clause. orthographic text because the former clarifies the text.
To sum up, lexical cohesion was the most extensively However, the accompanying text provides strong topical
used category of cohesion, followed by reference and con- focus with explanations not present in the image. The concept
junctives. This is not surprising because the texts were from underlying the image is a meronym of the accompanying text,
a common field. The results highlight the importance of lexi- which in turn is the hyponym. Thus, the two modes are
cal cohesion in constructing cohesive oral biology texts. regarded as “parts functioning in some larger whole” (Baldry
& Thibault, 2005, p. 21). The two semiotic modes represent
part–whole semantic relations in which the text extends the
SF-MDA of Pictorial Representations
image. The logical convergence (or recontextualization) of
The SF-MDA findings revealed that oral biology pictures in the two semiotic modes occurs through the process of semi-
students’ texts are strongly cohesive with their accompany- otic metaphor. A semiotic metaphor refers to the intersemiotic
ing verbal texts because they are not disconnected by frames. processes underlying the shifts in meaning occurring between
The visual structure of these representations is conceptual natural language and visual displays (O’Halloran, 2003).
because it represents “participants in terms of their more Highly cohesive multimodal texts facilitate the process of
10 SAGE Open
Figure 2. Ankyloglossia image (Noura). Figure 3. Commissural lip pits disorder (Ahmed).
learning new concepts. As Ozuru et al. (2009) state, “stu- textual and logical cohesive devices in tertiary multimodal oral
dents’ difficulty in learning new concepts can be alleviated to biology texts and the ways students used multiple modalities to
some extent by making text more cohesive which makes extend conceptual and logical meaning-making relations.
readers less dependent on pre-existing knowledge” (p. 239). Although this study cannot claim that its participants constituted
The results of the SF-MDA of the implicit verbal ele- a representative sample, the findings may offer significant peda-
ments in the oral biology images revealed that they included gogical insights. The findings indicate that oral biology texts
instances of cohesive devices, as they aim to illustrate and intertwine various cohesive patterns and that the students used a
thereby complement verbal texts. range of cohesive devices. Moreover, the results highlight the
For example, Noura’s verbal interpretation of the implicit importance of lexical cohesion in constructing cohesive oral
conceptual knowledge underlying the ankyloglossia visual biology texts, as it was the most extensively used category, fol-
image (Figure 2) was as follows: lowed by reference and conjunctives. The results of lexical
sense relations indicate that oral biology discourse is interdisci-
(29) Ankyloglossia or tongue-tie is a congenital oral anomaly. It
[R: Pro.] [L: Repetition] is caused by a short, thick lingual
plinary: It spans oral biology to include a number of subfields in
frenulum, which is a membrane connecting the underside of the biology. Moreover, logical relationships in oral biology dis-
tongue to the floor of the mouth. (Noura) course are expressed not only through conjunctions but also
through metaphorical forms of these conjunctions, to explain
This interpretation (or reading path) revealed its inclusion oral anomalies’ causes and consequences. The rare occurrence
of two types of cohesive devices: reference and lexical cohe- of incongruent “hidden” causal relations indicates the need to
sion. Implicit conceptual knowledge underlying the pictorial make such forms “visible” to students so that they can under-
image of the congenital lip pits disorder (Figure 3) showed stand and construct such grammatical metaphors.
that it included instances of reference, conjunctions, and lexi- The SF-MDA of cohesion in multimodal semiotic resources
cal cohesion: highlighted the processes underlying construction of conceptual
and linguistic knowledge of cohesive devices in English oral
(30) Epithelium-lined blind tracts located at the corners of the biology texts. Without such knowledge of cohesive devices, stu-
mouth (or commissure). Congenital lip pits may be shallow or dents are more likely to lose their grip on the conceptual flow of
several millimeters deep. They [R: Pro.] [L: Repetition] may information (J. M. O’Toole & Schefter, 2008) because taxo-
also [C: Extension: Add.] be observed near the midline of the nomic classifications contribute to the organization of oral biol-
vermilion border. (Ahmed) ogy texts. Although substitution and ellipsis cohesive devices
rarely occurred in the oral biology texts, the occurrence of the
The conclusion to the presentation and discussion of find- latter device revealed a key feature of this discourse—use of
ings on the eight participants’ multimodal texts along with rejoinders, bullet points and numbered lists. Rejoinders facili-
the implications follow. tate recall because they encode information in the most econom-
ical manner. This highlights the importance of these cohesive
devices in scientific texts. Analysis of visual images in biology
Conclusion indicates that SFL provides a theoretical basis for informing
Drawing on Halliday’s (2014) and Halliday and Hasan’s (2014) future developments in analysis of multimodality. The SF-MDA
cohesion analysis scheme, this study aimed to investigate of visual artifacts indicates that participants’ intuitive
Alyousef 11
interpretations of their meaning-making processes facilitated Declaration of Conflicting Interests

text-based analysis. Students’ interpretations contributed to our The author declared no potential conflicts of interest with respect to
understanding of how print and pictorial representations are the research, authorship, and/or publication of this article.
processed and integrated in scientific discourse. The formula-
tion of semiotic metaphors involved in shuttling between natu- Funding
ral language and visual displays is crucial for students’ learning The author disclosed receipt of the following financial support for
and understanding of meaning-making resources in this dis- the research, authorship, and/or publication of this article: This
course. All eight participants successfully represented textual article was funded by Deanship of Scientific Research and the
and logical cohesive devices, as evidenced by their high marks. Research Centre at the Faculty of Arts.
Finally, it should be noted, however, only a subset of writers’
full range of conceptual knowledge and logical meaning-mak- ORCID iD
ing potential has been presented. Hesham Suleiman Alyousef https://orcid.org/0000-0002-9280-
As the first to analyze textual and logical cohesive devices 9282
in tertiary multimodal oral biology texts, this discourse-
based study adds to our knowledge database. Further research References
should investigate use of textual and logical cohesive devices Abusharkh, B. (2012). Cohesion and coherence in the essay writing
in oral biology texts written by native English speakers and of Palestinian college students [Unpublished master’s thesis].
in oral biology textbooks. Hebron University.
Alyousef, H. S. (2013). An investigation of postgraduate business
students’ multimodal literacy and numeracy practices in finance:
Recommendations for Practice A multidimensional exploration. Social Semiotics, 23(1), 18–46.
Students’ learning and understanding of meaning-making http://dx.doi.org/10.1080/10350330.2012.740204
resources in scientific discourse are facilitated when they Alyousef, H. S. (2015). A multimodal discourse analysis of
meet the requirements and expectations of the discourse international postgraduate business students’ finance texts:
An investigation of theme and information value. Social
community of practice. The findings of SF-MDA of oral
Semiotics, 26(5), 486–504. http://dx.doi.org/10.1080/10350
biology text–image relations have a number of implications 330.2015.1124518
for teaching and learning scientific English. The results can Alyousef, H. S. (2016). A multimodal discourse analysis of
serve as an analytical tool for tutors when analyzing func- the textual and logical relations in marketing texts writ-
tions of these semiotic modes and when discussing with stu- ten by international undergraduate students. Functional
dents the implicit conceptual knowledge and logical Linguistics, 3(3), 1–29. http://dx.doi.org/10.1186/s40554-
meaning-making relations underlying each image. A focus 016-0025-1
on both micro-level (word or sentence) and macro-level Alyousef, H. S. (2017). A multimodal discourse analysis of textual
(across sentences and image–text relations) features yields cohesion in tertiary marketing texts written by international
highly cohesive and coherent texts. Most undergraduate undergraduate students. In J. Pelkey (Ed.), Semiotics 2016:
EFL/ESL students focus only on a text’s micro-level aspects, Archaeology of concepts (yearbook of the Semiotic Society of
America). (pp. 99–122). Philosophy Documentation Center.
but this study indicates that science tutors need to provide
Alyousef, H. S. (2018). The representation of experience in under-
undergraduate EFL/ESL students with a variety of multi- graduate business students’ texts: A functional analysis of
modal high-cohesion texts so that they can successfully con- multimodal meaning making resources in marketing texts.
struct conceptual knowledge and logical meaning-making Discourse and Interaction Journal, 11(1), 5–27. https://doi.
relations in oral biology texts. As Nichols et al. (2013) state, org/10.5817/DI2018-1-5
a focus on “the language conventions of concept-specific Alyousef, H. S. (2020). A multimodal discourse analysis of English
representations fosters the development of disciplinary dis- dentistry texts written by Saudi undergraduate students: A
course by transforming students’ social practices of working study of theme and information structure. Open Linguistics, 6,
with scientific knowledge” (p. 179). Finally, students need to 267–283. http://dx.doi.org/10.1515/opli-2020-0103
be encouraged to use the rejoinders bullet points and num- Baldry, A., & Thibault, P. J. (2005). Multimodal transcription
bered lists to avoid redundancy and facilitate their compre- and text analysis: A multimedia toolkit and coursebook.
Equinox.
hension and recall of conceptual information.
Bordet, G. (2014). The rhetorical role of “collocational chains” and
their implications in the building of scientific discourse strategies.
Acknowledgments European Journal of English Studies, 17(3), 235–250. https://doi-
The author is indebted to the two anonymous reviewers for their org.sdl.idm.oclc.org/10.1080/13825577.2013.867183
insightful and helpful comments. The author expresses his appre- Dreyfus, S. J., Humphrey, S., Mahboob, A., & Martin, J. R. (2016).
ciation to both the Deanship of Scientific Research at King Saud Genre pedagogy in higher education: The SLATE project.
University and the Research Centre at the Faculty of Arts for fund- Palgrave Macmillan.
ing this article. The author would also like to thank RSSU at King Drury, H., O’Carroll, P., & Langrish, T. (2006). Online approach to
Saud University for their technical support. teaching report writing in chemical engineering: Implementation
12 SAGE Open
and evaluation. International Journal of Engineering Lemke, J. (2005). Multiplying meaning: Visual and verbal semiot-
Education, 22(4), 858–867. http://www.ijee.ie/articles/Vol22- ics in scientific text. In J. Martin & R. Veel (Eds.), Reading
4/16_ijee1747.pdf science: Critical and functional perspectives on scientific dis-
Guo, L. (2004). Multimodality in a biology textbook. In K. course (pp. 87–113). Routledge.
O’Halloran (Ed.), Multimodal discourse analysis: Systemic- Liu, M., & Braine, G. (2005). Cohesive features in argumentative
functional perspectives (pp. 196–219). Continuum. writing produced by Chinese undergraduates. System, 33(4),
Hall, S., Basran, J., Paterson, K. B., Kowalski, R., Filik, R., & 623–636. https://doi.org/10.1016/j.system.2005.02.002
Maltby, J. (2014). Individual differences in the effectiveness Lovejoy, K. B. (1991). Cohesion and information strategies in
of text cohesion for science text comprehension. Learning and academic writing: Analysis of passages in three disciplines.
Individual Differences, 29, 74–80. http://dx.doi.org/10.1016/j. Linguistics and Education, 3(4), 315–343.
lindif.2013.10.014 Martin, J. R. (1992). English text: System and structure. John
Halliday, M. A. K. (2004). The language of science. In J. Webster Benjamins.
(Ed.), Collected works of M.A.K. Halliday (Vol. 5, pp. 162– Martin, J. R. (2001). Cohesion and texture. In D. Schiffrin, D.
178). Continuum. Tannen, & H. E. Hamilton (Eds.), Handbook of discourse
Halliday, M. A. K. (2014). Introduction to functional grammar analysis (pp. 35–53). Blackwell.
(Rev. C. M. I. M. Matthiessen, 4th ed.). Taylor & Francis. Martı́nez, I. A. (2003). Aspects of theme in the method and dis-
Halliday, M. A. K., & Hasan, R. (2014). Cohesion in English. cussion sections of biology journal articles in English. Journal
Routledge. of English for Academic Purposes, 2(2), 103–123. https://doi.
Hannus, M., & Hyönä, J. (1999). Utilization of illustrations during org/10.1016/S1475-1585(03)00003-1
learning of science textbook passages among low-and high- Mohamed-Sayidina, A. (2010). Transfer of L1 cohesive devices
ability children. Contemporary Educational Psychology, 24(2), and transition words into L2 academic texts: The case of
95–123. http://dx.doi.org/10.1006/ceps.1998.0987 Arab students. RELC Journal, 41(3), 253–266. http://dx.doi.
Hawes, T. (2015). Thematic progression in the writing of stu- org/10.1177/0033688210380569
dents and professionals. Ampersand, 2, 93–100. http://dx.doi. Mohammed, A. S. (2015). Conjunctions as cohesive devices in the
org/10.1016/j.amper.2015.06.002 writings of English as second language learners. Procedia:
Hessamy, G., & Hamedi, S. (2013). A comparison of the use of Social and Behavioral Sciences, 208, 74–81. https://doi.
cohesive devices in EFL learners’ performance on indepen- org/10.1016/j.sbspro.2015.11.182
dent vs. integrated writing tasks. Study in English Language Nichols, K., Hanan, J., & Ranasinghe, M. (2013). Transforming
Teaching, 1(1), 121–146. the social practices of learning with representations: A
Hsu, P.-L., & Yang, W.-G. (2007). Print and image integration of study of disciplinary discourse. Research in Science Educa-
science texts and reading comprehension: A systemic function, 43(1), 179–208. http://dx.doi.org/10.1007/s11165-011-
tional linguistics perspective. International Journal of Science 9263-0
and Mathematics Education, 5(4), 639–659. http://dx.doi. North, S. (2005). Disciplinary variation in the use of theme in
org/10.1007/s10763-007-9091-x undergraduate essays. Applied Linguistics, 26(3), 431–452.
Humphrey, S., & Hao, J. (2011). Deconstructing written genres http://dx.doi.org/10.1093/applin/ami023
in undergraduate biology. Linguistics & the Human Sciences, O’Halloran, K. (1996). The Discourses of Secondary School
7(1–3), 29–53. https://dx.doi.org/10.1558/lhs.v7i1-3.29 Mathematics [Unpublished doctoral thesis]. Murdoch University.
Hyland, K. (2005). Metadiscourse: Exploring interaction in writ- http://researchrepository.murdoch.edu.au/3360/
ing. Continuum. O’Halloran, K. (1998). Classroom discourse in mathematics: A
Jaipal, K. (2010). Meaning making through multiple modalities in multisemiotic analysis. Linguistics and Education, 10(3),
a biology classroom: A multimodal semiotics discourse analy- 359–388. http://dx.doi.org/10.1016/S0898-5898(99)00013-3
sis. Science Education, 94(1), 48–72. http://dx.doi.org/10.1002/ O’Halloran, K. (1999). Towards a systemic functional analysis of
sce.20359 multisemiotic mathematics texts. Semiotica, 124(1–2), 1–30.
Jones, J. (2006). Multiliteracies for academic purposes: A meta- http://dx.doi.org/10.1515/semi.1999.124.1-2.1
functional exploration of intersemiosis and multimodality in O’Halloran, K. (2003). Intersemiosis in mathematics and science:
university textbook and computer-based learning resources in Grammatical metaphor and semiotic metaphor. In A.-M.
science [Unpublished doctoral thesis]. University of Sydney. Simon-Vandenbergen, M. Taverniers, & L. J. Ravelli (Eds.),
http://hdl.handle.net/2123/2259 Grammatical metaphor: Views from systemic functional lin-
Karreman, J., & Loorbach, N. (2007, October 1–3). Paragraphs guistics (Vol. 236, pp. 337–365). John Benjamins.
or lists? The effects of text structure on web sites [Paper pre- O’Halloran, K. (2005). Mathematical discourse: Language, sym-
sentation]. The International Professional Communication bolism and visual images. Continuum.
Conference, Seattle, WA, United State. O’Halloran, K. (2008). Mathematical and scientific forms of
Korani, A. (2012). A survey of the cohesive ties—Reference and lexical knowledge: A systemic functional multimodal grammati-
cohesion—In history books of the second and third grades in guid- cal approach. In F. Christie & J. R. Martin (Eds.), Language,
ance school in Iran. Procedia: Social and Behavioral Sciences, 47, knowledge and pedagogy: Functional linguistic and sociologi-
240–243. http://dx.doi.org/10.1016/j.sbspro.2012.06.645 cal perspectives (pp. 205–236). Continuum.
Kress, G. (2003). Literacy in the new media age. Routledge. Okawa, T. (2008). Academic literacies in the Discipline of Nursing:
Kress, G., & van Leeuwen, T. (2006). Reading images: The gram- Grammar as a resource for producing texts [Unpublished mas-
mar of visual design. Routledge. ter’s thesis]. The University of Adelaide.
Alyousef 13
O’Toole, J. M., & Schefter, M. (2008). Patterns of student dif- Textalyser. (2004). Analysis Tool, Version 1.05. http://textalyser.
ficulty with science text in undergraduate biology courses. net/
International Journal of Learning, 15(1), 133–148. https://doi. van Leeuwen, T. (2005). Multimodality, genre and design. In S.
org/10.18848/1447-9494/CGP/v15i01/45565 Norris & R. Jones (Eds.), Discourse in action: Introducing
O’Toole, M. (1994). The language of displayed art. Leicester mediated discourse analysis (pp. 73–94). Routledge.
University Press. Wahid, R., & Wahid, A. (2020). A study on cohesion in the writing
Ozuru, Y., Dempsey, K., & McNamara, D. S. (2009). Prior knowl- of EFL undergraduate students. Journal on English Language
edge, reading skill, and text cohesion in the comprehension Teaching, 10(1), 57–68.
of science texts. Learning and Instruction, 19(3), 228–242. Yuniartiah, E., Hikmah, S., & Baihaqi, A. (2018). Cohesive
http://dx.doi.org/10.1016/j.learninstruc.2008.04.003 devices in EFL students’ expository writing (a content
Saudi Ministry of Education. (2016). Number of students regis- analysis). In AISELT 2018 Proceedings (pp. 95–108). http://
tered in dentistry colleges per sector, nationality, and gender. aiselt.untirta.ac.id/proceedings/index.php/aiselt/article/
Ministry of Education. view/107
ROMANIAN JOURNAL OF ENGLISH STUDIES
RJES 17 /2020
DOI: 10.1515/RJES-2020-0009
A MULTIMODAL DISCOURSE ANALYSIS IN MEDIA
ION-SORIN LUCA
West University of Timișoara
Abstract: This study attempts a multimodal discourse analysis of a newspaper article during the Brexit
campaign. The aim is to help the audience decode and evaluate photographs and texts from media by
providing a few strategies as guidance. The approach adopted for this analytical research is inspired by
Halliday and Matthiessen’s An Introduction to Functional Grammar (2004). Consequently, the objects
in a photograph and words in a text function similarly conveying information to the audience, and
additionally, involve a similar strategy of analysis. To sum up, these strategies are intended to improve
the audience’s comprehension of decoding article meaning and journalist’s intention.
Keywords: Discourse Analysis, linguistics, multimodal, photograph.
1. Introduction
This study attempts a multimodal discourse on several modes of analysis used in an

online newspaper article during the Brexit campaign. The article is from Daily Mail (May 15,
2016) and is interpreted according to different modes of analysis used in multimodality. This
study aims to help the audience paying attention to the approaches adopted by decoding and
evaluating from online newspaper photographs and sentences. Besides, attention will be paid
to objects in the image which are not taken at random but on purpose, and these modes
include objects, words, colours, gestures, symbols, and the relation between these.
Multimodal discourse takes account of the fact that many scholars paid attention to the
analysis of words, clauses, or texts, but only a few who take account of both of the non-verbal
and verbal language interpreting them together in the analysis of photographs or video clip
with the text. Firstly, the study opens with a discussion of the multimodal discourse approach
and the strategies used to decode and interpret the modes in the photograph. Moreover, there
should be mentioned that theories and strategies used are applied in a top-down order
according to Fairclough’s dimensions from his book Language and Power (1989) as a frame
and includes elements from Kress & van Leeuwen’s work Reading Images – The Grammar of
Visual Design (2006), and Halliday and Matthiessen’s An Introduction to Functional
Grammar (2004) that pays attention to words and clauses from a functional perspective.
Additionally, to the strategies mentioned, it will be added techniques of reading thoughts and
gestures in an article derived from Peas’ book The Definitive Book of Body Language (2004).
2. Theoretical Background
The term multimodality was used for the first time at the Sydney school of semiotics
by M.A.K. Halliday and refers to the modes of analysis used that could be applied to objects
and words inferred from semiotics. Besides, multimodality finds its roots in Prague School
that was focused on linguistics and slowly shifted also in art and theatre interpretation.
Additionally, these strategies developed in linguistics, and consequently, they were applied by
different scholars to old paintings and photographs analyzing costumes as a source of
information for a person status, age, religion, but also for period or event, settings, gestures or
actions (van Leeuwen, 2015: 448). According to van Leeuwen (2015: 447), multimodality
refers to the discourse which implies more characteristics of communication apart from
74
RJES 17 /2020
spoken discourse such as “voice, gestures, facial expressions or aspects of self-presentation”.

Multimodality involves strategies used by the semiotic analysis which implies semiotic modes
as image, sound, and language for interpretation developed alongside technology used in
media. Also, Kress’s (2012: 38) perspective regarding multimodality is seen as an
interdisciplinary field that comprises theories and approaches from different areas of research.
One of those is linguistics that as any other field is treated equally and is a part of
multimodality discourse that comes at the package as any other discipline to contribute to this
approach. Each discipline implies a different knowledge specific to its mode of analysis. For
instance, the analysis of the image that appeals to the history of art in which the language is
involved partially. Consequently, there could not be a complete analysis of a text without the
implication of other modes as it would be the knowledge of the subject which could imply
certain objectivity, but it is masked by different interpretations due to the subjectivity of the
participants (2012: 38). The carrier of the meaning in the case of a text is the language that
with the help of other modes creates a unique interpretation by a listener, a reader, or a
viewer, in this case of a newspaper.
The coherence of text implies the cohesion from a semiotic point of view creates the
link between internal and external elements of the text. Through the coherence of the text, the
real intentions of the writer or speaker are spread and picked up by the audience as an
“immaterial discourse (s)” interpreted according to their knowledge and ideology that stands
for “a specific configuration of discourse in any other text” (Kress, 2012: 36). An article is
represented through “an ensemble of modes” which involves more or less five modes related
to cultural knowledge and convention in the meaning. These modes are “layout, colour,
writing, image and font” (Kress, 2012: 39). Each of them has its role, “writing tells, image
shows, colour frames highlights, and layout and font” refer to the arrangement of objects from
picture involving a certain reason for this (2012: 39).
The genre is the principle that makes it possible for modes to be seen as a body that
implies several stages with “specific communicative functions” in a specific order with a
specific purpose(s). Usually, in a newspaper, it starts with a hook for catching the reader's
attention as “an enticement” (van Leuween, 2015: 454) to open somebody’s curiosity
regarding a popular subject. Thus, the order of the text or the objects from the image is firstly
settled to emphasize specific dates in arguments and persuade the reader or viewer. Besides,
the interpretation of the text from an article could be done according to the strategies of
Systemic Functional Linguistic, more specific, the transitive process. Consequently, the
transitivity process which is part of ideational metafunction includes six types of processes
that are represented through specific characteristics of verb functions in a clause. These
processes are material, relational, mental, verbal, existential, and behavioural. Each of them
according to the order presented implies characteristics as doing or happening, identification
or characterization, way of thinking over an issue, reaction to an issue derived from a cause,
description of an issue taken as real, and features from the consciousness reflected in the
action of the actor. Besides, many analysts mentioned above adopted Halliday’s systemic
functional linguistic to analyse also the photographs from newspapers attributing to the
objects from photograph functions of words.
3. The Approach Adopted
According to Norman Fairclough (1989: 109), discourse implies three stages for
critical analysis: “description, interpretation, and explanation of text”. The first one stands for
“the formal properties of the text” (Fairclough, 1989: 110) and the way it is given, written, or
spoken as an object of study. On the other hand, the interpretation refers to the “relationship
between text and interaction” (1989: 141) emphasizing certain features: “experiential,
75
RJES 17 /2020
relational, expressive, connective or a combination between these” (1989: 141). More or less
these are related to the three metafunctions from SFL elaborated by Halliday. Additionally,
the explanation is “the relationship between interaction and social context” (Fairclough, 1989:
26) that stands for the status of the participants and their knowledge or ideology reflected in a
specific social situation. Moreover, Halliday’s metafunctions are ideational, interpersonal, and
textual (Halliday & Matthiessen, 2004: 29). “The ideational metafunction” includes two
functions “the logical and experiential”. The first one stands for the meaning of a clause and
its “logical-semantic meaning” related to other clauses. Additionally, the interpersonal
metafunction implies different characteristics of the previous one which is inferred from
clauses as “offers, orders, asking questions, giving information or expressing attitudes”
specific to a certain topic (Halliday& Matthiessen 2004:30). Besides, Fairclough highlights
that Halliday’s metafunctions are intertwined and could be simplified involving a sociological
viewpoint than linguistic. Consequently, this study will accept Halliday’s functions specifying
that the textual function is following Fairclough’s viewpoint (2003: 27), in other words, being
encapsulated in the other two functions. Besides, the text interpretation using SFL requires the
reader to possess knowledge of grammar skills for splitting the text into clauses and identify
the transitivity process. For example, the headline of the article is split and noted as “(1a) D
Day hero Lord Bramall leads attacks on Boris Johnson’s ‘laughable’ claim (1b) the EU’s
superstate ambitions are like Hitler’s”. Thus, the verbs’ functions are interpreted to the
characteristics of the transitive process mentioned above. Also, splitting all the text here is not
possible for reasons of space. On the other hand, in the analysis of the image according to van
Leeuween (2008: 138), the analyst should take account of the “representation” of
photography that is emphasized in three dimensions. Firstly, it is “the distance” in which the
shot could be close or far. The next one is “the relation” which implies two elements one of
“involvement and another of detachment”. The first one is done from a frontal angle and the
other from the oblique angle. The second element is “one of power” and refers to the level at
which the viewer sees the image that could be high, eye-level, or low angle. Lastly, “the
interaction” dimension refers to the viewer and the actors from an image whether they interact
with him or not through their looks as a bridge of communication. If their look directly means
that they are close to us, if not, their look is indirectly and mean that they are just like
everybody else, strangers.
4. Analysis of the Article
4.1. Description
The article from Daily Mail which is analyzed is available at:
https://www.dailymail.co.uk/news/article-3591180/Boris-Johnsoncompares-EU-HITLER-
warns-ambitions-European-superstate-end-tragically.html.
The headline of the article from the very beginning stands as a suggestive element for
an assessment of the journalist regarding the old soldier’s statements who was present in WW
II against Boris Johnson’s viewpoints regarding Europe. Anyway, the five sentences
emphasizing the topics of the article as ‘a lead’ contradict the expectations of the reader.
Besides, through ‘intertextuality’ the ex-mayor of London labels the EU with Nazi European
Empire. The insertion of history in a political event is used to get an emotional effect from a
certain period of British history against Nazi Germany alluding to Germany’s dream of ruling
all Europe.
Thus, the text is divided into 35 paragraphs including a schema related to the opinion
reportage genre specific to the media field and reflects two interviews of two important actors
in the person of Lord Bramall, a retired military who fought in WW II and ex-mayor of
London, Boris Johnson. Their rhetoric remarks are expressed in the paragraphs which stand
76
RJES 17 /2020
for the specificity of the genre through introducing the most important aspect of information,
a summary of the news fact, the quotations of the interviews, the addition of the new
information, and the contextualization from a double perspective, situational context and
historical.
4.2. Interpretation
The headline of the article anchors the reader directly in the background and context
category until the end of the journalistic text standing for the main topic. The journalist appeals
to audience knowledge and imagination to raise curiosity through the importance of insinuated
consequences. Consequently, one of the signals, “D Day”, refers to a specific day in the
calendar used by the military. Besides, in history, it refers to the battle during World War II of
the allies against Nazi Germany. On the other hand, only mentioning Hitler’s name to the
audience rejects any association with one of the darkest pages of humankind in which he is
guilty of killing millions of people.
The first photograph is Boris Johnson’s and is situated in the proximity of the main
topic even if the image is not from the interview given to The Telegraph is related to the
headline. The distance of the shot reflects the familial zone and the closeness of him by the
viewer, and the head down and eyes up implies a persuasive strategy that reflects his status.
Additionally, the raised hand works for catching the audience’s attention like a magnet but
makes also an allusion to Hitler’s salute. Anyway, the photograph was chosen for the allusion
presented above as a support for text. Consequently, the photographer tries to express
impartiality according to the oblique angle, but the proximity says the contrary. The lectern is
the symbol of authority and the red color implies strength, determination, and good
communication with his audience giving him the status of a leader. Moreover, the cube was
chosen with the role of stability but at the same time for the value of traditions and taking
back control. In contrast, Lord Bramall’s photograph is smaller positioned on the right side of
the text from an angle which implies a shot taken from a long distance even if it is on zoom.
Even if the military uniform denotes respect, admiration, and security, the dimension of the
photograph in a similar way with those of advertising in a corner could be passed unnoticed.
The uniform is situated on the right side of the article as a symbol of tradition, honor, bravery,
but at the same time, the image of his looking out of the frame of shot could stand for a
different world of the photographer.
The message of the third photograph of the bus refers to the amount of money paid by
the UK to the EU and stands as one of the main factors of the referendum. First of all, the
350000000 pounds is the gross of money, not the net 250000000, where the amount of money
net per year is 11 billion and not 15 billion, and additionally, the leavers do not also mention
the EU’s money left in the UK such as regional funds and subsidies for farmers. In contrast,
Cooper’s photograph as opposition to Johnson represents her with eyes up, a feature of
superiority while her hands standing for the meaning of a reliable person. For Benn, the
photograph, according to his position, implies an air of arrogance due to his unpleasant
behaviour, related to his whistling lips and his hands in pockets looking uninterested in a
conversation. The last shot is given by the video clip paused on Boris Johnson’s image
standing as ‘Do it or else you will suffer the consequences!’ obviously, a clear ‘Vote leave!’ In
addition, imperative signals express also the intimidation of the audience.
The transitive process starts from the headline (1a) with the material process which
represents a percentage of 30.48 and stands for interpretation of characteristics such as doing
and happening. Consequently, the actor “D Day hero” is also the theme and implies a triple
meaning, a concern of the journalist, (re)action taken by the subject against a politician, and
the topic about the journalist wants to give details. As the predicator “leads” suggests, the
actor tries to clarify from experience the overstatements made by Johnson through the scope
77
RJES 17 /2020
“attacks”. Additionally, the apposition, “Lord Bramall”, highlights the identity of the actor
giving a tone of astonishment to the text amplifying curiosity. In contrast to clause 1, no. 6
contains details as the circumstance of the cause “for comparing the EU . . .” which expresses
the consequences of his intention. In both clauses, the real intention of D Day hero is
highlighted and it contradicts the politician’s aberration. From the inner perspective of the
actor, the topic is shifted in the outside world through the clauses 3bc, “fail” and “had done”
in which Johnson’s claims reflect a possible context in which the EU will collapse. These
statements are represented as a solid argument emphasized through a circumstance of location
(time), “for the past 2000 years”, and as a long previous experience that stands for the
incapability of being homogeneous as a state. This is related to Johnson’s competence as he
wrote “an acclaimed book” on Winston Churchill. Johnson’s rhetorical skills are perceived in
arguments and mentions of historical characters, for instance, through “history. . . repeated”
(12c) and the multiple subjects “Napoleon, Hitler, various people” (13ac) in which the main
topic is linked to the audience’s knowledge and the verbs appeal to distrust and fears mirrored
from actions of the thirst for power as Napoleon and Hitler. On the other hand, Johson’s
position (21b) is harshly criticized by his opponents as through the scope “a nasty, nasty
game” and ridiculized for his comments about European history related to political matters. In
contrast, the journalist mentions (39a) a possible leaving of Cameron and suggesting that
Johnson could be the one.
The relational process is represented by a percentage of 32.38 and stands for the outer
world perceived by actors involved as Johnson, Lord Bramall, and obviously, the journalist
according to the topic of the context through the two features, identification, and
characterization related to knowledge and interests. The process (1b), “is like Hitler’s”, starts
with the main concerning of the article and subject of the clause, “the EU’s superstate
ambitions”, which is reflected from the very beginning in Johnson’s position and relationship
with the EU and the way he interacts with the other actors involved in the Brexit referendum.
The negative features attributed to Johnson are as an effect of an ex-military member due to
his knowledge and experience and raises to the aimed audience curiosity and frustration. This
idea is repeated also in clause 2c intensive through the carrier represented by the EU and
Hitler to whom their attribute “similar political goals of a superstate” represent their desire of
possession but without mentioning a different ideological viewpoint gives a negative
connotation to the text. In contrast, the politician statement (4b) is sanctioned through the D
Day hero's evaluation, “simply laughable”. The relation of Lord Bramall with the context of
WWII and the present social situation of Brexit could pass unnoticed but the reality lies in
that experiences lived and in the absurd comparison with a peaceful context. Johnson’s
relationship with the opposition is perceived in attributed syntagma “dog whistle attacks”
(23). Besides, he is seen in (29b) by other politicians as “offensive and desperate” standing
for inexperience, childish and ill-mannered. In contrast, his partners see in him “an effective
historian” (37bc, 38b).
Additionally, the mental process represents actors’ way of thinking reflected in the
verbs related to this. Thus, Johnson (16c) emphasizes his feelings regarding the EU as a state
that should be respected, but this could not happen as the EU’s leadership creates chaos and
dissension. Lord Brammal (18b) expresses clearly his opinion about Johnson’s viewpoint
appealing to the audience's knowledge through the adverb “too well”. Additionally, Paddy
Ashdown (27b), a liberal democrat, thinks that the electors are exasperated by Johnson’s role-
play of acting PM Winston. The issue is not that Boris Johnson expresses the way he feels
regarding the EU but it is when there are more people (35a) who thinks the same.
The verbal process is seen as a (re)action and is represented by the verb “said” (2b)
which stands for a verbal reaction of the actors on the purpose to affect electors’ behavior.
The warning used by Boris Johnson regarding the history of the old continent (3a) offers
78
RJES 17 /2020
information known as true but introduced in present contexts through ‘intertextuality’ distorts
the real information regarding the UK benefits. Additionally, Johnson’s remarks not once are
contradictory. He accuses PM “undermining Nato”, and consequently, the security, which is
the main characteristic of the alliance and the EU in which almost all the EU’s states are, is
rejected (10b, 11a).
The existential process, represented by “there is” stands for the description of the main
topic which belongs to the context, in this case, the description of trust regarding the EU (15e,
16a, 34) which shoots arrows to the past which functions as predictable consequences of
context for future such as the dismemberment of the EU.
The behavioural process implies the features of consciousness and those of action
from mental and material processes (16b) which reflects the position of Johnson regarding the
EU as an effect of the outer world. This effect could be represented by the populism direction
which is spread at the moment in Europe by representatives as Le Pen, Salvini, and Orban.
Along the same line, his opponents describe his behaviour as he “flails his hysterical claims”
(22a) and is directly linked to the topic. Moreover, his behaviour (22b) is reflected in
Cooper’s enumeration which describes Johnson’s behaviour as deriving from his “shameful
lack of judgement”.
Taking account of SFL the analysis reflects the fact that the journalist wants to create a
strong bond with the audience denigrating the government and the EU before the Brexit
referendum through a relational process (32.38%). The implication aims at a productive effect
through the way the reader and the journalist (inter)act. This is seen in the percentage of the
second process, the material one, of 30.48 which reflects the direction of acting given by the
journalist. Furthermore, the information is given through the verbal process (23.81%) and is
adapted to the interest and style of a journalist to support the institution’s interest. The great
disappointment stands in the lower rate of mental (7.61%) and existential process (2.86%)
which should reflect the rational and affective thinking regarding the issues encountered in
the real world. And lastly, the mental behaviour (2.86%) being reduced almost completely to
the text in an article and omitting visible body action features could be inferred from the
verbal process and could be misinterpreted.
4.3. Explanation
From the very beginning, the aim of the article reflects political implications and is
addressed to all types of audiences with the role of changing the manner of action of the
readers. The arguments presented by journalist reflects three social concerns. Firstly, the EU
is presented by Johnson’s statements as if a wise saying stood for old habits die hard.
Secondly, an economic perspective is presented almost without raising flags as Johnson’s bus
covers the explanation of the issue. Lastly, as in any political campaign, there is a newspaper,
a television channel, or more to represent the interests of a certain party.
The first social concern hides a deep implication of a populism ideology that stands for
a specific kind of class. Here the politician appeals to the past to open some wounds of the
unhappy history of Europe. One of these reasons is the new world order after the second
world war in which Britain lost its supremacy once with its dismemberment. On the other
hand, the attack on Nazy Germany is a subtle allusion to nowadays Germany which from an
economical point of view is ‘breathing down its neck’ and the UK is not used to be in second
place. Anyway, having opinions on certain issues is something normal and solutions should
be found to such disagreements not turning back and walking away. That is why the reactions
of the opposition and the position of persons with experience as Lord Bramall never stop
coming. The issue related to the economy in which the amount of money paid to the EU
mentioned above and the arguments used by actors without experience or actors with
experience but without a cold analysis distort the audience’s opinion from reality and
79
RJES 17 /2020
persuade it through appealing to British nationalism. It is something very common to be

pushed in front of such political characters, experts in all fields that know all and nothing.
Furthermore, the article is used as a subtle conclusion to disseminate indirectly opinions
according to the institution’s interests.
5. Conclusion
The study of multimodal discourse has demonstrated that without mentioning the
strategies developed by these scholars it would be almost impossible to interpret a photograph
or a text similarly. Moreover, if the audience reads the article, the journalist’s aim is partially
achieved only thinking of the photographs presented as proof that stands for a hook. The most
difficult characteristic during the analysis is that of staying rational without any involvement
of the belief or the ideology in the interpretation of the photography and the text. Besides, an
interesting result could be obtained from a survey made on the photography and the text
within the general readership of the newspaper.
Taking account of these defining characteristics is no need for other interpretations
than the text and photographs stand for Brexit context which is also related to consequences
of such a referendum for EU countries submitted to common law and security. The concern is
the position adopted by the UK which could also be adopted by other states and could escalate
as an act of rebellion against Germany and Brussel.
References
Books:
Fairclough, Norman. 2003. Analysing Discourse. Textual analysis for social research, Routledge: London and
New York.
Fairclough, Norman. 1989. Language and Power, Longman: New York.
Halliday, M.A.K, Matthiessen Christian M.I.M. 2004. An Introduction to Functional Grammar, Third Edition,
Hodder Arnold: London.
Kress, Gunther and Van Leeuwen, Theo. 2006 (1996). Reading Images - The Grammar of Visual Design.
Second Edition. Routledge: London and New York.
Pease, Alan & Pease Barbara. 2004. The Definitive Book of Body Language. McPherson's Printing Group.
van Leeuwen, Theo. 2008. Discourse and Practice. New Tools for Critical Discourse Analysis, Oxford
University Press.
van Leeuwen, Theo. 2015. “Multimodality.” The Handbook of Discourse Analysis, Volume I and II, edited by
Deborah Tannen, Heidi E.Hamilton and Deborah Schiffrin, Blackwell Publishers,
Malden and Oxford.
Internet sources:
Schulthorpe, Tim. 2016. “D Day hero Lord Bramall leads attacks on Boris Johnson's 'laughable' claim the EU's
superstate ambitions are like Hitler's”. Daily Mail”. May 15. Available:https:// www.
dailymail.co.uk/news/article-3591180/Boris-Johnson-compares-EU-HITLER-warns-ambitions-
European- superstate-end-tragically.html [Accessed: August 22, 2020].
Note on the author

Ion-Sorin LUCA is a PhD student in Philology at the West University of Timișoara and
holds an MA in Philology. He works as a secondary school teacher. His research has partly
been published in prestigious journals and international volumes.
80
About Publish with us Subjects News Contacts En
Volume 17 (2020): Issue 1 (December 2020)
Romanian Journal of English Studies

Journal Details
Format: Journal
First Published: 20 Feb 2013
Publication timeframe: 1 time per year
Languages: English
Open Access
A Multimodal Discourse Analysis in Media

Ion-Sorin Luca
Published Online: 04 Jan 2021

Page range: 74 - 80
DOI: https://doi.org/10.1515/rjes-2020-0009
© 2020 Ion-Sorin Luca, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Download Cite Share Previous Next
PDF Preview

RJES 17 /2020
DOI: 10.1515/RJES-2020-0009
A MULTIMODAL DISCOURSE ANALYSIS IN MEDIA
ION -SORIN LUCA

West University of Timișoara
Abstract: This study attempts a multimodal discourse analysis of a newspaper article during the Brexit
campaign. The aim is to help the audience decode and evaluate photographs and texts from media by
providing a few strategies as guidance. The approach adopted for this analytical research is inspired by
Halliday and Matthiessen’s An Introduction to Functional Grammar (2004). Consequently, the objects
in a photograph and words in a text function similarly conveying information to the audience, and
additionally, invol ve a similar strategy of analysis. To sum up, these strategies are intended to improve
the audience’s comprehension of decoding article meaning and journalist’s intention.
Keywords: Discourse Analysis, linguistics, multimodal, photograph .
1. Introduction
This study attempts a multimodal discourse on several modes of analysis used in an

online newspaper article during the Brexit campaign. The article is from Daily Mail (May 15,
2016) and is interpreted according to different modes of analysi s used in multimodality. This
study aims to help the audience paying attention to the approaches adopted by decoding and
evaluating from online newspaper photographs and sentences. Besides, attention will be paid
to objects in the image which are not takenat random but on purpose, and these modes
include objects, words, colours, gestures, symbols, and the relation between these.
Multimodal discourse takes account of the fact that many scholars paid attention to the
analysis of words, clauses, or texts, but only a few who take account of both of the non-verbal
and verbal language interpreting them together in the analysis of photographs or video clip
with the text. Firstly, the study opens with a discussion of the multimodal discourse approach
and the strategies used to decode and interpret the modes in the photograph. Moreover, there
should be mentioned that theories and strategies used are applied in a top -down order
according to Fairclough’s dimensions from his book Language and Power (1989) as a frame
and includes elements from Kress & van Leeuwen’s work Reading Images – The Grammar of
Visual Design (2006), and Halliday and Matthiessen’s An Introduction to Functional
Grammar (2004) that pays attention to words and clauses from a functional perspective.
Additionally, to the strategies mentioned, it will be added techniques of reading thoughts and
gestures in an article derived from Peas’ book The Definitive Book of Body Language(2004).
2. Theoretical Background
The term multimodality was used for the first time at the Sydney school of semiotics
by M.A.K. Halliday and refers to the modes of analysis used that could be applied to objects
and words inferred from semiotics. Besides, multimodality finds its roots inrague P School
that was focused on linguistics and slowly shifted also in art and theatre interpretation.
Additionally, these strategies developed in linguistics, and consequently, they were applied by
different scholars to old paintings and photographs anal yzing costumes as a source of
information for a person status, age, religion, but also for period or event, settings, gestures or
actions (van Leeuwen, 2015: 448). According to van Leeuwen (2015: 447), multimodality
refers to the discourse which implies more characteristics of communication apart from
74

RJES 17 /2020
spoken discourse such as “voice, gestures, facial expressions or aspects of self -presentation”.
Multimodality involves strategies used by the semiotic analysis which implies semiotic modes
as image, sound, and language for interpretation developed alongside technology used in
media. Also, Kress’s (2012: 38) perspective regarding multimodality is seen as an
interdisciplinary field that comprises theories and approaches from different areas of research.
One of those is linguistics that as any other field is treated equally and is a part of
multimodality discourse that comes at the package as any other discipline to contribute to this
approach. Each discipline implies a different knowledge specific to its mode of analysis. For
instance, the analysis of the image that appeals to the history of art in which the language is
involved partially. Consequently, there could not be a complete analysis of a text without the
implication of other modes as it would be the knowl edge of the subject which could imply
certain objectivity, but it is masked by different interpretations due to the subjectivity of the
participants (2012: 38). The carrier of the meaning in the case of a text is the language that
with the help of other modes creates a unique interpretation by a listener, a reader, or a
viewer, in this case of a newspaper.
The coherence of text implies the cohesion from a semiotic point of view creates the
link between internal and external elements of the text. Through the coherence of the text, the
real intentions of the writer or speaker are spread and picked up by the audience as an
“immaterial discourse (s)” interpreted according to their knowledge and ideology that stands
for “a specific configuration of discourse in ny a other text” (Kress, 2012: 36). An article is
represented through “an ensemble of modes” which involves more or less five modes related
to cultural knowledge and convention in the meaning. These modes are “layout, colour,
writing, image and font” (Kress, 2012: 39). Each of them has its role, “writing tells, image
shows, colour frames highlights, and layout and font” refer to the arrangement of objects from
picture involving a certain reason for this (2012: 39).
The genre is the principle that makes itossiblep for modes to be seen as a body that
implies several stages with “specific communicative functions” in a specific order with a
specific purpose(s). Usually, in a newspaper, it starts with a hook for catching the reader's
attention as “an enticement” (van Leuween, 2015: 454) to open somebody’s curiosity
regarding a popular subject. Thus, the order of the text or the objects from the image is firstly
settled to emphasize specific dates in arguments and persuade the reader or viewer. Besides,
the interpretation of the text from an article could be done according to the strategies of
Systemic Functional Linguistic, more specific, the transitive process. Consequently, the
transitivity process which is part of ideational metafunction includes six types ofrocesses p
that are represented through specific characteristics of verb functions in a clause. These
processes are material, relational, mental, verbal, existential, and behavioural. Each of them
according to the order presented implies characteristics as doing or happening, identification
or characterization, way of thinking over an issue, reaction to an issue derived from a cause,
description of an issue taken as real, andfeatures from the consciousness reflected in the
action of the actor. Besides, manyanalysts mentioned above adopted Halliday’s systemic
functional linguistic to analyse also the photographs from newspapers attributing to the
objects from photograph functions of words.
3. The Approach Adopted
According to Norman Fairclough (1989: 109), d iscourse implies three stages for
critical analysis: “description, interpretation, and explanation of text”. The first one stands for
“the formal properties of the text” (Fairclough, 1989: 110) and the way it is given, written, or
spoken as an object of st udy. On the other hand, the interpretation refers to the “relationship
between text and interaction” (1989: 141) emphasizing certain features: “experiential,
75

RJES 17 /2020
relational, expressive, connective or a combination between these” (1989: 141). More or less
these are related to the three metafunctions from SFL elaborated by Halliday. Additionally,
the explanation is “the relationship between interaction and social context” (Fairclough, 1989:
26) that stands for the status of the participants and their knowledge or ideology reflected in a
specific social situation. Moreover, Halliday’s metafunctions are ideational, interpersonal, and
textual (Halliday & Matthiessen, 2004: 29). “The ideational metafunction” includes two
functions “the logical and experiential”. The first one stands for the meaning of a clause and
its “logical-semantic meaning” related to other clauses. Additionally, the interpersonal
metafunction implies different characteristics of the previous one which is inferred from
clauses as “offers, orders, asking questions, giving information or expressing attitudes”
specific to a certain topic (Halliday& Matthiessen 2004:30). Besides, Fairclough highlights
that Halliday’s metafunctions are intertwined and could be simplified involving a sociological
viewpoin t than linguistic. Consequently, this study will accept Halliday’s functions specifying
that the textual function is following Fairclough’s viewpoint (2003: 27), in other words, being
encapsulated in the other two functions. Besides, the text interpretatio n using SFL requires the
reader to possess knowledge of grammar skills for splitting the text into clauses and identify
the transitivity process. For example, the headline of the article is split and noted as “(1a) D
Day hero Lord Bramall leads attacks onBoris Johnson’s ‘laughable’ claim (1b) the EU’s
superstate ambitions are like Hitler’s”. Thus, the verbs’ functions are interpreted to the
characteristics of the transitive process mentioned above. Also, splitting all the text here is not
possible for reasons of space. On the other hand, in the analysis of the image according to van
Leeuween (2008: 138), the analyst should take account of the “representation” of
photography that is emphasized in three dimensions. Firstly, it is “the distance” in which the
shot could be close or far. The next one is “the relation” which implies two elements one of
“involvement and another of detachment”. The first one is done from a frontal angle and the
other from the oblique angle. The second element is “one of power” and refers to the level at
which the viewer sees the image that could be high, eye -level, or low angle. Lastly, “the
interaction” dimension refers to the viewer and the actors from an image whether they interact
with him or not through their looks as a bridge of communication. If their look directly means
that they are close to us, if not, their lo ok is indirectly and mean that they are just like
everybody else, strangers.
4. Analysis of the Article
4.1. Description
The article from Daily Mail which is analyzed is available at:
https://www.dailymail.co.uk/news/article -3591180/Boris -Johnsoncompares-EU-HITLER-
warns-ambitions -European-superstate-end-tragically.html .
The headline of the article from the very beginning stands as a suggestive element for
an assessment of the journalist regarding the old soldier’s statements who was present in WW
II against Boris Johnson’s viewpoints regarding Europe. Anyway, the five sen tences
emphasizing the topics of the article as ‘a lead’ contradict the expectations of the reader.
Besides, through ‘intertextuality’ the ex -mayor of London labels the EU with Nazi European
Empire. The insertion of history in a political event is used to get an emotional effect from a
certain period of British history against Nazi Germany alluding to Germany’s dream of ruling
all Europe.
Thus, the text is divided into 35 paragraphs including a schema related to the opinion
reportage genre specific to the media field and reflects two interviews of two important actors
in the person of Lord Bramall, a retired military who fought in WW II and -mayor ex of
London, Boris Johnson. Their rhetoric remarks are expressed in the paragraphs which stand
76

RJES 17 /2020
for the specific ity of the genre through introducing the most important aspect of information,
a summary of the news fact, the quotations of the interviews, the addition of the new
information, and the contextualization from a double perspective, situational context and
historical.
4.2. Interpretation
The headline of the articleanchors the reader directly in the background and context
category until the end of the journalistic text standing for the main topic. The journalist appeals
to audience knowledge and im agination to raise curiosity through the importance of insinuated
consequences. Consequently, one of the signals, “D Day”, refers to a specific day in the
calendar used by the military. Besides, in history, it refers to the battle during World War II of
the allies against Nazi Germany. On the other hand, only mentioning Hitler’s name to the
audience rejects any association with one of the darkest pages of humankind in which he is
guilty of killing millions of people.
The first photograph is Boris Johnson ’s and is situated in the proximity of the main
topic even if the image is not from the interview given The to Telegraph is related to the
headline. The distance of the shot reflects the familial zone and the closeness of him by the
viewer, and the head dow n and eyes up implies a persuasive strategy that reflects his status.
Additionally, the raised hand works for catching the audience’s attention like a magnet but
makes also an allusion to Hitler’s salute. Anyway, the photograph was chosen for the allusion
presented above as a support for text. Consequently, the photographer tries to express
impartiality according to the oblique angle, but the proximity says the contrary. The lectern is
the symbol of authority and the red color implies strength, determinatio n, and good
communication with his audience giving him the status of a leader. Moreover, the cube was
chosen with the role of stability but at the same time for the value of traditions and taking
back control. In contrast, Lord Bramall’s photograph is smal ler positioned on the right side of
the text from an angle which implies a shot taken from a long distance even if it is on zoom.
Even if the military uniform denotes respect, admiration, and security, the dimension of the
photograph in a similar way with those of advertising in a corner could be passed unnoticed.
The uniform is situated on the right side of the article as a symbol of tradition, honor, bravery,
but at the same time, the image of his looking out of the frame of shot could stand for a
different world of the photographer.
The message of the third photograph of the bus refers to the amount of money paid by
the UK to the EU and stands as one of the main factors of the referendum. First of all, the
350000000 pounds is the gross of money, not the net 250000000, where the amount of money
net per year is 11 billion and not 15 billion, and additionally, the leavers do not also mention
the EU’s money left in the UK such as regional funds and subsidies for farmers. In contrast,
Cooper’s photograph as opp osition to Johnson represents her with eyes up, a feature of
superiority while her hands standing for the meaning of a reliable person. For Benn, the
photograph, according to his position, implies an air of arrogance due to his unpleasant
behaviour, related to his whistling lips and his hands in pockets looking uninterested in a
conversation. The last shot is given by the video clip paused on Boris Johnson’s image
standing as ‘Do it or else you will suffer the consequences!’ obviously, a clear ‘Vote leave!’ In
addition, imperative signals express also the intimidation of the audience.
The transitive process starts from the headline (1a) with the material process which
represents a percentage of 30.48 and stands for interpretation of characteristics such as doing
and happening. Consequently, the actor“D Day hero” is also the theme and i mplies a triple
meaning, a concern of the journalist, (re)action taken by the subject against a politician, and
the topic about the journalist wants to give details. As the predicator “leads” suggests, the
actor tries to clarify from experience the oversta tements made by Johnson through the scope
77

RJES 17 /2020
“attacks”. Additionally, the apposition, “Lord Bramall”, highlights the identity of the actor
giving a tone of astonishment to the text amplifying curiosity. In contrast to clause 1, no. 6
contains details as the circumstance of the cause “for comparing the EU . . .” which expresses
the consequences of his intention. In both clauses, the real intention of D Day hero is
highlighted and it contradicts the politician’s aberration. From the inner perspective of the
actor, the topic is shifted in the outside world through the clauses 3bc, “fail” and “had done”
in which Johnson’s claims reflect a possible context in which the EU will collapse. These
statements are represented as a solid argument emphasized through a circu mstance of location
(time), “for the past 2000 years”, and as a long previous experience that stands for the
incapability of being homogeneous as a state. This is related to Johnson’s competence as he
wrote “an acclaimed book” on Winston Churchill. Johnso n’s rhetorical skills are perceived in
arguments and mentions of historical characters, for instance, through “history. . . repeated”
(12c) and the multiple subjects“Napoleon, Hitler, various people” (13ac) in which the main
topic is linked to the audienc e’s knowledge and the verbs appeal to distrust and fears mirrored
from actions of the thirst for power as Napoleon and Hitler. On the other hand, Johson ’s
position (21b) is harshly criticized by his opponents as through the scope“a nasty, nasty
game” and ridiculized for his comments about European history related to political matters. In
contrast, the journalist mentions (39a) a possible leaving of Cameron and suggesting that
Johnson could be the one.
The relational process is represented by a percentage o f 32.38 and stands for the outer
world perceived by actors involved as Johnson, Lord Bramall, and obviously, the journalist
according to the topic of the context through the two features, identification, and
characterization related to knowledge and intere sts. The process (1b), “is like Hitler’s”, starts
with the main concerning of the article and subject of the clause, “the EU’s superstate
ambitions”, which is reflected from the very beginning in Johnson’s position and relationship
with the EU and the way he interacts with the other actors involved in the Brexit referendum.
The negative features attributed to Johnson are as an effect of an ex-military member due to
his knowledge and experience and raises to the aimed audience curiosity and frustration. This
idea is repeated also in clause 2c intensive through the carrier represented by the EU and
Hitler to whom their attribute “similar political goals of a superstate” represent their desire of
possession but without mentioning a different ideological viewpoi nt gives a negative
connotation to the text. In contrast, the politician statement (4b) is sanctioned through the D
Day hero's evaluation,“simply laughable”. The relation of Lord Bramall with the context of
WWII and the present social situation of Brex it could pass unnoticed but the reality lies in
that experiences lived and in the absurd comparison with a peaceful context. Johnson’s
relationship with the opposition is perceived in attributed syntagma “dog whistle attacks”
(23). Besides, he is seen in29b) ( by other politicians as “offensive and desperate” standing
for inexperience, childish and ill -mannered. In contrast, his partners see in him “an effective
historian” (37bc, 38b).
Additionally, the mental process represents actors’ way of thinking reflected in the
verbs related to this. Thus, Johnson (16c) emphasizes his feelings regarding the EU as a state
that should be respected, but this could not happen as the EU ’s leadership creates chaos and
dissension. Lord Brammal (18b) expresses clearly his opinion about Johnson’s viewpoint
appealing to the audience's knowledge through the adverb“too well”. Additionally, Paddy
Ashdown (27b), a liberal democrat, thinks that the electors are exasperated by Johnson’s role -
play of acting PM Winston. The issue isotnthat Boris Johnsonexpresses the way he feels
regarding the EU but it is when there are more people (35a) who thinks the same.
The verbal process is seen as a (re)action and is represented by the verb “said” (2b)
which stands for a verbal reaction ofhet actors on the purpose to affect electors’ behavior.
The warning used by Boris Johnson regarding the history of the old continent (3a) offers
78

RJES 17 /2020
information known as true but introduced in present contexts through ‘intertextuality’ distorts
the real informa tion regarding the UK benefits. Additionally, Johnson’s remarks not once are
contradictory. He accuses PM “undermining Nato”, and consequently, the security, which is
the main characteristic of the alliance and the EU in which almost all the EU’s states ar e, is
rejected (10b, 11a).
The existential process, represented by “there is” stands for the description of the main
topic which belongs to the context, in this case, the description of trust regarding the EU (15e,
16a, 34) which shoots arrows to the pastwhich functions as predictable consequences of
context for future such as the dismemberment of the EU.
The behavioural process implies the features of consciousness and those of action
from mental and material processes (16b) which reflects the position of Johnson regarding the
EU as an effect of the outer world. This effect could be represented by the populism direction
which is spread at the moment in Europe by representatives as Le Pen, Salvini, and Orban.
Along the same line, his opponents describe hi s behaviour as he “flails his hysterical claims”
(22a) and is directly linked to the topic. Moreover, his behaviour (22b) is reflected in
Cooper’s enumeration which describes Johnson’s behaviour as deriving from his “shameful
lack of judgement”.
Taking acc ount of SFL the analysis reflects the fact that the journalist wants to create a
strong bond with the audience denigrating the government and the EU before the Brexit
referendum through a relational process (32.38%). The implication aims at a productive ef fect
through the way the reader and the journalist (inter)act. This is seen in the percentage of the
second process, the material one, of 30.48 which reflects the direction of acting given by the
journalist. Furthermore, the information is given through ethverbal process (23.81%) and is
adapted to the interest and style of a journalist to support the institution ’s interest. The great
disappointment stands in the lower rate of mental (7.61%) and existential process (2.86%)
which should reflect the rationaland affective thinking regarding the issues encountered in
the real world. And lastly, the mental behaviour (2.86%) being reduced almost completely to
the text in an article and omitting visible body action features could be inferred from the
verbal process and could be misinterpreted.
4.3. Explanation
From the very beginning, the aim of the article reflects political implications and is
addressed to all types of audiences with the role of changing the manner of action of the
readers. The arguments presented by journalist reflects three social concerns. Firstly, the EU
is presented by Johnson’s statements as if a wise saying stood for old habits die hard.
Secondly, an economic perspective is presented almost without raising flags as Johnson ’s bus
covers the explanation of the issue. Lastly, as in any political campaign, there is a newspaper,
a television channel, or more to represent the interests of a certain party.
The first social concern hides a deep implication of a populism ideology th at stands for
a specific kind of class. Here the politician appeals to the past to open some wounds of the
unhappy history of Europe. One of these reasons is the new world order after the second
world war in which Britain lost its supremacy once with itsismemberment.
d On the other
hand, the attack on Nazy Germany is a subtle allusion to nowadays Germany which from an
economical point of view is ‘breathing down its neck’ and the UK is not used to be in second
place. Anyway, having opinions on certain issuesis something normal and solutions should
be found to such disagreements not turning back and walking away. That is why the reactions
of the opposition and the position of persons with experience as Lord Bramall never stop
coming. The issue related to theeconomy in which the amount of money paid to the EU
mentioned above and the arguments used by actors without experience or actors with
experience but without a cold analysis distort the audience’s opinion from reality and
79

RJES 17 /2020
persuade it through appealing to British nationalism. It is something very common to be

pushed in front of such political characters, experts in all fields that know all and nothing.
Furthermore, the article is used as a subtle conclusion to disseminate indirectly opinions
according to the institution’s interests.
5. Conclusion
The study of multimodal discourse has demonstrated that without mentioning the
strategies developed by these scholars it would be almost impossible to interpret a photograph
or a text similarly. Moreover, if the audience reads the article, the journalist’s aim is partially
achieved only thinking of the photographs presented as proof that stands for a hook. The most
difficult characteristic during the analysis is that of staying rational without any involvement
of the belief or the ideology in the interpretation of the photography and the text. Besides, an
interesting result could be obtained from a survey made on the photography and the text
within the general readership of the newspaper.
Taking account of these defining characteristics is no need for other interpretations
than the text and photographs stand for Brexit context which is also related to consequences
of such a referendum for EU countries submitted to common law and security. The concern is
the position adopted by the UK which could also be adopted by other states and could escalate
as an act of rebellion against Germany and Brussel.
References
Books:
Fairclough, Norman. 2003. Analysing Discourse. Textual analysis for social research, Routledge: London and
New York.
Fairclough, Norman. 1989. Language and Power , Longman: New York.
Halliday, M.A.K, Matthiessen Christian M.I.M. 2004. An Introduction to Functional Grammar , Third Edition,
Hodder Arnold: London.
Kress, Gunther and Van Leeuwen, Theo. 2006 (1996). Reading Images - The Grammar of Visual Design .
Second Edition. Routledge: London and New York.
Pease, Alan & Pease Barbara. 2004. The Definitive Book of Body Language . McPherson's Printing Group.
van Leeuwen, Theo. 2008. Discourse and Practice. New Tools for Critical Discourse Analysis , Oxford
University Press.
van Leeuwen, Theo. 2015. “Multimodality.” The Handbook of Discourse Analysis , Volume I and II, edited by
Deborah Tannen, Heidi E.Hamilton and Deborah Schiffrin, Blackwell Publishers,
Malden and Oxford.
Internet sources:
Schulthorpe, Tim. 2016. “ D Day hero Lord Bramall leads attacks on Boris Johnson's 'laughable' claim the EU's
superstate ambi tions are like Hitler's”. Daily Mail”. May 15. Available: https:// www.
dailymail.co.uk/news/article -3591180/Boris -Johnson -compares-EU-HITLER-warns-ambitions -
European - superstate-end-tragically.html [Accessed: August 22, 2020].
Note on the author

Ion -Sorin LUCA is a PhD student in Philology at the West University of Timișoara and
holds an MA in Philology. He works as a secondary school teacher. His researchpartly
has
been published in prestigious journals and international volum es.
80
Recommended articles from Trend MD
We recommend
“And this is the view from outside my window”: On text and image interplay in university website blogs Multimodal exploration of civic stance in the English Language Arts classroom
Renáta Tomášková, Topics in Linguistics, 2017 Candance Doerr-Stevens, Education + Training, 2017
“Not Even All Physicians Know Chinese Medicine!”: Analysing the Legitimation Strategies of Chinese Open data?
Medicine in the Estonian Media Erik Borglund et al., Records Management Journal, 2014
Katre Koppel et al., Journal of Ethnology and Folkloristics, 2020
Representing Health
Hedging in Popular Scientific Articles on Medicine International Journal of Health Care Quality Assurance, 2005
Alexandra Csongor et al., Acta Medica Marisiensis, 2013
Interdiscursivity in corporate financial communication: an analysis of earnings videos
Strategies to Discredit Opponents: Russian Presentations of Events in Countries of the Former Soviet Union Kumaran Rajandran, Corporate Communications: An International Journal, 2020
Ludmilla A’Beckett, Psychology of Language and Communication, 2013
Ethical aspects of voice assistants: a critical discourse analysis of Indonesian media texts
The Migration Crisis as It Seems: Speech Manipulation Technology in US Internet Media Anisa Aini Arifin et al., Journal of Information, Communication and Ethics in Society
Ksenia Nikitina, American, British and Canadian Studies, 2016
Powered by
Plan your remote conference with Sciendo
Find out more
Publish with us Contact

De Gruyter Poland Sp. z o.o.
Latest News Bogumila Zuga 32a
Sciendo is a De Gruyter company 01-811 Warsaw, Poland
About Sciendo
info@sciendo.com
Contacts
+48 22 701 50 15
Terms
Privacy
Publishing and Ethical Policies
Our partners:
Copyright: © 2021 Sciendo Website by Northern Comfort

International Journal of Systemic Functional Linguistics Vol 4 (1) 2021, 1-4
International Journal of Systemic Functional Linguistics

Journal Homepage: https://ejournal.warmadewa.ac.id/index.php/ijsfl
Multimodal Critical Discourse Analysis on Advertisement of

Colgate
Ni Made Sila Ulati

Universitas Warmadewa
surrendersiwa.2017@gmail.com
ABSTRACT ARTICLE INFO

This study aims to analyze the advertisement of Colgate in audiovisual How to Cite:
formed, when Colgate advertisement is one of the Super Bowl commercial Ulati, N. M. S. (2021). Multimodal
break in USA. This study applies on semantic approach focused on the Critical Discourse Analysis on
multimodal discourse analysis. In the semiotic sense, signs are words, images, Advertisement of Colgate.
sounds, gestures, and objects are also called multimodal, as it is said (Anstey International Journal of Systemic
& Geoff, 2010) a text is said to be a multimodal text if the text has two or Functional Linguistics, 4(1), 1-4.
more semiotic systems. There are five types of semiotic systems, e.g.
linguistic, visual, audio, gestural, and layout. The method applied in this study Doi:
is descriptive qualitative method. The result of this study is Colgate Total https://doi.org/10.22225/ijsfl.v4i1.4064
advertisement covers five aspects in a multimodal semiotic system, namely
linguistic, visual, audio, gestural, and location aspects. These five aspects Article History:
come together to make the ad appearance very attractive and easy to Received :
remember. Revised :
multimodality, advertisement, analysis. Accepted :
Keywords:
Multimodality;
Advertisement;
Analysis;
give the first priority to its marketing aspects.

I. INTRODUCTION Regard with marketing are advertisers. These ad
In any business field that an entrepreneur is makers will compete to present advertisements that
engaged in, strategic marketing is needed to are as attractive as possible, easy to remember by
promote a product. In the world of marketing, the public, have unique characteristics that are
advertising is the spearhead of making a product different from other advertisements, have a
able to survive in the market and become the main beautiful background, and many other factors that
market in society, which acts as the main support the appearance of an advertisement. In an
consumer. Therefore, every business sector will advertisement, of course there is a main message
International Journal of Systemic Functional Linguistics, Volume 4, Nomor 1, 2021. CC-BY-SA 4.0 License
1
that the ad maker wants to convey, which is an conducting a multimodal analysis of a toothpaste
important goal in producing a product. This advertisement, namely Colgate. Colgate
message is expected delivered to the community advertising is one of the most popular fish in the
according to the expectations of the producers, USA, because it includes Super Bowl
hoping that their products are favored by commercials. Super Bowl commercials,
consumers. colloquially known as Super Bowl ads, are high-
The message in an advertisement is contained profile television commercials featured in the U.S.
in the signs that appear in an advertisement. Signs television broadcast of the Super Bowl, the
in linguistics are known as semiotics (Halloran, championship game of the National Football
2004). In semiotics there is an analysis called League (NFL). Super Bowl commercials have
multimodal analysis. The study of multimodal text become a cultural phenomenon of their own
is defined by as a study of signs that are present in alongside the game itself, as many viewers only
society (Anstey and Bull, 2010). In more detail, watch the game to see the commercials. Super
Machin (2012) states that semiotics involves a Bowl advertisementshave become iconic and well
study that not only refers to the study of “signs” in known because of their cinematographic quality,
everyday conversation, but semiotics also unpredictability, surreal humor, and use of special
examines everything that refers to something else. effects.
In this study, the researcher was interested in
II. METHOD III. RESULT AND DISCUSSION

This research will focus on analyzing The following will describe step by step how
semiotic aspects, especially in terms of multimodal critical discourse analysis in Colgate's
multimodal critical discourse analysis in an ad is included in the Super Bowl commercial break
advertisement that is included in the super bowl in the USA, namely:
commercial break, namely Colgate toothpaste ad
which features actor Luke Wilson, in audiovisual 3.1 Linguistic Aspects
format, with duration 0:30 seconds. In this Colgate The spoken words found in the advertising
advertisement, aspects will be analyzed such as column spoken by Luke Wilson as the main
linguistic, visual, audio, gestural, and layout character and two other figures as supporting
aspects. characters, are as follows:
Data collection is done by looking carefully
every second and minute of every display on
Colgate advertisements. For the linguistics aspect,
the researcher will focus on observing every word
spoken by Luke Wilson as a commercial star and
other characters in the advertisement. Besides the
word orally, the researcher will observe the
written word that is displayed in the
advertisement. For the visual aspect, researchers
focus on every display that appears in the
advertisement. For the audio aspect, it is focused
on the music that is the background for the
advertisement. When analyzing the gestural
aspects, the researcher will examine the
movements and body velocities and facial The linguistic analysis in this Colgate Total
expressions of the participants. And finally, for the toothpaste ad is based on language meta
layout aspect, will be focused on how layout of functions: experiential, interpersonal, and textual.
. The analysis is focused on verbal elements in this
2
ad. Experiential function analysis ~
interpersonal. In addition, the verbal meaning in 3.4 Gestural Aspects
the total Colgate ads, strengthened and In the images below, we can see that there is
emphasized by it types of writing that are a movement of two people talking with their faces
manifested in form a visual symbol which is in the very close. This indicates that the speaker is very
form of the symbol of the product, as follows: confident with his breath. The message to be
conveyed from the image's gesture is, if you apply
3.2 Visual Aspects Colgate Total, your confidence will increase and
The visual analysis of Colgate Total's make the other person comfortable because your
advertisements is carried out by applying a breath feels very fresh.
multimodal theory and specifically also analyzes
its generic structure. The three images below show
the activity in an office as the background. In the
three images, you can see the main actor
communicating with a very close mouth distance.
Speaking at such a close distance shows that the
speaker's breath is very fresh so that he feels very
confident with the other person when talking. The
message to be conveyed is that the ingredient in
Colgate Total is very strong to keep the mouth
3.5 Layout Aspects
fresh and its freshness for a long time.
The layout of the images is arranged in an
orderly to give the clear message of the Colgate
Total advertisement, from the office setting, two
people talking very closely, and to the ingredient
logo of Colgate Total as well.
3.3. Audio Aspects

The music accompanied the Colgate Total IV. CONCLUSION
commercial is an instrumental that is “easy
listening”, and is complemented by a “ring” A product that is advertised, if many people are
sound, which is the sound when the elevator able to remember the words, visuals, accompanying
music, and actors in the advertisement, it means
door opens. The impression that music wants to
that the message in the product has successfully
convey is a relaxed, fresh, and comfortable
reached the audience. It can be predicted that these
atmosphere. This is reinforced by the use of a
products will be able to survive in the market. This
written message which is the ingredient of
ad for Colgate Total toothpaste products displays
Colgate Total, which makes fresh breaths if
clear and interesting information about what people
applying Colgate Total. In addition, there is a
want from a product, namely its ingredients and the
very clear verbal message to support Colgate
benefits of using it. Messages in the form of
Total's freshness, e.g. “sensitivity, strengthens
language carried out through spoken language and
teeth, it kills germs through my whole now”
written language make the audience understand
3
more about what is being promoted. The short
duration but contains an overall message that is
important to convey makes this ad a special
impression for the audience. This advertisement
covers five aspects in a multimodal semiotic
system, namely linguistic, visual, audio, gestural,
and location aspects. These five aspects come
together to make the ad appearance very attractive
and easy to remember.
REFERENCES
Anstey, M., Bull, G. (2010). “Helping teachers to
explore multimodal texts”. Curriculum and
Leadership Journal. Vol. 8(16).
Machin, D., Mayr. A. (2012), How To Do
Critical Discourse Analysis in Multimodal,
Sage, Los Angeles
Halloran. (2004). Multirnodal Discourse
Analysis Systemic-Functional Perspectives,
Continuum, New York-London.
4
Register Login
CURRENT ARCHIVES ANNOUNCEMENTS ABOUT  SEARCH
HOME ARCHIVES VOL. 4 NO. 1 (2021) Articles Open Journal Systems
Multimodal Critical Discourse Analysis on advertisement of

Colgate
Ni Made Sila Ulati  PDF

Universitas Warmadewa
LANGUAGE
PUBLISHED
DOI: https://doi.org/10.22225/ijsfl.v4i1.4092
English
2021-11-04 Bahasa Indonesia
ABSTRACT
ISSUE
This study aims to analyze the advertisement of Colgate in audiovisual

Vol. 4 No. 1 (2021)
formed, when Colgate advertisement is one of the Super Bowl commercial
break in USA. This study applies on semantic approach focused on the
SECTION
multimodal discourse analysis. In the semiotic sense, signs are words,
images, sounds, gestures, and objects are also called multimodal, as it is Articles
said (Anstey & Geoff, 2010) a text is said to be a multimodal text if the text
has two or more semiotic systems. There are five types of semiotic systems,
e.g. linguistic, visual, audio, gestural, and layout. The method applied in Copyright (c) 2021 International
this study is descriptive qualitative method. The result of this study is Journal of Systemic Functional
Colgate Total advertisement covers five aspects in a multimodal semiotic Linguistics
system, namely linguistic, visual, audio, gestural, and location aspects.
These five aspects come together to make the ad appearance very
INFORMATION
attractive and easy to remember.
This work is licensed under a Creative
Commons Attribution-NonCommercial- For Readers
Keywords: multimodality, advertisement, analysis
ShareAlike 4.0 International License. For Authors
For Librarians
Authors who publish with this journal
agree to the following terms:
Authors retain copyright and

grant the journal right of first
publication with the work
simultaneously licensed under a
Creative Commons Attribution
License that allows others to
share the work with an
acknowledgement of the work's
authorship and initial
publication in this journal.
Authors are able to enter into
separate, additional contractual
arrangements for the non-
exclusive distribution of the
journal's published version of
the work (e.g., post it to an
institutional repository or
publish it in a book), with an
acknowledgement of its initial
publication in this journal.
Authors are permitted and
encouraged to post their work
online (e.g., in institutional
repositories or on their website)
prior to and during the
submission process, as it can
lead to productive exchanges, as
well as earlier and greater
citation of published work (See
The Effect of Open Access).
Abstract viewed = 15 times

PDF downloaded = 14 times
ISSN: 2392-3113
Rhetoric in Argentina
1/2015
SALVIO MARTÍN MENÉNDEZ

UNIVERSIDAD NACIONAL DE MAR DEL PLATA, UNIVERSIDAD DE BUENOS AIRES,
CONSEJO NACIONAL DE INVESTIGACIONES CIENTÍFICAS Y TÉCNICAS
SALVIOMENENDEZ@GMAIL.COM
A strategic approach to multimodal discourse analysis
Summary
In the paper the author argues that multimodal discourse analysis examines the traditional issues of rhetoric
from a pragmatic-discursive perspective and therefore may be regarded as a part of rhetoric. Starting from
systemic-functional linguistics, the author proposes a method for studying the multimodality of discourse
which highlights the notion of strategy, defined as a plan for achieving a particular purpose.
Key words
Rhetoric, multimodality, strategy, pragmatic-discursive perspective, systemic-functional linguistics
License
This work is licensed under a Creative Commons Attribution 3.0 Poland. The content of the license is
available at http://creativecommons.org/licenses/by/3.0/pl/
Res Rhetorica, ISSN 2392-3113, 1/2015, p. 43
SALVIO MARTÍN MENÉNDEZ

UNIVERSIDAD NACIONAL DE MAR DEL PLATA, UNIVERSIDAD DE BUENOS AIRES,
CONSEJO NACIONAL DE INVESTIGACIONES CIENTÍFICAS Y TÉCNICAS
SALVIOMENENDEZ@GMAIL.COM
A strategic approach to multimodal discourse analysis
1. Introduction
M.A.K. Halliday argues in “Ideas about language” (1997) that traditionally lan-
guage has been described in two different ways: as a resource and as a system of
rules. Viewed as a system of rules, language is a tool for cognitive organization
and relates to thinking. Viewed as a resource, however, language is a tool for com-
munication and relates to action and interaction. Action and interaction are what
link language to its defining feature: instrumentality. Language use is shaped by
how people interact with one another in a given socio-cultural context. This di-
scussion of action and interaction is not to downplay the importance of thinking,
but simply to place it in the context to human interaction.
Two traditions of language description, then, are clearly traceable: one rooted
in logic and the other in rhetoric. Those who look at language as a system of rules
think in terms of ‘logical’ grammar. On the other hand, those who treat language
as a set of resources, tend to think in terms of rhetoric. Therefore, from a logical
point of view, linguistics can be seen as part of logic, while from an anthropologi-
cal point of view, it can be seen as part of rhetoric.
The rhetorical tradition has been less successful in academic circles than the
‘logical’ grammar tradition exemplified in functional theories of language. Indeed,
logical approaches, such as structuralism and generative grammar, have almost
completely overshadowed functional approaches. Functional approaches are rhe-
torical because they orientate language in relation with how speakers interact in
specific situations within a particular culture. Speakers choose what they consider
to be the best resources available in order to achieve their goals, and combine them
in a way that they consider successful.
This paper examines the relationship between two aspects of interaction that
functional linguistics has always treated as mutually exclusive - the relationship
between what has traditionally been called the verbal and the non-verbal. Until re-
cently, an almost exclusive focus on grammar and discourse (with differing degrees
of emphasis on each) had meant that other components of discourse and situation
Salvio Martín Menéndez, A strategic approach to multimodal discourse analysis ● 43

were relegated to a secondary or complementary position in relation to the cen-
trality of verbal language. If sociolinguistics is taken as a general framework for
language description (Mesthrie 1994), pragmatics (Leech 1983, Levinson 1983),
rhetorical analysis (Albaladejo Mayordomo 2005, Anscombre and Ducrot 1983,
Carel and Ducrot, 2005, Eemeren, F.E. van 2009, Pujante 2003) and discourse
analysis (Brown and Yule 1983, Fairclough 1992, 1995, Lavandera 1985, 2014)
provide clear examples of this unequal relationship.
Systemic functional linguistics (Halliday 1994) can be placed within the rheto-
rical tradition because it understands language as a system of meaning resources
within social semiotics (Halliday 1979, Hodge and Kress 1998). A verbal language
is just one system within the general framework of systems that make up social
semiotics. This concept means that any given language is always part of a wider
system of choices that can represent meaning. Language is characterized by two
complementary axes of linguistic patterning, the paradigmatic and the syntagma-
tic. The syntagmatic refers to how words in a sentence are combined (e.g., ‘John
likes dogs’ rather than ‘Dogs like John’) while the paradigmatic refers to how
words are selected (e.g. ‘John likes dogs’ rather than ‘John likes cats’). The two
axes are interwoven but they can be focused on separately for the purposes of
analysis.
In fact, any actual discourse arguably has three dimensions (Menéndez 2005):
grammar (Halliday and Hasan 1976, Halliday and Matthiessen 2004), strategy
(Gumperz, 1982, Menéndez 2005, 2010a, 2010b, van Dijk 1982, van Dijk and
Kinstch 1982) and genre.(Bakhtin 1944, Martin and Rose 2008). These three
analytical dimensions should be seen from a multimodal perspective (Kress 2010),
because discourse is the place where different modes appear. So, a discourse is a
multimodal unit. A multimodal perspective allows us to analyze how relationships
between the resources of each mode are established and why these relationships
occur in the way they do.
Accordingly, a method of multimodal discourse analysis will be presented. The
method consists of analyzing discourse strategies (Menéndez 2012) to show how
the different resources of each mode appear in relation to one other.
2. The multimodal perspective

Multimodality describes communication practices in terms of the textual, aural,
linguistic, spatial, and visual resources - or ‘modes’ - used to compose messages.
Consequently, in discourse analysis, multimodality is considered to be a perspec-
tive (Jewitt 2009) rather than a theory. Based on systemic functional linguistics,
which understands verbal language as part of a social semiotic system (Halliday

1978, Hodge and Kress 1998.), it can be seen as a reformulation of the pragmatic-
-discursive perspective (Verschueren 1999, Menéndez 2005).
The central concept of systemic linguistics is that verbal language can be de-
scribed in terms of a set of choices of meaning, a set of options, such as singular/
plural, past, present, future tense, positive/negative polarity, etc., and the actual
choices made by speakers. Systemic linguistics can thus be seen as a forerunner
of the multimodal approach. This aspect is true because verbal language is a part
of social semiotics. Verbal language always appears in actual discourse in relation
to other modes. So, multimodality claims that the meaning of the discourse can
only be understood if the relation of the different resources can be demonstrated
(Menendez 2012). It is not only a matter of describing the previous meaning of
the options in the different semiotic systems. Meaning is the result of how these
different resources interact in discourse.
A discourse always presupposes an active subject who makes choices according
to his or her interactional needs in a specific socio-cultural context - whether or
not these choices are deliberate or their effects consciously intended. The subject
produces this discourse in a particular register (Halliday 1978) following the co-
nventions of use of a particular discursive genre (Bakhtin 1944). The multimodal
perspective adds to this approach the perception that choices are made not only at
the level of verbal language, but also simultaneously at other levels.
Within a social semiotics, different semiotic systems have different . grammar
Grammar can be described as a meaning potential and together the different se-
miotic systems make up a ‘paradigm’. A paradigm is always a reconstruction. It
represents the various options of each system and allows the discourse of a culture
to be formulated as a set of all the potential options. A particular discourse is the
realization of the paradigmatic options from the different systems as chosen by the
subject in a particular situation.
The different semiotic systems are paradigmatic disjunctions (X or Y). Like
the linguistic sign, discourse can be located on two axes: the axis of simultaneity
and that of succession. Indeed, from a multimodal perspective, discourse can be
characterized precisely as simultaneity in succession. Therefore, a mode never ap-
pears in isolation but always in relation to other modes. It is through the simulta-
neous interaction of the modes that a strategic combination appears. It is important
to note that there is that no way modes can be ranked in order of importance. The
importance of one mode over others depends on register and genre.
As mentioned earlier, one of the aims of multimodal discourse analysis is to
describe and explain the different resources combined through discourse strate-
gies (Menéndez 2000). A strategy is a plan with a particular goal. This paper aims
to show that that multimodality needs to focus on strategies precisely because

different resources are combined in each of the modes that constitute a particular
discourse. Discourse analysis is basically strategic since it sees discourse as a
combining of semiotic systems (understood as available options), register (under-
stood as options realized as discourse, i.e. resources) and genre (understood as a
convention of use). It should be noted, however, that discourse analysts normally
focus on the resources they consider most important for the task in hand, even if
other resources may also be present.
Register has been characterized as a variety of language in use determined by
situation (Halliday 1978, Ghadessy 1993). Traditionally, it has been characterized
by three variables: field, tenor and mode. However, there are two basic problems
with this characterization. The first is its static nature; the second is that it depends
directly on the functions of language used by systemic-functional linguistics to
describe grammar.
Discourse genres have been traditionally defined as conventions of language
use (Bakhtin 1944, Hasan 1994, 1995). These conventions shape discourses in
so far as they act as pre-conditions of possible interpretations. Genre is thus the
cultural context that enables a discourse to be interpreted in a certain way. In this
sense, genre analysis is based on and conditioned by grammatical description and
strategy analysis.
The question is how to explain the relationship between register and discourse
strategies. Register is realized by discourse strategies within the framework of a
genre. Any language variation shown within a given register in relation to diffe-
rent resources is interpreted within this framework, which acts as a set of guideli-
nes for interpretation (Menéndez 2010, 2012).
Just as grammar can describe any semiotic system, discourse can explain the
functioning of any strategy, and genre can frame an interpretation for any conven-
tion of use, so the choices a speaker makes can be interpreted according to the
genre that frames them. Discourse, then, is doubly conditioned. On the one hand,
it depends on the set of options speakers actually have (ideally all the possible
options they might have as members of a particular discourse community); on the
other hand, it depends on the genre conventions that shape the culture to which the
speakers belong.
A discourse genre can be considered as the intersection of a set of recurring
discursive strategies and a set of dominant cultural features. These features, of
course, are not explicit because they are part of the cultural and communicative
competence of the members of a society (Hymes 1972). Therefore, certain sets of
strategies are expected in certain genres; but this does not mean they have to be
displayed. These strategies establish the register and the different styles that can
be found within it. From this point of view, the strategic dimension of genre is

clear and can explain how discourse is interpreted in real life.

Cultural features are present in all discourses, but their degree of activation and
prioritization determine possible interpretations. For example, all discourses have
a political dimension from the point of view of genre, but this dimension is not
always useful for interpreting the discourse: it may be important, but it does not
have to be.
So, a discourse genre should be considered as a general framework of cultu-
ral and historical interpretation. It comprises both cultural features and a set of
recurrent discursive strategies. The relationship between these is what makes a
particular interpretation possible and any interpretation is always framed by genre.
Above all, it depends on combining semiotic resources in the form of plans - in
other words, discursive strategies.
3. An example
To show how a multimodal perspective can help to clarify discourse, let us
consider two situations through which a teacher of Spanish as a second language
might wish to present a grammar item. The teacher decides to use two interactio-
nal situations from different films, but we will use them in order to analyze the
interaction that takes place in them from a strategic point of view.. It is very im-
portant to point out that it does not matter which films are used; for our purposes
what matters is the situation from which the meaning of a dialogue can be inferred
and explained in relation to the different modes that can be analyzed.
Semiotic Semiotic Semiotic

SOCIAL System A System B System C
SEMIOTICS
Option 1 Option 1 Option 1
PARADIGMATICS Option 2 Option 2 Option 2
System of Option 3 Option 3 Option 3
available options ........ ........ ........
Option n Option n Option n
GENRE
Context of culture
MODE A MODE B MODE C

MULTIMODALITY Recourse 1 Recourse 1 Recourse 1
SINTAGMATICS
Set of realized
resources MULTIMODAL DISCOURSE
REGISTER
Context of situation
Figure 1: A general schema of multimodal discourse analysis

Each of the two situations shown in Figure 2 (below) simultaneously combines
a set of resources from different modes by means of a discourse strategy. The
photographs show clearly the multimodal nature of any discourse, because they
allow us to infer the register, i.e., the variation of language use according to the
situation.
Figure 2: Two film dialogues
The language teacher’s first task would be to analyze the situations and then
select suitable content for the Spanish as a second language class. However, the
background knowledge required for this analysis does not need to be taught. It
represents the minimum knowledge necessary to develop the teaching materials if
the materials are to be successful. It is here that the multimodal approach and the
strategic method are important.
The general theme of the Spanish lesson is “attitudes of the speaker speaker,” a
topic which traditional grammars call “modality” and which Systemic Functional
Linguistics explains by taking mode as the realization of interpersonal function
(Halliday and Mathiessen 2004). The language function or strategy to be taught
is “Asking for something something.” In informal situations this function is typi-
cally realized by the imperative, but this resource could also be interpreted as an
order or as an appeal depending on situational variables.
In order to describe the strategy involved in choosing a suitable exponent to
perform the language function, two elements need to be specified:
1) Cultural Assumptions (degree of knowledge)

2) Intervening Modes (strategic analysis)
A cultural assumption is an idea or knowledge that is common throughout a

community. Of course, not all the members of a community share exactly the

same cultural assumptions and the degree of conformity to in-group expectations

varies among individuals. But individuals must share some assumptions in order
to be a part of their community. Cultural assumptions are part of our cognitive
environments (Sperber and Wilson 1996).
Table 1 shows the different degrees to which cultural assumptions may be reco-
gnized in a particular context - in this case, the context of a film. The continuum
ranges from complete recognition to complete lack of recognition depending on
the student’s cultural background and familiarity with the film. Obviously, not all
students of Spanish as a second language will share the Anglo-Saxon assumptions
of the A Streetcar Named Desired or An Affair to Remember.
Cultural Assumptions Situation 1 Situation 2

+ known [degree 1] A Streetcar Named Desire An Affair to Remember
+/- known [degree 2] A film starring Marlon Brando A film starring Cary Grant
and/or Vivien Leigh and/or Deborah Kerr
- known [degree 3] A movie A movie
-/- known [degree 4] Not identified Not identified
Table 1: Degree of recognition of Cultural Assumptions
Table 2 shows the important intervening modes in relation to the goal of the
analysis (teaching Spanish as a second language).
Discourse Strategy: Asking for something

Resources (realized options)
Mode
Situation 1 Situation 2
1.Plane General American
2.Space House (private) Deck of a ship (semipublic)
3.Time 1950s 1950s
4.1.Gaze Woman at object Man’s and woman’s eyes meet
Man at woman
4.Gestuality
4.2.Arms Man towards object Man stretching out his arm with a note.
Woman towards object Female hand rests on his chest.
5.Verbal (possible dialogue ) Deme /Dame esto ya Por favor, tomá/tome esto
(Give that [now/right now] ) (Please, take this)
6.Dress Man : informal Formal
Woman: formal
7.Type of interaction Impolite (violent) Polite (tense)
Table 2: Intervening modes

The focus of interest here is the “attitudes of the speaker speaker.” These can be
identified in both films by comparing the way the characters combining resources
from different modes in order to further their different goals. In fact, the same
modes are salient in both interactions but they can be interpreted quite differently.
SOCIAL
SEMIOTICS
System System System
PARADIGMATICS
Gestuality Verbal Interactional
System of
relationship
available options
GENRE: Ordinary interaction
REGISTER
Gaze/Arms
MULTIMODAL Imperative
DISCOURSE Deme/Dame
Set of realized Impolite eso ya
resources (violent) (Give me that
now)
Figure 3: Multimodal analysis of a scene from A Streetcar Named Desire
SOCIAL
SEMIOTICS
System System System
PARADIGMATICS
Gestuality Verbal Interactional
System of
relationship
available options
GENRE: Ordinary interaction Por favor,

Tome/tomá esto
(Please,
take this)
REGISTER
Gaze/Arms
MULTIMODAL Imperative
DISCOURSE
Set of realized Polite
resources (tense)
Figure 4: Multimodal analysis of a scene from An Affair to Remember

For example, as mentioned earlier, the imperative could be used not only to give
orders. Its function will depend on how it is combined with other modes. In the two
examples shown above, this verbal resource is combined with gesture (direction
of the gaze and arms) and the degree of politeness in each situation. When the
situation is violent, the imperative is the expected resource for giving orders. When
there is tension (involving a certain degree of violence mitigated by other factors,
such as social class and education) the imperative may be used to make a request.
In the two examples above, the inferred dialogue is shown by additional discourse
markers: “now” (“ya”) in Situation 1; and “please” (“por favor”) in Situation 2.
4. Conclusion
This paper has shown how multimodal discourse analysis can be applied in
practice. The analysis focuses on strategies because these can be used to recon-
struct discursive planning in particular situations. So, the complexity of discourse
can be accounted for in terms of different combinations of resources from various
modes.
According to Verscheuren (1999), the understanding of meaning requires a
“pragmatic perspective” based on the megastructure of language (Verschueren
1999). Following my own formulation of this concept (Menéndez 1997) shown
in Figure 5, the present paper has focused mainly on the second feature (negotia-
bility) in order to demonstrate that a rhetorical perspective has to be understood
mainly as a strategic perspective.
VARIABILITY
option in each paradigms
of a semiotic system
MULTIMODAL NEGOTIABILITY
PRAGMATIC combination of resources
PERSPECTIVE as discursive strategies
ADAPTABILITY
Genre adecaution
of discursive strategies
Figure 5: The concept of “pragmatic perspective” as formulated by Menéndez (1997)

References
Albaladejo Mayordomo, T. (2005), “Retórica, comunicación, interdiscursividad.” Revista de

Investigación Lingüística 8: 7-33.
Anscombre, J. A., Ducrot, O. (1983), La argumentación en la lengua. Madrid: Gredos.
Bakhtin, M. (1944), “El problema de los géneros discursivos.” In Estética de la creación verbal.
México: Siglo XXI.
Brown, G., Yule, G. (1983), Discourse Analysis. Cambridge: CUP.
Carel, M., Ducrot, M. (2005), La semántica argumentativa. Una introducción a la teoría de los
bloques semánticos. Buenos Aires: Colihue.
Fairclough, N. (1992), Discourse and Social Change. Cambridge: Polity Press.
Fairclough, N. (1995), Critical Discourse Analysis. London: Longman.
Ghadessy, M. (1993), Register Analysis: Theory and Practice. London: Routledge and Kegan Paul.
Gumperz, J. (1982), Discourse Strategies. London: Cambridge University Press.
Halliday, MAK. (1978), Language as Social Semiotics. London: Arnold.
Halliday, MAK. (1994), “Systemic Theory.” In The Encyclopaedia of Language and Linguistics,
Volume 8, edited by R.E. Asher. Oxford: Pergamon Press.
Halliday, MAK. (1997), “Ideas on Language.” In On Language and Linguistics, Collected Papers
Volume I, edited by Jonathan Webster. London: Continuum.
Halliday, MAK., Hasan, R. (1976), Cohesion in English. London: Longman.
Halliday, MAK., Mathiessen, Ch. (2004), An Introduction to Functional Grammar. London: Arnold.
Hasan, R. (1994), “Situation and the Definition of Genres.” In What´s going on here?Complementary
Studies of Professional Talk, edited by A.D. Grimshaw. Norwood: Ablex.
Hasan, R. (1995), “The Conception of Context in Text.” In Discourse and Society: Systemic Functional
Perspectives, edited by P. Fries, M. Gregorey. Ablex: Norwood.
Hodge, R. and Kress, G. (1998), Social Semiotics. New York: Cornel University Press.
Hymes, D. (1972), On Communicative Competence. Philadelphia: University of Pennsylvania Press.
Jewitt, C. (ed.) (2009), The Routledge Handbook of Multimodal Analysis. London: Routledge.
Kress, G. (2010), Multimodality. A Social Semiotic Approach to Contemporary Communication.
London: Routledge.
Lavandera, B. (1985), Curso de lingüística para el análisis del discurso. Buenos Aires: CEAL.
Lavandera, B. (2014), Variación y significado. Y discurso. Buenos Aires: Paidós.
Leech, G. (1983), Principles of Pragmatics. London: Longman.
Levinson, S. (1983), Pragmatics. Cambridge: CUP.
Martin, J. and Rose, D. (2003), Working with Discourse. London: Continuum.
Menéndez, S M. (1997), Hacia una teoría del contexto discursivo. Ph.D. Dissertation. Facultad de
Filosofía y Letras, Universidad de Buenos Aires. Unpublished.
Menéndez, S M. (2000), “Estrategias discursivas: principio metodológico para el análisis pragmático
del discurso.” In: Lengua, discurso, texto (I Simposio Internacional de Análisis del discurso),edi-
ted by J.J. de Bustos and P. Chareadeau,1203-1219. Madrid: Visor.
Menéndez, S M. (2005a), Gramática, análisis del discurso e interpretación crítica: las relaciones no
tan evidentes. In Proceedings of the International Conference on Critical Discourse Analysis.
Universitat de Valencia, Valencia, CD.
Menéndez, S M. (2005b) ¿Qué es una estrategia discursiva?. In Actas del Congreso Internacional:
Debates Actuales. Las teorías críticas de la literatura y la lingüística, edited by S. Santos and J.
Panesi, Facultad de Filosofía y Letras, UBA, Buenos Aires. CD.
Menéndez, S M. (2010ª), “Opción, registro y contexto. El concepto de significado en la lingüística

sistémico-funcional.” Tópicos del seminario, 23: 221-239.

Menéndez, S M. (2010b), “Evaluación y estrategias discursivas. El caso de las proclamas militares en
la Argentina (1930-1976)”. Páginas de guarda 10: 28-41.
Menéndez, S M. (2012), “Multimodalidad y estrategias discursivas: un abordaje metodológico.”
Revista Aled 12(1): 57-74.
Pujante, D. (2003), Manual de retórica. Madrid: Castalia.
Sperber D., Wilson, D. (1986), Relevance: Communication and Cognition. Harvard: Harvard
University Press.
Verschueren, J. 1999. Understanding Pragmatics. London: Arnold.
van Dijk, T. (1984), Prejudice and Discourse. An Analysis of Ethnic Prejudice in Cognition and
Conversation. Amsterdam: John Benjamins.
van Dijk, T., Kinstch, W. (1982), Strategies for Discourse Comprehension. New York: Academic
Press.
van Eemeren, F.H. (ed) (2009), Examining Argumentation in Context. Fifteen Studies on strategic
maneuvering., Amsterdam; John Benjamins

DOI: https://doi.org/10.24297/jal.v10i0.8196
A Multimodal Discourse Analysis of the Interactive Meaning in Public Service Advertisement
Shuting Liu
School of Interpreting and Translation Studies, Guangdong University of Foreign Studies,
No.2 Baiyun Dadao Bei, Baiyun District, Guangzhou, China
liushuting0124@qq.com
Abstract
On the basis of Kress and van Leeuwen’s Visual Grammar based on Halliday’s Systemic Functional Linguistics,
this study explores the interactive meaning in three public service advertisement multimodal discourses, adding
evidence to the assumption that Systemic Functional Linguistics can be applied to the multimodal discourse
analysis of public service advertisement in a feasible and operational manner.
Keywords: Multimodal Discourse Analysis, Systemic Functional Linguistics, Visual Grammar, Interactive
Meaning, Public Service Advertisement
1 Introduction
Nowadays, advertisements are increasingly penetrating into people's daily life, playing an increasingly important
role. Whether through traditional media tools, such as newspapers, books, letters, or modern media tools, such
as television, computers, mobile phones or other network terminals, advertisements can be found anywhere at
any time, as if advertising has become one of the indispensable elements of human society. Generally speaking,
advertisement can be divided into two categories, namely, commercial advertisement and public service
advertisement (PSA). The criteria for classifying the two types of advertisements are also quite simple, that is,
whether they are profit-oriented or not. From research in recent years, it can be seen that experts and scholars
on advertising intend to set their research focus on commercial advertising, while research on PSA is
comparatively less. In addition, research perspectives need to be expanded in an all-round way, and research
connotations need to be further explored.
With information technology rapidly developing, visual language becomes more and more important in all
aspects, especially in advertising. The multimodal feature of advertising is an evidence of its use of various
semiotic resources, such as language, image, sound, and color, to better convey the connotation and obtain the
best advertising effect. Multimodal discourses exist widely and experts have carried out a lot of research on the
interpretation of multimodal discourses. Among these studies, multimodal discourse analysis (MDA) has taken
a place, taking Systemic Functional Linguistics (SFL) as its theoretical basis and providing new research ideas
and perspectives for discourse analysis. As a new research method of discourse analysis, MDA has become a
research focus of western scholars in recent years, especially in the fields of linguistics and social semiotics. Kress
and van Leeuwen [1, 2], Lemke [3], Royce [4], O 'Halloran [5, 6] and Baldry [7] all made positive and significant
contributions to MDA. Chinese linguistics Li [8], Hu [9] and Zhu [10] introduced the theory of MDA. Nowadays,
with the gradual development of MDA, more and more researchers are focusing on this field. However, there is
a lack of MDA focusing on PSA, which leaves a lot of gaps to be filled.
2 SFG, VG, MDA and PSA
In 1990s, Halliday’s SFL theory started to be applied in the study of MDA. Among the scholars who applied SFL
to MDA, Kress and van Leeuwen [1] put forward a theory that becomes prevailing in the field - they proposed
1523
Visual Grammar (VG) to analyze visual images. On the other hand, studies on discourse analysis starts from
1950s attracting wide attention from scholars and researchers.
2.1 Systemic Functional Grammar (SFG)
As mentioned previously, much research conducted in the field of MDA are based on the theory of social
semiotics, which in its narrow sense, refers to the Systemic Functional Grammar (SFG) of Halliday. The tenet of
SFG is to regard language as a kind of social semiotic and study language from the perspective of its interaction
with the social context. According to Eggins [11], SFG “has been applied to interpreting the grammar of other
semiotic modes, such as visuals [1, 2], sound [12] and action [13].
As Halliday [14] indicates, “In a functional grammar, a language is interpreted as a system of meanings,
accompanied by forms through which the meanings can be realized”. Therefore it can be said that SFG focuses
on the means of realizing the functions of language. According to Halliday’s framework, in language structure
and human communication there are three meta-functions, namely, ideational function, interpersonal function
and textual function.
1) Ideational Function: Language is used for describing, reflecting or interpreting the world around us, the
individual experience of ourselves, consisting of “goings-on, happening, doing, sensing, meaning, and being
and becoming” [14].
2) Interpersonal Function: Language is used for having communication or interaction with people, establishing
and maintaining relations with them, influencing other people’s behavior, expressing viewpoints of our own
on the things in the world, and eliciting or changing other people’s points of view. In this way, language is
used to enable people to participate in communications, take own their roles, and express their own and
understand others’ feelings, attitude and judgements.
3) Textual Function: It refers to the fact that language itself has a mechanism to turn a random list of sentence
into a living passage by organizing any stretch of written or spoken discourse into a coherent and unified
text [14]. Therefore, in textual function, language is used to relate what is said or written to the real world
or other linguistic events.
2.2 Visual Grammar (VG)
Inspired by Halliday’s ideas of social semiotics, Kress and van Leeuwen proposed and developed a method of
social semiotic analysis of visual communication and established a descriptive framework of multimodality. This
theory of Kress and van Leeuwen carried out for reading images is called Visual Grammar (VG). In light of
Halliday’s three meta-functions of language - ideational, interpersonal and textual meanings, they put forward
three meta-functions of images – representational, interactive and compositional meanings. The corresponding
relation between the three meta-functions in SFL and VG is shown in Table 1[15].
SFL VG
Ideational Representational
Interpersonal Interactive
Textual Compositional
Table 1 Three strands of meta-functions in SFL and VG
1) Representational Meaning: “Any semiotic system has to be able to represent, in a referential or pseudo-
referential sense, aspects of the experiential world outside its particular system of signs” [1]. That is to say,
1524
the semiotic system has to be able to represent objects and their relations in the world outside the
representational system. According to SFL, language “represents and constructs our perception of reality in
the form of ‘goings-on’ or processes of various types, which incorporate different categories of participants
(e.g. actors, goals, receivers, sensors, attributes)” [15].Adapting this model to the analysis of images, Kress
and van Leeuwen [1] divide representational meaning into two main categories of processes: narrative and
conceptual.
2) Compositional Meaning: “Any semiotic system has to have the capacity to form texts, complexes of signs
which cohere both internally and with the context in and for which they were produced” [1]. The visual and
verbal elements in multimodal discourses form a compositional whole. When expressing the textual meta-
function, compositional resources would appear to be: (1) horizontal structure when presenting visual
information as Given or New and (2) vertical structure when presenting visual information as Ideal and Real
[1].
3) Interactive Meaning: “Any semiotic system has to be able to project the relations between the producer of
a sign or complex sign, and the receiver/ reproducer of that sign” [1]. That is to say, any semiotic system has
to be able to project a particular social relation between the producer, the viewer and the object represented.
According to Kress & van Leeuwen [1], there are two kinds of participants involved in an image, namely,
represented participants (the people, the places and things depicted in images) and interactive participants (the
people who communicate with each other through images, namely the producer and the viewers). Between
these participants, there are three kinds of relations. First, relations between represented participants; Second,
relations between interactive and represented participants, which refers to the interactive participants’ attitudes
towards the represented participants; And third, relations between interactive participants, which refers to the
things interactive participants do to or for each other through images.
Interaction happens whenever the producers who produce the images and the viewers who appreciate the
images expressed their own ideas on the ways of images interpretation. Some of this interaction is direct and
immediate where the two parties have conversations face to face. While other interactions may be neither direct
nor immediate due to the absence of both sides. When examining an interaction, four key factors shall be taken
into careful consideration: visual contact, social distance between the represented participants and the viewers,
perspective and modality accounting for the degree of factuality and truthfulness of visual images with regard
to the real world [1].
2.3 Multimodal Discourse Analysis (MDA)
According to Xin [16], there are many well-known researchers studying MDA: O'Toole, Kress, van Leeuwen,
Lemke, O'Halloran, Thibault and Ventola who studies MDA from the perspective of social semiotics; Scolion,
Norris and Jewitt from the perspective of interactive sociology; Forceville and Holsanova from the perspective
of cognitive. Kress and van Leeuwen's [1] framework documents the system of meaning for the ideational,
interpersonal and textual which are respectively labeled as representational, interactive and compositional
in visual communication. Reading Images explores how figures, places and things in the image compose the
visual design to realize the compositional, interactive and representative meanings. As cited from Li (2003), many
multimodal discourse analysts made great contributions to MDA. For example, O'Toole [17] analyzes images
from the viewpoint of stratification; Lemke [3] explores the interrelations between the graphics and the verbiage
in scientific articles. Royce [4] explores the image-verbiage relations in advertisements. In analyzing three-
dimensional discourses, a few analysts have done tentative studies. For example, Martinec [13] makes a
multimodal study of body movements and the space design of the play.
In comparison with the prosperous studies on MDA overseas, China has just witnessed a budding stage in recent
years. Several Chinese scholars have realized the importance of multimodality. A series of introductory papers
to multimodality [8, 9, 10, 16, 18, 19, 20] and studies which apply the theories of MDA in specific types of
discourses [21, 22, 23, 24] have been done. Li [8] is the first to introduce the theory of MDA and reviews Kress
1525
and van Leeuwen's VG in her article. Afterwards Hu [9], Zhu [10], Xin [16] and Zhang [20] propose their opinions
on MDA, systemically introducing the theories of MDA and outlining the research status, the limitations and the
future of MDA. As for the application of the theory, plenty of research has been done. Hu and Dong [25] makes
a multimodal analysis of PPT; Ding [26] probes the modality in the illustrations in fairy tales. Several researchers
such as Li [27], Wang [23] and Hu [9] make a multimodal analysis of print advertisements and they mainly focus
on the interaction between the image and the verbiage.
2.4 Public Service Advertisement (PSA)
Public Service Advertisement is to diffuse beneficial social values to the public by using diverse artistic
demonstrational styles which can easily attract audience’s attention, affecting their views and behavior towards
the society and get a better solution to social problems. There are plentiful studies on advertising discourse in
social context in the past few decades. Leech [28] first studied advertising discourses from the view of stylistics.
Cook [29] put an emphasis of advertising discourse on the social implication of advertising language. One of
the Chinese frontiers, Huang [30], applies Systemic-Functional Grammar in advertising analysis and gives a
description of advertising from some aspects. However, they lack systemic analysis of inter-semiotic grammatical
connections in PSA. Therefore, this thesis chooses PSA as data to explore the interactive meaning by using MDA.
3 Methodology
The purpose of this study is to analyze how the multimodal discourses of PSA interact and exert the effects.
Thus, the present study attempts to provide an answer to the following questions:
Q1: How is the interactive meaning constructed in visual mode?
Q2: How is interactive meaning facilitated in verbal mode?
Q3: How are the verbal and visual modes combined to construct the interactive meaning?
In terms of data collection, three PSA examples are chosen for example analysis to have their multimodal
discourse analyzed in this study. The author mainly choose the examples for analysis from the Internet by
browsing through websites entitled “world’s creative public service advertisement” and alike. These examples
are respectively issued by World Wildlife Fund (WWF), goodparent.pl, and Cancer Patient Aid Association.
This study will analyze the visual and verbal messages in the PSAs respectively and then study the interaction
relations and effects of the verbal and visual modes. Data analysis is composed of three sections. The first section
is concerned with the interactive meaning in terms of the four elements of VG, namely, visual contact, social
distance, perspective and modality; the second section analyzes how the interactive meaning is constructed in
verbal language with respect to mood system and attitude system. The last part explores how the visual modes
and the verbal modes cooperate to construct the interactive meaning in PSA multimodal discourse.
4 Discussions
In this section, three PSA examples will be analyzed respectively in detail in order to inquire how various
modalities work together to realize the interactive meaning and to persuade the public in advertising discourses.
4.1 Analysis on Example 1 “Everyone Pays”
1526
Figure 1 “Everyone Pays”
(1) Visual contact
In terms of visual contact, this image can be counted as both a “demand image” and an “offer image” because
that some of the represented, the two big polar bear, look directly at the viewers, while others, the two small
polar bear and the penguin, do not look directly at the viewers. On one hand, “the participant’s gaze demands
something from the viewers, demands that the viewers enter into some kind of imaginary relation with him or
her” [1]. That is to say, the represented participant wants the viewers to take some action or to establish some
kind of social rapport with the viewers. In this case, the represented, polar bear and penguin standing on isolated
ice, want the viewers, human, to connect the light turning on with the severe conditions of their habitats and
then come up with solutions to ease the issue to protect these endangered animals. On the other hand, the
small polar bear and penguin do not look directly at the viewers but gaze somewhere else. According to Kress
and van Leeuwen [1], this kind of image “offers the represented participants to the viewers as items of
information, objects of contemplation, impersonally, as though they specimens in a display case”. In this case,
the represented, as objects of contemplation, provides information to the viewers. Viewers should make use of
this information, which is that the earth’s climate is undergoing severe changes, and do some thinking on taking
measures to save the animals.
(2) Social distance
The represented in this image could be divided into two parts, namely, the light and the picture with animals on
the wall. In terms of the light, a long shot is employed with the object shown out of the viewer’s reach with much
background information, which suggests a far social distance; while in terms of the animals, a very long shot is
employed with a total of eight torsos shown to the viewers, which entails a public distance. Both the long shot
and the very long shot indicate a rather far distance, which gives the viewers a feeling of objectivity. That is to
say, the viewers are more easily convinced of the truthfulness of the advertisement and thus earnestly take
actions to solve the objective problem.
1527
(3) Perspective
For the horizontal dimension, a frontal angel is adopted to depict both the picture and the light. With a frontal
angle, the image transmits involvement, indicating that climate issue and earth-protecting mission are world-
wide and in need of every one’s efforts and contributions. For the vertical angle, an eye-level angel is chosen to
transmit equality between the represented and the viewers, showing that the viewers are all in the same boat
when facing the issue shown in the image.
(3) Modality
The degree of modality of an image is assessed from the following aspects: color, contextualization,
representation, depth, illumination and brightness. This advertisement is truthful and reliable for the reason that
it has high modality. For contextualization and representation, much details of the background as well as the
represented participants are depicted, for instance, the wallpaper pattern on the wall, the small pieces of ice, the
frame of the picture and the posture of the penguin. For color, the image has fully saturated colors, several
colors and different shades of colors. In addition, there are also the play of light and shade as well as different
degrees of brightness, especially areas around the lighting lamp.
The verbal components identified in this PSA mainly include this one sentence on the upper side of the image:
(1)“When you leave the light on, you are not the only one who pays” with the words “light on” in bold. The
words (2)“panda.org/climate” on the right lower corner and the sign beside indicate the institution issuing this
advertisement which is “WWF (World Wildlife Fund)” and its website. The verbal text (1) is a declarative sentence
to give a moderate description on the consequences of not saving the energy, which is to make climate warming
worse and leave some animals in an endangered situation. What is particular about this sentence is that it
intends to convey the idea in an implicit but not explicit way, leaving spacious room for contemplation to the
readers. The latter part of the sentence “you are not the only one who pays” intrigues the readers to reflect on
the question “who else will pay” and may come up with answers themselves.
To begin with, in terms of visual contact, this image is a “demand image” and an “offer image”, which demands
people to take immediate actions and have deep contemplation by making direct visual contact and offering
information of the worsening climate change; the far social distance in this image delivers objectivity;
involvement and equality is conveyed through the perspective of the image; and this image is of high modality.
As for the verbal text of the image, a declarative sentence is used to lead the viewers into thinking about the
consequences of not being environmentally friendly. Therefore, the visual and the verbal text of the image
integrate to address the topic issue – environmental protection and wildlife protection and to urge the human
society to take actions to save the planet by doing everyone’s own bit.
4.2 Analysis on Example 2 “Broken Child”
1528
Figure 2 “Broken Child”
(1) Visual contact
In terms of visual contact, this image can be counted as an “offer image” because the represented, the child
crying heart-brokenly against the wall with his limbs of serious injury, do not make direct eye contact with the
viewers. Therefore, the image offers the represented participants to the viewers as an item of information or an
object of contemplation. The viewers shall consider the meaning this image intends to convey carefully to
explore the messages contained.
(2) Social distance
The represented in this image is depicted using a long shot for the reason that the viewers can see the whole
human figure with space around it. By employing a long shot, a far social distance is suggested. At far social
distance, the phenomenon in the image is presented in an objectively and impersonally manner. With the long
shot, the image presents the objects for the viewer’s contemplation, leading the viewers to think about the
possible reasons behind such phenomenon as well as to find out effective and practical solutions to such issues.
(3) Perspective
For the horizontal dimension, an oblique angel is adopted to depict the child crying his heart out in the room.
The oblique angle detaches the viewers from the represented, correspondent with the detachment conveyed by
the far social distance between the represented participant and the viewers. For the vertical dimension, a low
angle is adopted since the viewers are requested to look up to the represented participant. The low angle is
suggestive of the importance of the portrayed participant and the significance of having the problem solved.
(4) Modality
This advertisement has high modality in terms of contextualization and representation, much details of the
background as well as the represented participants are depicted, such as the door and the bolt and the lock on
1529
the door, the connector plugs on the wall, the doorsteps, the tiles on the floor, the broken pieces of the child,
the facial expression of the child and so on. For color, the image has several colors and different shades of colors.
In addition, there are also the play of light and shade such as the changing of light on the doorstep.
The verbal components identified in this PSA mainly include this one sentence on the upper right side of the
image: (1)“YOU CAN LOSE MORE THAN YOUR PATIENCE” with all the letters capitalized. On the lower right
corner of the image is the sign of the institution who issued this advertisement and its website
(2)“www.goodparent.pl”. The letters in verbal text (1) are all in uppercase, which shows the producer’s emphasis
on the issue. This sentence brings the viewers to contemplate on the question “what else will I lose if I lose my
patience” and they may come up with unexpected answers themselves. From the name of the website, readers
may assume that this advertisement is to teach parents how to be good parents. And in this case, this
advertisement is to teach parents to be patient to their children.
To begin with, in terms of visual contact, this image is an “offer image”, which offers the viewers information for
contemplation; the far social distance in this image delivers objectivity; an oblique and low angle is adopted in
terms of perspective; and this image is of high modality. As for the verbal text of the image, a declarative
sentence is used to lead the viewers into thinking about the consequences of not being patient to the children.
Therefore, the visual and the verbal text of the image integrate to address the topic issue – parenting and child
education and to urge parents to have contemplation on their loss-of-patience issue to their children.
4.3 Analysis on Example 3 “Cigarettes Smoke People”
Figure 3 “Cigarettes Smoke People”
1530
(1) Visual contact
In this image, the represented object do not have their eyes depicted, not to mention whether they are looking
directly at the viewers. In this case, the viewers may take it as both a “demand image” and an “offer image”, or
consider it as neither a “demand image” nor an “offer image”, which depends on the viewers’ imagination and
association. As far as the author is concerned, the represented in the first image, the persona sitting on the bed,
may make direct eye contact with the viewers; while the two persons sitting face to face at a dinner may not
look directly at the viewers because it seems like they are having a conversation so they may just look at each
other. In this way, both the “demand” and the “offer” images are shown to the viewers, leaving the viewers to
intrigue their imagination and take relevant practices to the issue addressed.
(2) Social distance
In this image, there are two parts when referring to the represented. The first part of the represented, the persons
with cigarettes in their fingers, is depicted in the medium long shot since the full figure is shown, entailing a
close social distance with the viewers. The second part of the represented, the cigarettes being hold in persons’
hands, is depicted in the long shot since the object is shown out of the viewer’s reach with much background
information. The object is there for the viewer’s contemplation. In this case, an invisible barrier exists between
the viewer and the object possibly due to the extreme harm of the object – cigarettes can do harm to people’s
health as the image shows.
(3) Perspective
For the horizontal dimension, an oblique angel is adopted to depict the represented, both the people holding
cigarettes and the cigarettes. The oblique angel suggests that there is detachment rather than involvement
between the represented and the viewers. For the vertical dimension, a high angle is chosen for the viewers may
need to look down on the represented participants, which suggests the inferiority of the represented participants
in terms of health conditions for the reason that cigarettes are doing harm to their physical quality.
(4) Modality
This image is highly truthful for it has moderately high modality. To begin with, this image has high modality in
terms of contextualization and representation. A great deal of details of the background and the presented
participants are depicted, for example, the jewelries on the women’s hands, the smoke of the cigarettes, the
lighting lamps on the both sides of the bed, the ashtray and phone on the night table, the coffee cups and sugar
containers on the table, the lipstick on the brink of the cup, etc.
Different from the previous two examples, this PSA consists of two images, one above and the other below. The
verbal components of these two images is the same, which is on the lower right corner and upper right corner
of the images respectively. The verbal message contained is this image include (1)“CIGARETTES SMOKE PEOPLE”
with all the letters in uppercase and the sign and name of the institution who issued this advertisement which
is (2)“Cancer Patient Aid Association”. The verbal text (1) is a simple declarative sentence but with a logic quite
unmatched with the rules of English grammar or daily conversation because in normal cases, this sentence
should be “people smoke cigarettes”, but not the other way round. With the subject and object switched around,
the viewers may have a deeper impression on the image and be more easily influenced by this advertisement.
To begin with, in terms of visual contact, it is not sure whether this image is a “demand image” or an “offer
image” because the presented participants’ faces and bodies are “smoked up” by cigarettes; the close and far
1531
social distance between the two represented participants and the viewers embody different meanings; an
oblique and low angle is adopted in terms of perspective; and this image is of high modality. As for the verbal
text of the image, a declarative sentence is used to impress the viewers with unfamiliar logical relations.
Therefore, the visual and the verbal text of the image integrate to address the topic issue – smoking as bad
habits and to urge smokers to rethink their smoking habits to improve their health conditions.
5 Conclusions
Based on the previous analysis on three PSA examples in the field of wildlife protection, parenting, and health
respectively, there are some major findings. Firstly, although there is some distinction in image representation,
for example, in terms of visual contact, social distance, perspective and modality, the four aspects work together
rather than separately to construct the interactive meaning. Their corresponding to each other is evidence to
their cooperation. Secondly, the verbal text in the images tend to leave some space for contemplation to the
readers. Through the way the producers organize the verbal text, the viewers may start asking themselves
questions on the issue proposed, and may come up with their own new ideas on the current issues, but not
reading somebody else’s solutions to the problems without self-reflection. Thirdly, these images are generally
of high modality to make them truthful and reliable. In terms of representation and contextualization, these
images are all of high modality, with detailed environment description and exquisite background setting. Images
of high modality turn out to be more truthful and can leave deeper impression on the readers, thus increasing
the possibility of influencing their behaviors.
Needless to say, this study, like any other studies, has its limitations, which further research may refer to and
make some improvements. Firstly, the example analysis part may feature subjectivity due to the author’s
personal ideas or the reason that some visual images are difficult to define and analyze in a precise manner.
This could be a disadvantage of the study. For further research, the research methods for this kind of MDA may
be improved and more other analytical methods shall be developed or used for MDA. Secondly, due to
limitations on time and space, the data collected in the thesis may not suffice for the findings. For further
research, more PSA examples shall be collected and analyzed for a more comprehensive study in the future.
Thirdly, this study explore the static images only, while in this era of information technology, dynamic
multimodal discourses such as videos, films, and television advertisements are prevailing. Therefore, for future
research, dynamic multimodal discourse analysis shall be taken into consideration, studying how sound, action
and other semiotic resources work together to construct interactive meaning. Lastly, due to the author’s limited
theoretical ability and rare literature, the findings in this study may need to be testified with the advances of
academic research in the future.
The authors hereby state that there is no conflicts of interest in terms of the writing and publishing of this paper.
References
1. Kress, G. and van Leeuwen, T. Reading Images: The grammar of visual design[M]. London: Routledge. 1996.
2. Kress, G. and van Leeuwen, T. Multimodal Discourse: The modes and media of contemporary
communication[M]. London: Arnold. 2001.
3. Lemke, J. Multiplying meaning: visual and verbal semiotics in scientific text[A]. In Martin, J. & Veal, R. (eds.),
Reading Science: Critical and Functional Perspectives on Discourses of Science[C]. London: Routledge,
1998: 87-113.
4. Royce, T. Synergy on page: exploring inter-semiotic complementarity in page-based multimodal text[A].

In JASFL Occasional Papers (No.1) [C]. Tokyo: Japan Association of Systemic Functional Linguistics. 1998.
1532
5. O 'Halloran, K. Independence, interaction and metaphor in multimodal texts[J]. Social semiotics. 1999, 9(3):
317-354.
6. O 'Halloran, K. Towards a systemic-functional analysis of multi-semiotic mathematics texts[J]. Social

Semiotic. 1999, 124(1/2):1-29
7. Baldry, A. Phase and transition type and instance: patterns in media texts as seen through a multimodal
concordance[A]. In O’Halloran, K. (ed.), Multimodal Discourse Analysis: Systemic-Functional
Perspectives[C]. London: Continuum. 2004:83-108.
8. Li Zhanzi. Social Semiotic Analysis of Multimodal Discourse[J]. Foreign Languages Research, 2003,5:1-8.
9. Hu Zhuanglin. Multi-patterning in the study of social semiotics[J]. Language Teaching and Research,
2007,1:1-10.
10. Zhu Yongsheng. Theoretical Basis and Research Methods of Multimodal Discourse Analysis[J]. Foreign
Languages Journal, 2007, 5:82-86.
11. Eggins, S. An introduction to systemic functional linguistics[M]. London: Continuum. 2004.
12. van Leeuwen, T. Speech, Music, Sound[M]. New York: St Martin’s Press. 1999.
13. Martinec, R. Cohesion in action[J]. Semiotic, 1998:120(1/2), 161-180.
14. Halliday, M. An introduction to functional grammar[M]. London: Edward Arnold. 1994.
15. Cui Yan. Multimodal Discourse Analysis of Interactive Meaning of Flat Public Service Advertisements[D].
Master Thesis of Shandong University. 2010.
16. Xin Zhiying. New Development of Discourse Analysis and Multimodal Discourse Analysis[J]. Journal of
Social Sciences, 2008,5:208-211.
17. O'Toole, M. The language of displayed art[M]. London: Leicester UP. 1994.
18. Li Miaoqing. Analysis of Multimodal Discourse Analysis of Adapted Films - Take "Raise the Red Lantern"
as an Example [J]. Film Literature, 2007,8:98-100.
19. Gu Yuguo. Analysis of Multimedia and Multimodal Learning [J]. Technology Enhanced Foreign Language
Education, 2007, 2:3-12.
20. Zhang Delu. Exploring the Comprehensive Theoretical Framework of Multimodal Discourse Analysis[J].
Foreign Languages in China, 2009,1:24-30.
21. Ye Qichang. Social Semiotic Analysis of Hypertext Multilingualism[J]. Foreign Language Teaching and
Research, 2006,6:437-442.
22. Wang Hongyang. Interpretation of Multimodal Function in Cummings's Poetry[J]. Foreign Language
Teaching, 2007,5:22-25.
23. Wang Hongyang. Research on the Interactive Meaning of Multimodal Advertising Discourse[J]. Journal of
Sichuan International Studies University, 2007,6:31-34.
24. Zheng Haicui, Zhang Maizeng. Multimodal Discourse Construction of Harmony Concept [J]. Foreign
Languages Journal, 2008,2:107-112.
1533
25. Hu Zhuanglin, Dong Jia. Multimodal Construction of Meaning-Discourse Analysis of One-on-One

Demonstration Competition[J]. Technology Enhanced Foreign Language Education, 2006, 3:3-12.
26. Ding Jianxin. Research on the Social Semiotics of Modality in Visual Grammatical Fairy Tale Illustrations[J].
Journal of Sun Yat-sen University, 2007, 5:39-42.
27. Li Songqing. Social Semiotics and Advertising Discourse Analysis[J]. Journal of Anhui University of Science
and Technology, 2005,2:56-61.
28. Leech, G. English in advertising[M]. London: Longman. 1966.
29. Cook, G. The discourse of advertising[M]. London: Routledge. 1992.
30. Huang Guowen. Theory and Practice of Discourse Analysis - A Study of Advertising Discourse[M]. Shanghai:
Shanghai Foreign Language Education Press. 2001.
1534
A Multimodal Discourse Analysis of Two Logos about Lanzhou China
Junbang He and Shunliang Shi
Science Insights 2014; 6(1):100-104

Science Insights is published by The Bonoi Academy of Science & Education, Winston-Salem, NC
27157, USA
Copyright © 2014 The Bonoi Academy of Science & Education. All rights reserved.
ISSN: 2329-5856
The online version of this article, along with updated information and services, is located on the
World Wide Web at:
www.bonoi.org
Permissions: Requests for permissions to reproduce figures, tables, or portions of articles originally published
in Science Insights can be obtained via our Permission Application System, a service of the Copyright
Clearance Center. If you cannot access to this system, you can request permission through toour Editorial
Office. Once the online version of the published article for which permission is being requested is located,
Request Permissions in the middle column of the Web page under Services. Further information about this
process is available in the Permissions and Rights Question and Answer document.
Submission: Information about submission to Science Insights is online at:

http://www.bonoi.org/node/74
LANGUAGE SHORT COMMUNICATION
A Multimodal Discourse Analysis of Two Logos

about Lanzhou China [PC]
Junbang He,*, Δ Shunliang Shi *
BACKGROUND Logos are typical multimodal discourses *: School of Foreign Languages, Lanzhou
University of Technology, Lanzhou, China
combining languages, pictures, colors and so on with spa-
Δ: Correspondence to: Dr. Junbang He, No.
tial distribution and often used in propagandizing a city. 287, Langongping Street, Qi Li He District,
Lanzhou, the provincial capital city of Gansu Province in School of Foreign Languages, Lanzhou
University of Technology, Lanzhou
north-west China, released two logos about the image and 730050, China. Tel: +86-139-1989-6045;
the tourism successively to propagandize the city. Email: hejb@lut.cn
OBJECTIVE The purpose of the study was to analyze the Submitted: 26 November 2013
logos from the perspective of linguistics and to identify Revised: 14 December 2013
Accepted: 14 December 2013
how images and words in the logos work together to fulfill
[PC]: Publisher Copyright Paper
the aim of propagandizing the city, what the respective
advantages and disadvantages of the two logos are, and
how to achieve a better propaganda aim of the city logos.
METHOD Based on the Kress and Van Leeuwen’s visual
grammar, this paper attempts to adopt a multimodal anal-
ysis and descriptive-analytical method to two logos about
Lanzhou China and make a comparison between them,
analyzing the representational meaning, the interactive
meaning and the compositional meaning of the two logos.
CONCLUSION We found that images and words can work
BASE
together to create a synergic effect of propagandizing the
city, and multimodality is an effective way to present such
information about the characteristics of a city in meaning-
making process of logos. Through the interaction of these
different semiotic elements, the logos express much more
BASE
abundant information than separate ones.■ SCIENCE INSIGHTS 2014; 6(1):100-104.
How to Cite This Paper: He J, Shi S. A multimodal discourse analysis of two logos about Lanzhou China. Sci- BASE
ence Insights 2014;6(1):100-104.
Keywords: Multimodal discourse analysis –Logos –Lanzhou

BASE
A
CCORDING to the World crucial in tourism to deliver or present most important industries and has
Tourism Organization, China the information about tourism re- been increasingly developing in Lan-
BASE
will become one of the largest sources and unique cultures in China zhou China, which attracts not only
tourist destinations all over the world to domestic and foreign tourists and domestic tourists but a great number
in the next decade. Therefore, it is investors. Tourism is also one of the of foreign friends. Along with the de-
12 JANUARY 2014 SCIENCE INSIGHTS VOL. 6, NO. 1 www.bonoi.org 100

He et al. MDA of Logos about Lanzhou
velopment and the increasingly im-

portant status of tourism in the overall
development, propagandizing the city
in an effective way is becoming more
and more important. Therefore, the
local government released one vivid
logo about the tourism of Lanzhou
and another one about the general im-
age of Lanzhou successively. The
logo of a city is one of the most effec-
tive and common forms with very
vivid and rich information which
combines languages, number, pictures
and so on with spatial distribution and
is “the meaningful text realized by
symbols by any more than one semi-
otic code”(1), that is , multimodal dis-
course. Figure 1: The two logos used in this MDA modality. (A) The logo about
Two books mark the birth of Mul- Lanzhou tourism was released in 2012. Lanzhou is one of the most im-
timodal Discourse Analysis (MDA): portant cities on the ancient silk road and the unique provincial capital city
the Language of Displayed Art (7) and in China through which the Yellow River is running, and “Dunhuang
Reading Images (4). Kress and Van Feitian” is one of the major symbols of Gansu province even the north-
Leeuwen propose a comprehensive west China. The images of the Yellow River in a zigzag form and the two
and systematic theoretical framework flying “Dunhuang Feitian” in the logo form an ensemble image of the Chi-
of visual grammar, providing theoret- nese character “兰” (Lan). On the images of the Yellow River, there is
ical foundation and methodology to some typical design of Ma Jia Yao culture representing the ancient boom-
understand and analyze multimodal ing culture of Yellow River watershed, and the color of the background rep-
discourses. Since Li (1) introduced resents the loess plateau of the North-west China. The lingual parts in-
Kress and Van Leeuwen’s visual clude two points: Lanzhou Tour; Lanzhou as the Center of Tour in North-
west China. Available from: http://www.linbang0769.com/60/n-256.html.
grammar to China, many articles have
Last accessed date: December 14, 2013. (B) The logo about the image of
appeared in periodicals and magazines
Lanzhou city was released in 2013. The Yellow River Mother Sculpture is
around China. Hu (3) and Zhu (1) one of the most typical and representative symbols of Lanzhou city, and its
have made comprehensive explana- image is the center of the logo; the design of Ma Jia Yao culture gives
tion to the theoretical foundation of prominence to the characteristics of the culture of Yellow River watershed;
MDA and its research methods as the three stripes (in blue, orange and green respectively) symbolize the
well as its application in social prac- blue sky, the Yellow River and the loess plateau, and the green mountains
tices. In addition, a number of schol- of Lanzhou region respectively. The lingual parts means Lanzhou China is
ars apply MDA to diverse fields such the city of the Yellow River. Available from:
as textbooks (2), advertisement (9), http://www.gs.xinhuanet.com/news/2013-05/24/c_115888461.htm. Last
academic speech presentation (11), accessed date: December 14, 2013.
film or film poster (12), picture (10),
news (13) and many other aspects of 2. Interactive meaning is about the
Multimodal Discourse Analysis
MDA. But there are only a few logo relationships among the three compo-
researches which adopt the perspec- The aim of MDA is to take represen- nents of an image: the designer, the
tive of visual grammar of multimodal tational, interactive and compositional participant, and the viewer. It can be
discourse analysis, and most of the meanings labeled in the various com- realized by four major factors: contact,
researches mainly involve logo of municative modes into consideration distance, attitude and modality, which
web page (7) or enterprise logo (13). to analyze how they work together to can work together to create complex
This paper attempts to adopt Kress create a complete discourse or com- and subtle relations between the
BASE
and van Leeuwen’s methodology to municative event. viewer and the participant.
briefly analyze two logos from Lan- 1. Representation can either be 3. Composition relates the repre-
zhou, China and discuss how different narrative, presenting unfolding actions sentational and interactive meanings
modes work together to create a com- and events, processes of change, tran- of the image to each other through
prehensive meaning to fulfill the sitory spatial arrangements, or con- three interrelated systems: infor-
BASE
propaganda aim. ceptual, representing participants in mation value, salience, and framing.
terms of their class, structure or mean- (4)
METHODS AND MATERIALS ing.

The MDA of visual grammar pro- River with the typical design of Ma When the participant looks directly at
vides theoretical foundation and study Jia Yao culture representing the an- the viewer, or there are eye contacts
method for the present modality anal- cient booming culture of Yellow Riv- between them, it is “demand” image;
ysis, in which a descriptive-analytical er watershed, the flying Feitian with when there is no eye contact, “offer”
method is adopted. fluttering ribbons and the bright yel- image (4).
low background representing the loess In both logos, the participant is
DISCUSSION plateau. All these elements make up not a living human but quasi-human,
the main portion of the whole logo there is no eye contact between the
To conduct propaganda of Lanzhou and interconnect with each other in a participant and potential viewer, and it
tourism and the city, the local admin- classificatory, analytical and symbolic is taken as an imaginary “contact”,
istration adopts various approaches way, making the logo full of dynam- that is, personal relationship between
including the above vivid logos, ics, hope, enthusiasm and delight and the participant and the viewer does
which present the typical information highlighting the exact topic and mean- not exist. These images are “offer”,
in a general way to help viewers in- ing of the logo. Therefore, the logo is only expressing the meaning of in-
stantly know the general characteris- very typical and appropriate to present formation that provides potential
tics, charm and representatives of the the information of Lanzhou city and viewer with very general and brief but
city. tourism in northwest China. representative characteristics, symbols
The second logo is conceptual and other information about Lanzhou.
Representational Meaning without vectors. The main images of The verbal mode also “offers” the
concept include the Yellow River, similar information.
In the representational analysis, Kress
blue sky, green land, the words in
and van Leeuwen draw into the con- Distance
both languages and the Yellow River
cept of “vector” to distinguish narra-
Mother Sculpture. There also exists an There are six types of distance be-
tive and conceptual representations. In
the narrative pattern, participants are ensemble Chinese character “ 兰 ”. tween the participant and the viewer:
Three aspects of the conceptual mean- intimate distance, close personal dis-
connected by a vector or vectors and
represented as doing something to or ing are involved — classificatory: all tance, long personal distance, close
the above elements are subordinate to social distance, long social distance
for each other, while the conceptual
pattern represents participants in the concept “Lanzhou City” or “Lan- (the whole person shown, and sur-
terms of their generalized, more or zhou Tourism”; analytical: each of the rounded by space) and public social
elements is the possessive attribute, distance. The different types are the
less stable and timeless essence (4).
and the ensemble image “ 兰 ”, the standards for judging the distance be-
In the first logo, the typical imag-
character “兰州” and the words “Lan- tween the participant and the viewer
es include the Yellow River in a zig-
zhou” are the carriers; symbolical: the in an image (4).
zag form and two “Dunhuang Feitian”
Yellow River Mother Sculpture is the In the logos, the whole bodies of
constituting an ensemble image of the
center of the logo and one of the most Feitian and the Yellow River Mother
Chinese character “兰”. One Feitian’s Sculpture are shown and surrounded
arm and the Yellow River intercross typical and representative symbols of
by space, reflecting the long social
to form the most obvious vector, and the city, the design of Ma Jia Yao cul-
ture gives prominence to the charac- distance between the participant and
the ribbons of Feitian also form vec- the viewer, which indicates that the
tors. The vectors indicate a narrative teristics of the culture of Yellow River
watershed, the blue sky and green viewer need to learn about this city
representation, which presents unfold- better and more thoroughly. It is the
ing actions or events about the city land show that excellent environment
is always the beautiful dream that lo- case especially for nonlocals, for
tourism and transitory spatial ar- many of them virtually have some
rangements. It could be understood as cal people have been trying to fulfill
with unremitting efforts, and the three misunderstandings and wrong ideas
an invitation to this wonderful city about the city. Attracted by the logos
through which the Yellow River is stripes (in blue, orange and green re-
spectively) symbolize the blue sky, and the transmitted information, peo-
running and as an implication that ple would like to visit Lanzhou and
there is something amazing and the Yellow River and the loess plateau,
and the green mountains of Lanzhou northwest China, which is the propa-
unique in this region of the loess plat- ganda aim of the logos. Even some
eau. At the same time, the ensemble respectively. So this logo also shows
BASE
the landforms and the characteristics local people do not know and under-
image “ 兰 ”, the Chinese characters stand the city, its culture and its char-
“兰州”, the seal with the characters of the city and fulfills the goal of
propagandizing effectively. acteristics well, so they could be also
“旅游” and the English words “Lan- attracted by the logos and would like
zhou” and “tour” distinguish the logo Interactive Meaning to learn more about their city.
BASE
as a conceptual representation as well. Attitude

In addition, there are other conceptual Contact
elements such as the flowing Yellow

Attitude is used to analyze the repre- through the city is very close and the bigger Feitian above and the
sentation of a subjective image, and common to people. In the second logo, smaller Feitian in the distance is per-
perspective is the way in which imag- the sculpture is at eye level and the spective and has comparatively high
es bring about relationships between viewer has a straight angle, which modality, while the second logo is not
represented participant and the viewer. suggests a relatively equal relation- perspective. As for illumination, there
Horizontal perspective includes ship between them. When visiting the is very low scale of shadows and light
frontal angle which makes the viewer city and appreciating the Yellow Riv- in the first logo (lowly illuminated),
feel that he or she is experiencing the er Mother Sculpture, which is one of and there is not any in the second one
image personally or feels what the the landmarks of the city and connotes (abstracted from illumination). Finally,
participant feels and oblique angle that the Yellow River has nourished brightness values can also contrast to
which conveys a message that the generations of Chinese people, visi- a greater or lesser degree: in the first
viewer as a bystander holds an indif- tors would like to look at it closely logo, there is a range of yellow hue
ferent and unconcerned attitude. On and take photos with it, and this from dark brown to yellowish even
the other hand, vertical perspective shows equal relationship between the pale yellow, which is of high scale of
involves a view angle of looking participant and the viewer. brightness; in the second logo, there is
down, which represents a forceful no scale of brightness. Above all, the
Modality
position the viewer stands, a view an- logos have medium sensory modality
gle of looking up, which shows a There are eight markers of visual mo- generally, but the modality of the first
forceful position of the participant, dality, color saturation, color differen- one is much higher than the second
and a straight angle which suggests a tiation, color modulation, contextual- one.
relatively equal relationship between ization, representation, depth, illumi-
the participant and the viewer (4). nation and brightness. Each of these Compositional Meaning Analy-
From horizontal perspective, both dimensions can be seen as a scale of sis
of the logos have a frontal angle high, medium or low sensory modali- Information Value
which shows involvement, that is, the ty (4).
viewer is “involved” with the repre- For the first logo, the hue of image Information value refers to the place-
sented participants, what the viewer and the background from brown to ment of elements that endows them
sees in the logos is part of the world yellowish is lowly saturated, the black with the specific informational values
we are involved in, the viewer could words and characters are of lowest attached to the various “zones” of the
feel that he or she is experiencing the saturation, and the red seal is of high image: left and right, top and bottom
image personally or feels what the saturation; the color differentiation is (the ideal and the real), center and
participant feels, and the viewer could not very obvious, but the degree of margin (nucleus and subservience) (4).
enjoy and appreciate the beautiful color modulation is high. For the se- In the first logo, Feitian is placed
scenery and culture such as the Yel- cond logo, the bright colors of blue, on the top as the ideal, and the image
low River running through the city, yellow and green make it highly satu- of Yellow River is placed at the bot-
the characteristics of the landforms, rated, while the white image of the tom as the real. The character “Lan-
and rich culture represented in Feitian Yellow River Mother Sculpture, the zhou Tour” and the ensemble image
and so on when visiting Lanzhou or black words and the characters are of “兰” are presented in the center as
being in the region. absent of color; the color differentia- the nucleus of the information; other
From vertical perspective, two tion is very obvious with high degree elements are in some sense subservi-
logos show different interactive mean- of color modulation. On the other ent. In the second logo, the Sculpture
ings. In the first logo, the viewer has a hand, the logos have medium contex- and the characters “ 中国兰州 ” are
view angle of looking up at the repre- tualization scale with an articulated presented in the center as the nucleus
sented participants Feitian, which but not very detailed background of of the propaganda, while other ele-
shows a forceful position of the par- the Yellow River, the yellow land and ments are presented as margin or
ticipant over the viewer. Feitian, fly- the sky in bright light. The detail of background. In both logos, there are
ing in the sky and serving or praising representation is reduced and ab- similar valuation between the image
the Buddha and other gods, do not stracted to much lesser and coarse and the verbal mode to present the
belong to earthly world and have a degree, and the texture is also omitted similar valuation.
BASE
forceful position in the aspect of art, altogether - the Yellow River, Feitian
Salience
culture or religion. When visitors go and the Sculpture are then represented
to Dunhuang, they would also look up merely by the lines that trace the con- Salience is realized by such factors as
at the Feitian in the murals on the wall tour with plain flat mono-color, and placement in the foreground or back-
or vault. In addition, the Yellow River other elements are presented just by ground, relative size, contrasts in to-
BASE
is at eye level, which is a straight an- stripes in different colors in the se- nal value (or color), differences in
gle and suggests a relatively equal cond logo. As for depth, the first logo, sharpness, etc. to attract the viewer’s
relationship, that is, the river running with the zigzag Yellow River below, attention to different degrees (4).

In the first logo, the image of “Feitian”, the Yellow River, and the MDA: Multimodal Discourse Analysis
Feitian, the river, the ensemble image word “Lanzhou” makes up the major Conflict of Interests
“兰”, the big character “兰州” and the framing of elements. In the second
logo, three stripes in different colors None
red seal “旅游” are most salient with
big size, sharpness of focus and obvi- form lines, broken lines or shared References
ous color contrasts. Both of the two lines to show not only the differences 1. Li Z. Social semiotic approach to mul-
of the elements but also the connec- timodal discourse. Foreign Lang Res
elements Feitian and the Yellow River 2003; 5:1-8.
placed in the foreground are potent tions between them, thus creating a 2. O’Toole. M. The Language of dis-
cultural and regional symbols and rep- comprehensive meaning of the logos. played art. London: Leicester Universi-
resentative. In the second logo, the ty Press, 1994.
CONCLUSION 3. Kress G.T. van Leeuwen T. Reading
Sculpture and the characters “中国兰 Images: the grammar of visual design.
州” are placed in the foreground with London: Routledge, 1996.
relative big size and most salient. As The two logos about Lanzhou are 4. Hu Z. Multimodalization in social semi-
composed of different modes such as otics. Lang Teach Ling Stud 2007;1:1-
for the general salience on the page, 10.
the image is of higher salience with vivid visuals which are the symbols or
5. Zhu Y. Theory and methodology of
more than half percentage of the logos. representatives of the city and very multimodal discourse analysis. Foreign
These most salient elements are the brief verbals which focus on the topic Lang Res 2007; 5:82-6.
most eye-catching in the composition, and the propaganda to correspond to 6. Chen Y, Wang H. Ideational meaning
the characteristics and ultimate goals of image and text-image relations. J
not just because they are placed in the Ningbo Univ (Education Edition)
foreground and form the biggest ele- of propaganda of logos. The multi-
2008;1:124-9.
ments in the pictures but also because modal discourse analysis of logos 7. Zhang W. Multimodal discourse analy-
they are in sharper focus and present overcomes the limitations of mono- sis from the perspective of social se-
lingual methods which ignores the miotics. J Northeast Norm Univ (Philos
the major topic and meaning of the Soc Sci) 2012;2:224-5.
logos. interaction of different semiotic
8. Hu J, Zeng L. Constructing semiotics
modes and provides a more effective in multimodel discourse: a study of ac-
Framing way to create and understand the ademic speech presentation. Foreign
Framing refers to the presence or ab- meaning of such city logos. Accord- Lang Their Teach 2007;3:12-5.
ing to the analysis, we could see mul- 9. Qiao Y. A multimodal analysis to a
sence of framing devices (realized by movie poster-“Slumdog Millionaire”,
elements which create dividing lines, timodality is an effective way to pre-
the Biggest Winner at 81st Oscar. Lit-
or by actual frame lines) that discon- sent information about the characteris- eratures (Lang Res) 2010; 8:144-6.
nect or connect elements of the image, tics of a city in meaning-making pro- 10. Zeng F. A Study of image-text rela-
signifying that they belong or do not cess of logos. Through the interaction tions and their patterns in multimodal
of these different semiotic elements, text:comments on three theories of im-
belong together in some sense (4). age-text relations. J Sichuan Int Stud
In the first logo, there is media- the logos express much more abun- Univ 2010;4:60-4.
degree-connection between the image dant information than separate ones. 11. Wang J. A case study of visual
elements. Two zigzag lines that form In addition, based on the above analy- semiosis by multimodal critical dis-
sis, we could also see the first logo is course analysis. J Int Commun
the edges of the Yellow River run 2012;4:51-6.
through the middle of the picture, and relatively more vivid, informative and
12. Wang L. A Study about a baidu logo
one line is also crossed with one dynamic than the second one.■ based on multimodal discourse analy-
Feitian’s arm to show the connection. Author Contributions sis. J Panzhihua Univ 2011;28:76-8.
The blurred line between the image 13. Zhang S. A comparative multimodal
The paper is written by Junbang He. Pro- analysis of a foreign enterprise logo
part and the verbal part also show the fessor Shunliang Shi is the tutor who of- and a domestic enterprise logo. North
close connection, and the word “在” fers very earnest tutorial instruction in the Liter 2010;7:96-7.■
and “Lanzhou Tour” are connected work.
closely by the image. The image of Abbreviations
BASE
BASE

 Publishing Home  Announcements  English   Login  Register
SEARCH
Advance search
CURRENT ARCHIVES ANNOUNCEMENTS ABOUT 
Home / Archives / Vol. 6 No. 1 (2014) / Original Article
JOURNAL CONTENT A Multimodal Discourse Analysis of Two Logos about Lanzhou China
LATEST PUBLICATIONS
PDF
Published Jan 12, 2014
DOI https://doi.org/10.15354/si.14.or004
Junbang He 
School of Foreign Languages, Lanzhou University of Technology, Lanzhou, China.
Abstract
BACKGROUND Logos are typical multimodal discourses combining languages, pictures, colors and so on with spatial distribution and o en
used in propagandizing a city. Lanzhou, the provincial capital city of Gansu Province in north-west China, released two logos about the
Make a Submission image and the tourism successively to propagandize the city.
ABOUT JOURNAL OBJECTIVE The purpose of the study was to analyze the logos from the perspective of linguistics and to identify how images and words in
the logos work together to fulfill the aim of propagandizing the city, what the respective advantages and disadvantages of the two logos
SUBMISSION are, and how to achieve a better propaganda aim of the city logos.
METHOD Based on the Kress and Van Leeuwen’s visual grammar, this paper attempts to adopt a multimodal analysis and descriptive-
INFORMATION
analytical method to two logos about Lanzhou China and make a comparison between them, analyzing the representational meaning, the
interactive meaning and the compositional meaning of the two logos.
TOPICS
CONCLUSION We found that images and words can work together to create a synergic e ect of propagandizing the city, and multimodality is
an e ective way to present such information about the characteristics of a city in meaning-making process of logos. Through the interaction
of these di erent semiotic elements, the logos express much more abundant information than separate ones.
Keywords 
References 
How to Cite 
Issue 
Section 
Copyright Information 
Similar Articles
Brian Finley, Preventive Strategies in Medicine: Cancer Screening , Science Insights: Vol. 2017 No. 4 (2017)
You may also start an advanced similarity search for this article.
Copyright © 2009-2021 The Bonoi Group | Insights Publisher | RIN 531477 | All Rights Reserved
pISSN: 2372-8191 || eISSN: 2329-5856
The work published on this site is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Our Privacy Policy is subject to change. Please note that by clicking on a link, you consent to the updated version.
Cookies are used by this site. To decline or learn more, visit our Cookies page.
International Journal of Academic Research in Business and Social Sciences
Vol. 1 1 , No. 9, 2021, E-ISSN: 2 2 2 2 -6990 © 2021 HRMARS
A Multimodal Analysis of Cartoon in Jordan

Isra Mahmoud ALjrah, Rafeah Legino, Rusmadiah Anwar
To Link this Article: http://dx.doi.org/10.6007/IJARBSS/v11-i9/11037 DOI:10.6007/IJARBSS/v11-i9/11037
Received: 09 July 2021, Revised: 04 August 2021, Accepted: 30 August 2021
Published Online: 10 September 2021
In-Text Citation: (ALjrah et al., 2021)

To Cite this Article: ALjrah, I. M., Legino, R., & Anwar, R. (2021). A Multimodal Analysis of Cartoon in Jordan.
International Journal of Academic Research in Business and Social Sciences, 11(9), 521–546.
Copyright: © 2021 The Author(s)

Published by Human Resource Management Academic Research Society (www.hrmars.com)
This article is published under the Creative Commons Attribution (CC BY 4.0) license. Anyone may reproduce, distribute,
translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full
attribution to the original publication and authors. The full terms of this license may be seen
at: http://creativecommons.org/licences/by/4.0/legalcode
Vol. 11, No. 9, 2021, Pg. 535 – 546

http://hrmars.com/index.php/pages/detail/IJARBSS JOURNAL HOMEPAGE
Full Terms & Conditions of access and use can be found at

http://hrmars.com/index.php/pages/detail/publication-ethics
535
A Multimodal Analysis of Cartoon in Jordan

Isra Mahmoud ALjrah, Rafeah Legino, Rusmadiah Anwar
Faculty of Art & Design, Universiti Teknologi MARA 40450 Shah Alam Selangor Malaysia
Email: israhassan_2010@yahoo.com, rafeahl@uitm.edu.my, rusma9252@uitm.edu.my
Abstract
A cartoon is a kind of two-dimensional painting which can also be animated. The definition
has changed in recent years, and these days the utilisation of cartoons is referred to as
ordinarily non-realistic or could be the semi-realistic artistic style of drawing or painting, a
picture or an arrangement of images expected for parody or humour, or could be a movie
that depends on a succession of paintings for its liveliness. Any artist who makes cartoons is
called a cartoonist. In Jordan, cartoons are regarded as political commentary; they are seen
as an expression of some political points that should be noted, but many of the cartoons are
not linked to the Jordanian cultural discourse and are not analysed due to the lack of previous
studies which have been done in the Middle East. Also, there is a need to highlight the
importance of the interaction of semantic and linguistic elements in Jordanian media to
determine the more common and suitable pattern. Therefore, this study aims to be an
essential reference for Jordanian cartoonists and researchers to develop the cartoon industry
in Jordan and increase its influence and influence in society, as it is in developed societies.
Through a review of previous studies on the analysis of social discourse and presented studies
on the analysis of multimedia in cardboard in developed countries, the evaluation of the
importance played by cartoons in various areas will explain the gap in previous studies in the
Middle East and particularly in Jordan. The potential future study, the data analysis through
modality, could be a possible topic for future research to explore interpersonal meaning
created between authors and viewers and the effect of social media on the multimodal
analysis of cartoons in Jordan.
Keywords: Analysis, Cartoon, Discourse, Social, Multimodal, Jordan
Introduction
Cartoons are rich sources of information. Being practical specialised tools, Cartoons are vital
to the aggregate instruction of each individual from the general public they depict. In the
western world, the level of flexibility and respecting rights has no restrictions and boundaries,
in the two sides: social and political, so any of these studies around this issue will be great,
and the result is linked to the social discourse in the western society, so the cartoon themes
are relevant to the address in western culture, and the cartoon models are analysed based on
the theme used (Gill & Singh, 2015). While the general public and legislative issues are so
delicate to exposure and feedback, Cartoon is utilised as a non-coordinate route for disclosure
in the Middle East. New ages in the Middle East are witnessed. In Jordan, they began to use
the latest innovations of strike back Jordan (Al-Mahadin, 2003).
536
In Jordan, Cartoon is regarded as a political comment; they are seen as an expression

making some political points that should be noted, but many of the cartoons are not linked
to the Jordanian culture discourse. They are not analysed due to the lack of theme
understanding and studies, so by having a closer look at this issue, we can realise that there
was a gap in the previous studies done in the middle east. Therefore, this study will cover the
gap between the earlier studies and the need for more investigation on how the cartoon social
discourse in the Middle East reflects the society in the Middle East and Jordan by linking it
with the Jordanian social address in Jordan. Also, there is a need to highlight the importance
of the interaction of semantic and linguistic elements in Jordanian Media to determine the
more common and suitable pattern.
Images present a fascinating and challenging field of interest. There is no single
definition of images, though different meanings and usage can overlap. According to Ludwig
Wittgenstein, to make sense of a term such as an image, one must perceive a complex
network of relations between the different meanings. Taking Wittgenstein’s point of view,
there cannot be absolute clarity in terms of a philosophical concept of the image, nor a
designated set of tools to guide research in advance. The image remains a focal point in visual
culture studies which are broader than image studies. David Morgan's definition of visual
culture makes the basis of this article an analytical approach. Visual culture is the act of seeing
and attendant intellectual, emotional, and perceptual sensibilities to build, maintain, or
transform the world in which people live. The study of visual culture is the analysis and
interpretation of images and the way of seeing (gazes) that configure the agents. The
objective is to be an essential reference for the Jordanian cartoonists and researchers to
develop the cartoon industry in Jordan and increase its influence and influence in society as
it is in developed societies. Through a review of previous studies on the analysis of social
discourse and presented studies on the analysis of multimedia in cardboard in developed
countries and review the importance played by the cartoon in various areas will explain the
gap in previous studies in the Middle East and particularly in Jordan.
Critical Discourse Analysis

Dialect is proof of the amazingness of man on the planet rather than different animals—
human impacts to other people and creatures which do not express their sentiments,
purposely and reliably. Verbal highlights of dialect interest the members to convey, and the
dialect used in correspondence is the discourse, where “discourse is a dialect used over a
sentence” (Stubbs, 1983, p.1). The investigation of discourse is, fundamentally, the
examination of dialect being used (Mary-Totry, 2015). Essential discourse investigation (CDA),
notwithstanding, is a type of discourse diagnostic examination that dominatingly investigates
the way social power is mishandled, strength and disparity are established, duplicated and
opposed by content and discourse in the social and political setting" (Van-Dijk, 1993a). Al-
Ali’s cartoons influenced millions of people throughout the Arab world’s His cartoons were
not intended to entertain the readers but instead always conveyed political messages. In
addition to expressing his personal views, they were sarcastic and daring reflections of the
feelings of the Palestinian refugees. The loss of Palestine was the main inspiration for his
cartoons. Therefore, he dedicated most of his comics to the suffering of his people, especially
the poor living in the refugee camps. His work was also dedicated to the oppressed people of
the Arab world and oppressed people worldwide. Although most of his cartoons were very
pessimistic, some were full of hope, dreams, and aspirations for a better life for the Arab
people and the Palestinians. Al-Ali addressed taboo issues while avoiding the strict censorship
537
imposed on Arab newspapers. His cartoons were simple, straightforward to understand and
were often published next to editorials with political messages. The comics spoke to and
about ordinary people regarding their political or economic or even mimic issues. His readers
waited eagerly to see his drawing on the last page (which became their front page) of many
Arab dailies in Lebanon, Kuwait, Tunisia, Abu Dhabi, Egypt, London and Paris.
On the other hand, many Arab countries prohibited him from entering and banned his
cartoons from their local newspapers (Mandell, 2016). The straightforward idea of essential
discourse examination (CDA) tries to handle, reveal and repulse social separation.) Handala’s
figure can be seen at festivals and parades alongside other Palestinian iconographic symbols,
57 such as the map of villages destroyed during the Nakba. Handala has also been adopted
as the formal icon of UNESCO’s freedom organisation. A 2012 film about Arab life was greatly
admired throughout the Arab world (Arjan, 2010). In 2007 a documentary film called The Icon
was made about his life, and in 2008 he was a dominant figure at the 60th commemoration
of Al-Nakba. Many exhibitions of his work have been shown worldwide under the title
“Shooting the Witness.” They have advanced methodologies and speculations for
investigating multimodal discourse, which depend on Halliday’s (2004) foundational
utilitarian phonetics. Halliday’s (2014) meta work is considered the essential precept of
fundamental practical hypothesis to break down multimodal discourse. It gives a brought
together stage to offer speculations that show how unique semiotic assets cooperate and
build the planned importance (O’Halloran, 2008).
Gunther Kress and Theo van Leeuwen (Mary-Totry,2013) are the occasions of
principle researchers who did ponder towards multimodality. Pictures and visual examples of
a different scope of writings, including ads, were the focal point of Kress and Van-Leeuwen’s
(1996, 2006) investigation following Halliday’s (2004) fundamental practical approach. The
message quickly, clearly and descriptively. Often these means have enabled the masses to
see their social, economic and political reality in a new and critical light. Social, economic and
political cartoons are a popular tool of expression in the media. Cartoons appear every day in
the newspapers, often adjacent to the editorials.
In many cases, cartoons are more successful in demonstrating ideas and information,
which are complex verbal explanations that require a significant investment of time by the
writer and the reader. Comics attract attention and curiosity, can be read and understood
quickly and can communicate subversive messages camouflaged as jokes that bring a smile
to the reader's face. Cartoons become more effective and successful in countries with strict
censorship and widespread illiteracy, among them many countries in the Arab world. Dialect
is characterised by Eggins (2004) as relevant, useful, semantic and semiotic. She additionally
pronounces that the most explained semiotic framework is dialect as in any communication;
the conversationalists get associated with passing on precisely a message the crowd ought to
know about and the correct decision of word. Etymological or visual assets alone aren't as
imperative and successful as when they are as one with different assets in a necessary process
(Nugroho, 2009). Christie (2005) contends that multimodality has been with us since
individuals used to draw or paint on the divider to speak with each other.
Previous studies focused on critical discourse without relating it to the culture In the
Middle East and considering the younger generation connected to social media technologies
such as Facebook and Twitter. This study is more updated regarding the Middle Eastern and
Jordanian culture and focused more on the contemporary Character Model in the Jordanian
caricature. As Jordanian authorities increasingly relied on press gag orders in 2016 to prevent
public reporting on sensitive issues. In 2016, authorities imposed gag orders on news stories
538
such as a complaint by orphans against the Ministry of Social Development; a street assault
on an Egyptian worker in Jordan; a security operation in the northern town of Irbid in March
in which seven militants and one policeman were killed; an attack on a General Intelligence
Directorate (GID) office north of Amman that led to four deaths; and the cases of Amjad
Qourshah and Nahed Hattar. On August 29, Jordan’s media commission prohibited local press
outlets from publishing any news about the king or the royal family other than information
circulated by the royal court. A legislative overhaul of the penal code was put to parliament in
2015, proposing to amend at least 180 articles of the 1960 code. At the time of writing,
lawmakers had still to pass the reforms. For the first time, the draft amendments provided
alternatives to imprisonment, such as community service.
Multimodal in Cartoon
The 'multimodality' term started from two weighty works of Kress and van Leeuwen, precisely
'Perusing Images' (1990) and 'Perusing Images: The Grammar of Visual Design’ (1996). These
two books were the base of multimodality look into (Kaltenbacher, 2004). The connection
between different semiotic modes and their significance in the making of each semiotic
method in the plan of semiotic items (e.g. magazines, leaflets) are the primary concern in the
investigation of multimodality (Kress & Van-Leeuwen, 2001). To be sure, in a multimodal
content, verbal and visual components are united to pass on the message better. They
featured that writings have dependably been multimodal as the composed content may not
exist without spaces, shading and edges (Kaltenbacher, 2004). To be sure, reports with a few
methods of discourse can be dissected through devices and strategies given by multimodality.
The synchronous nearness of picture and sound through PC, web and TV makes multimodality
conspicuous these days. Kress and Jewitt (2003) allude mode to the "regularised set of assets"
which develop significance, for example, motion, discourse, picture and so forth. Having
mindfulness about limitations in a culture and being familiar with the capacity of every mode
would decide how every individual picks the methods to communicate something specific
appropriately. It merits specifying that ways, by and large, go with and supplement each
other.
Distinctive semiotic assets have been broken down utilising this hypothetical structure
to recognise their significance in various discourses. Kress and Jewitt (2003) uncover that a
mix of multiple modes would pass on the message, and all the included ways pass on the
surmised importance since every method is in charge of one player in the message. This reality
makes an all the more intense multimodal correspondence. It has been demonstrated that
the watchers' mindfulness and comprehension would increase if at least two semiotic assets
are combined. Each semiotic support has its part in building up content and incorporating
them to create a more conceivable brought together range. Since the 1990s, distinct
researchers (O’Toole, 1994; Lemke, 1998; Martinec, 1998; Van Leeuwen, 1999; O’Halloran,
2005; and Jones, 2006) investigated multimodal discourse examination (for example,
activities, compositions, model and design) from Halliday's Systemic Functional point of view.
Michael O'Toole (1994), Gunther Kress and Theo van Leeuwen (1996) stretched out this
hypothesis to the understanding of the ways that visual modes understand. They are essential
in social settings, and they were the pioneers of the exploration of multimodality.
Gunther Kress and Theo van Leeuwen (1996, 2006) are the cases of fundamental
researchers who completed examinations on multimodality. Their system depends on the
semantic connection inside semiotic assets with a specific goal to make between related
frameworks and structures. They investigated metafunction and rank in artworks, design,
539
showed craftsmanship and engineering. Kress and van Leeuwen (2001) alluded to Halliday’s
hypotheses of social semiotics as the purpose of take-off of social semiotic multimodal
examination. Metafunction as the most persuasive piece of foundational utilitarian belief for
multimodal discourse examination Halliday's (2004). Following Halliday's SFL hypothesis Kress
and van Leeuwen give another social semiotics way to investigate the visual mode. Pictures
and seeing examples of various writings, including promotions, was the focal point of Kress
and van Leeuwen's (1996, 2006) examination following Halliday's (2004) fundamental
utilitarian approach. Their approach featured that meta functions can separate a picture or a
multimodal message during the time spent appreciating significance from the visual
components introduced (Sørensen, 2011).
As it was said, social semiotics is the reason for the hypothesis of multimodality. Siegel
(2006) claims that multimodality can be effectively comprehended by semiotics as “it offers
a state of mind about significance and content that does not benefit dialect over other sign
frameworks”(Siegel, 2006, p. 68). Van Leewen (2005, p.120) trusts that “pictures are
additionally used to get things done to or for or with individuals: induce, educate, and clarify
et cetera”. Meurer (2010) contends that advances in innovation realise another important
pattern that provokes semantics to react to this pattern by new types of proficiency, for
example, multi-education, which incorporate both visual and phonetics components. In such
a manner, educators themselves should know about the difficulties and troubles of perusing
and understanding multimodal content. Consequently, to outfit understudies with the
abilities essential to peruse multimodal messages thoroughly and basically, multi-proficiency
comprises different methods of sign making and not simply dialect (Meurer, 2010).
It ought to be noted discoveries demonstrated that illustrated talk has a huge part in
the forty-fourth presidential race of America and such factors as utilising film images, triumph
in sports, bolster from vital people and associations on the planet, bolster from Democrats,
bolster from mate, bolster from nature, bolster from the notable statue in American, bolster
from individuals and being contrasted with contenders have influenced the triumph of Obama
in the forty-fourth presidential decision of America.
The Perspective of Multimodal in Asian Countries

Malaysia is a multilingual and, in addition to the multicultural nation, which additionally
grasps a significant number of races and non-natives from Western to Middle East nations.
According to (Salveen Kaur Gill A/P Sakdev Singh 2015), he chose only spotlights on the main
stories of the “Versatile World” magazines, which give promoting data relating to the new
results of the well-known brands, for example, Samsung, Apple, and BlackBerry explored as
far as a conceptual point of view to distinguish visual and etymological components utilising
Kress and van Leeuwen's portrayal investigation and Halliday's Transitivity framework
individually. Different viewpoints, for example, the format of the "Versatile World '' magazine,
can be broke down in other research to see how unique semiotic components are assembled
to pass on a solid message.
About Hamid and Zafar (2017) Study “construction of south Asian politics: a study of
Cartoons”, The Critical discourse analysis (CDA) of the selected Asian Cartoons enhances the
literal meanings of the text regarding power abuse and inequality. These Cartoons are
installed to control governmental issues in the South Asian area. By a similar token, the
examination ties the printed discourse with the mental design of the sketch artist that
portrays his “intrinsic complex” as well as his uneasiness and sensitivity. The denotative and
obvious articulations of discourse help expound the grimy picture of governmental energy
540
issues. Furthermore, the data analysis through modality can be a potential topic for future
research to explore interpersonal meaning created between authors and viewers.
Term and Role of Cartoon

The caricature is derived from the Italian word “caricature” or “carcase”, which means
“exaggerate”, cartoons play a persuasive social and political party. Exaggerations can
represent the sufferings of nationals by handling the issues of the general public, breaking
down financial matters, and investigating problems. Political exaggerations can make help
abused individuals by reprimanding existing conditions and unjustifiable political practices.
Personifications can bargain strongly with social issues since they can escape restriction. It is
conceivable to state that exaggerations can remedy issues quicker than composing words
(Hafiz, 2006). By definition, (matty,2013;). Subsequently, "exaggeration” basically implies a
"stacked picture" (Wikipedia, 2009). As Marshall McLuhan (Boeschoten, 2006) says, this is
valid for cartoons as a medium. Exaggerations can be eulogistic and additionally fundamental.
Much has been composed concerning why personifications are compelling from a stylish point
of view (Gombrich, 1960; Ramachandran and Hirstein, 1999) and how personifications are
made from a useful point of view (Gautier, 1985; Redman, 1984). However, in library and
data science (LIS), personifications have been minimally considered from an exploration point
of view.
It has not been wholly discoursed or bantered about to utilise exaggerations to
recognise negative and feeble focuses. The consequences of this examination can enable
custodians to accomplish a superior status and convey the actual place of the library, library
assets, and librarianship. Administrators must be the first part to distinguish and comprehend
the inadequacies and disadvantages of our calling. This target can be seen in Toghan’s (Hafiz,
2006) saying: exaggeration isn’t only a joke, be that as it may, it is [a] change and sanitisation
[weapon]. The creators need to empower and animate more dialogue and discussion,
advance basic reasoning, attract custodians’ thoughtfulness regarding thoughts portrayed by
personifications, and advertise librarianship as an appealing and significant calling, express a
number or a repertoire of meanings at the same time because at this case they will be the
type of artists who can affect or lead the masses towards their rights here.
Today, the media assume a viable part in improving the consciousness of individuals
and their choice making. The more a subject is critical for individuals, the more
Media are focused on, and Pictures can influence crowds on their feelings and drive their view
of news towards the objective. Among different pictures, exaggerations can be brought up,
which are seen today in numerous daily papers, magazines and news locales nearby critical
news (Soltanifar & Bakhshi, 2010). Over the twentieth century, photographic, film, and
televisual pictures have assumed an inexorably essential part in developing and documenting
notable occasions. The appearance of broad communications has prompt a multiplication of
pictures in non-inventive correspondence. In any case, pictures likewise have a focal impact
on how creative discourse s conceptualise history; Mitchell has instituted the expression
“pictorial turn” for this expanding "images" in contemporary correspondence, taking up
Richard Rorty's contention that the historical backdrop of logic has advanced through a
progression of worldview changes, or “turns”: The photo of antiquated or medieval sense as
worried about things, the rationality of the seventeenth through the nineteenth century as
worried about thoughts, and the illuminated contemporary philosophical scene with words
have extensive credibility. Hence, pictures are not just the world view in which modern
culture must characterise its uncertain issues. However, they are adding the medium in which
541
these issues are represented. One of the variables that helped Barack Obama win the forty-
fourth presidential race was his solid media group. This article manages the part of pictorial
discourse s in media, concentrating on cartoons, in the forty-fourth presidential race of
America.
Cartoon in Jordan
In March, Jordan’s Ministry of Social Development issued amendments to Jordan’s 2008 Law
on Associations. The amendments place onerous restrictions on establishing civil society
groups and grant the government legal authority to dissolve groups on vague grounds or deny
their ability to obtain foreign funding without justification. If enacted, the changes would
severely hamper the ability of NGOs to form and operate. By November, the amendments
were still under consultation and had not been submitted to parliament.
Since the amended Public Gatherings Law took effect in March 2011, Jordanians no
longer require government permission to hold public meetings or demonstrations. However,
Amman hotels and other venues continued to seek permission to host public meetings and
events. In April, authorities cancelled a concert by the famous Lebanese music group
Mashrou’Leila, reportedly over the claim that their songs “contain lyrics that do not comply
with the nature of the Jordanian society.” The day before the concert was held, authorities
reversed the decision, but organisers said they did not have time to stage it. The previous
three decades in Jordan have conveyed to the cutting edge gendered morals of presence that
looks to engrave a dream of womanliness that opposes the deterministic and abusive social
parts forced by the prevailing male-centric mores and qualities. Emancipatory techniques
have involved the notorious protection of social and political praxis. Yet, no maintained
exertion has been applied to question the epistemological grounds of these unbending and
naturalised abusive developments. In that capacity, I give a point-by-point diagram crafted
by a conspicuous Jordanian illustrator/caricaturist, Imad Hajjaj, however, with specific
accentuation on the portrayals of underestimated gatherings, principally ladies, that have
risen in his work of late years. Remembering the unpredictability of his portrayals and the
trouble of evaluating the different digressive practices reflected in his career, the paper might
confine itself to the part such toons play in setting up and fortifying pictures, social images,
commonplaces, tropes, what're more generalisations.
A progression of inquiries have been rationally postured while subjecting these
Cartoons to the rigours of verbose analysis: 1 whose voice develops in these Cartoons? Do
they specifically speak or mirror the hidden social practices, dialect amusements, and
learning/control divisions of the society inside which they have tried to discover a crowd of
people? What are the procedures of emblematic demolition, allegorical portrayals,
iconoclasm, furthermore, dichotomising that underlie their creation? Is Imad Hajjaj the visual
artist likewise, Imad Hajjaj is the craftsman as far as utilising the style and morals of the
capable columnist to depoliticise the tasteful circle (Felski, 1989: 175)? I have contended
somewhere else (Al-Mahadin, 2001) that the situation of ladies starts with the development
of their subjectivity in discourse. The conclusive connection between information/control
exhibited by Michel Foucault in a profoundly rooted vocation on the nexus amongst
subjectivity and learning/control as uncovered by discourse suggests that such subjectivity
be situated in the content and stood up to through movements in address. Political and social
practices have occasioned the Cartoons under investigation yet the literary procedures
utilised to duplicate and propagate the man-centric task of womanliness development
through stereotyping. Establishing those Cartoons in a women’s activist venture of desultory
542
the examination ought to underscore the significance of deconstructing such renderings of

womanliness if ladies’ strengthening is ever to wind up an undeniable reality. In this way,
“our comprehension of the issues of “genuine” ladies can't lie outside the “envisioned”
builds in and through which “ladies” rise as subjects”(Rajeswari S. Rajan, 1994, p. 10). The
situation of ladies in Jordan ought not to be limited to the standard techniques for altering
laws and presenting enactments, yet additionally, be mixed with a women’s liberation
established in a conviction that “each portrayal depends on and duplicates a particular
rationale of the genuine; this sensible, genuine advances its portrayal” (Peggy-Phelan, 1993,
p. 10).
Conclusion
In conclusion, this study contributes to the body of knowledge by suggesting a model for a
better understanding of Cartoons as a Social communicative tool in Jordan and specifically
how Cartoons are used as social statements. Also, this study contributes to the media
production industry in the Middle East and Jordan, mainly by providing a holistic framework
that can help improve and develop the Quality of cartoon production. This study focuses on
how the new cartoon and media communication between the new Jordan generations plays
a vital role in highlighting the social cartoons in Jordan and popularising them to be used as
an important way to criticise society in Jordan. In March, Jordan’s Ministry of Social
Development issued amendments to Jordan’s 2008 Law on Associations. The amendments
place onerous restrictions on establishing civil society groups and grant the government legal
authority to dissolve groups on vague grounds or deny their ability to obtain foreign funding
without justification. If enacted, the changes would severely hamper the ability of NGOs to
form and operate. By November, the amendments were still under consultation and had not
been submitted to parliament. Since the amended Public Gatherings Law took effect in March
2011, Jordanians no longer require government permission to hold public meetings or
demonstrations. However, Amman hotels and other venues continued to seek permission to
host public meetings and events. In April, authorities cancelled a concert by the famous
Lebanese music group Mashrou’ Leila, reportedly over the claim that their songs “contain
lyrics that do not comply with the nature of the Jordanian society.” The day before the concert
was held, authorities reversed the decision, but organisers said they did not have time to stage
it.
Acknowledgement
Thank you to the Institute of Postgraduate Studies (IPSis) UiTM for the financial support of this
publication.
References
Brown, G., & Yule, G. (1983). Discourse Analysis Cambridge: Cambridge University Press.
Boeschoten, R. V. (2006). Media. Critical Perspectives on International Business 2(3):
220-229.
Christie, C. (2005). Editorial. Journal of Politeness Research 1 (1): 1-7.
Eggins, S. (2004). Introduction to systemic functional linguistics. A&C Black.
Felski, R. (1989). Beyond feminist aesthetics: Feminist literature and social change. Harvard
University Press.
Gill, S. K., & Singh, S. (2015). A multimodal analysis of cover stories on mobile phones: An
ideational perspective/Salveen Kaur Gill A/P Sakdev Singh (Doctoral dissertation,
543
University of Malaya).
Gautier, D. (1985). The art of caricature. New York: Perigee Books.
Halliday, M. A. K. (2004). An introduction to functional grammar (3rd ed.), London: Edward
Arnol
Halliday, M., & Matthiessen, C. (2004). An introduction to functional grammar (3rd ed.).
London: Arnold.
Hafiz, S. A. (2006). Caricature, the laughing tragedy: Content analysis of caricatures and
cartoons published in Egyptian newspapers, namely Alakhbar-Alahram-Alwafd-
Alaraby Alosbou'-Sawt Aloma-Alahaly-Rose Alyossef magazine during the period from
March 1st to December 31st 2005.
Jones, J. (2006). Multiliteracies for academic purposes : A metafunctional exploration of
intersemiosis and multimodality in university textbook and computer-based
learning resources in science.
Jewitt, C., & Kress, G. (2003). A multimodal approach to research in education. Trentham
Books in association with the Open University.
Kress, G., & Van Leeuwen, T. (1996). 2006. Reading images: The grammar of visual design.
Kress, G. R., & Van Leeuwen, T. (1996). Reading images: The grammar of visual design.
Psychology Press.
Kaltenbacher, M. (2004). Perspectives on Multimodality: From the early beginnings to the
state of the art. Information Design Journal & Document Design, 12(3).
Kress, G., & Van Leeuwen, T. V. (2001). Multimodal discourse: The modes and media of
contemporary communication.
Lemke, J. L. (1998). Metamedia literacy: Transforming meanings and media. Handbook of
literacy and technology: Transformations in a post-typographic world, 283-301.
Woniak, M. (2014) Mirror, Mirror on the Wall:Political Cartoons of the Arab Spring.
Martinec, R. (1998). Cohesion in action. Semiotica, 120(1), 161-180.
Management Communication, Department of Language and Business Communication
Aarhus School of Business Aarhus University.
Meurer, J. L., & Machado, F. R. (2010). Context, genre and multimodality: An analysis of an
on-line news article. Nonada Letras em Revista, 12(12), 71-84.
Nugroho, A. D. (2009). The generic structure of print advertisement of Elizabeth Arden’s
INTERVENE: A multimodal discourse analysis. k@ ta, 11(1), 70-84.
O'Halloran, K. (2008). Mathematical discourse: Language, symbolism and visual images.
A&C Black.
O'toole, M. (1994). The language of displayed art. Fairleigh Dickinson Univ Press.
O’Halloran, K. L. (2008). Systemic functional-multimodal discourse analysis (SFMDA):
constructing ideational meaning using language and visual imagery. Visual
Communication (Vol. 7, pp. 443–475).
O’Halloran, K. L. (2005) Mathematical Discourse: Language, Symbolism and Visual Images.
London: Continuum.
Phelan, P. (1993). The ontology of performance: representation without reproduction.
Ramachandran, V. S., & Hirstein, W. (1999), The science of art: A neurological theory of
aesthetic experience. Journal of Consciousness Studies 6: 15-51.
Redman, L. (1984). How to draw caricatures. Chicago: Contemporary Books.
Al-Mahadin, S. (2003). Gender representations and stereotypes in cartoons: a Jordanian
case study. Feminist Media Studies, 3(2), 131-151.
Rajan, R. S. (1994). Imagined Women: Women, Gender, Culture and Postcolonialism.
544
Rhodes, G., & McLean, I. A. (1990). Distinctiveness and expertise effects with homogeneous
stimuli: Towards a model of configural coding. Perception 19: 773-794.
Stock, C. (2008). George w. bush caricatures.
Stubbs, M. (1983). Discourse analysis: The sociolinguistics analysis of national language.
OxfordBasil: Blackwell.
Sorensen, L. (2011). A Multimodal approach for advertising for professionals in the film,
television and photography business. Retrieved from Department of Language and
Business Communication Aarhus School of Business Aarhus University.
Siegel, M. (2006). Rereading the signs: Multimodal transformations in the field of literacy
education. Language Arts, 84(1), 65-77.
Siegel, M., & Panofsky, C. P. (2009). Designs for multimodality in literacy studies:
Explorations in analysis. In 58th yearbook of the national reading conference (pp.
99-111). Oak Creek, WI: National Reading Conference, Inc.
Soltanifar, M., & Bakhshi, B. (2010). The Role of Pictorial Discourses in the Forty-Fourth
Presidential Election of America.
Shafiq, Q., Hameed, M. B., & Zafar, M. A. (2017). DISCURSIVE (RE) CONSTRUCTION
OF SOUTH ASIAN POLITICS: A STUDY Of CARICATURES. European Journal of Research
in Social Sciences Vol, 5(1).
Van Dijk, T. A. (1993a). Principles of Critical Discourse Analysis: Discourse in society.
Al-Homoudi, I. A., Thakur, J. S., Naik, R., Auner, G. W., & Newaz, G. (2007). Anatase TiO2
films based CO gas sensor: Film thickness, substrate and temperature effects. Applied
Surface Science, 253(21), 8607-8614.
Alessandri, I., Comini, E., Bontempi, E., Faglia, G., Depero, L. E., & Sberveglieri, G. (2007).
Cr-inserted TiO2 thin films for chemical gas sensors. Sensors and Actuators B: Chemical,
128(1), 312-319.
Alterkop, B., Parkansky, A. N., Goldsmith, A. S., & Boxman, A. R. L. (2003). Effect of air
annealing on opto-electrical properties of amorphous tin oxide films. Journal of Physics
D: Applied Physics, 36, 552.
Ang, G. T., Toh, G. H., Bakar, M. Z. A., Abdullah, A. Z., & Othman, M. R. (2011). High sensitivity
and fast response SnO2 and La-SnO2 catalytic pellet sensors in detecting volatile
organic compounds. Process Safety and Environmental Protection, 89(3), 186-192.
Balasubramanian, G., Dionysiou, D. D., Suidan, M. T., Baudin, I., & Laine, J.-M. (2004).
Evaluating the activities of immobilised TiO2 powder films for the photocatalytic
degradation of organic contaminants in water. Applied Catalysis B: Environmental,
47(2), 73-84.
Chen, W., Zhang, J., Fang, Q., Li, S., Wu, J., Li, F., & Jiang, K. (2004). Sol-gel preparation of thick
titania coatings aided by organic binder materials. Sensors and Actuators B: Chemical,
100(1-2), 195-199.
Chen, Y., & Dionysiou, D. D. (2006). TiO2 photocatalytic films on stainless steel: The role of
Degussa P-25 in modified sol-gel methods. Applied Catalysis B: Environmental, 62(3-4),
255-264.
Garzella, C., Comini, E., Tempesti, E., Frigeri, C., & Sberveglieri, G. (2000). TiO2 thin films by a
novel sol-gel processing for gas sensor applications. Sensors and Actuators B: Chemical,
68(1-3), 189-196.
Kermanpur, A., Ghassemali, E., & Salemizadeh, S. (2008). Synthesis and characterisation of
microporous titania membranes by dip-coating of anodised alumina substrates using
sol-gel method. Journal of Alloys and Compounds, 461(1-2), 331-335.
545
Mohammadi, M. R., & Fray, D. J. (2007). Semiconductor TiO2-Ga2O3 thin film gas sensors
derived from particulate sol-gel route. Acta Materialia, 55(13), 4455-4466.
Ruiz, A. M., Cornet, A., & Morante, J. R. (2004). Study of La and Cu influence on the growth
inhibition and phase transformation of nano-TiO2 used for gas sensors. Sensors and
Actuators B: Chemical, 100(1-2), 256-260.
Ruiz, A. M., Sakai, G., Cornet, A., Shimanoe, K., Morante, J. R., & Yamazoe, N. (2004).
Microstructure control of thermally stable TiO2 obtained by hydrothermal process for
gas sensors. Sensors and Actuators B: Chemical, 103(1-2), 312-317.
Senguttuvan, T. D., Rai, R., & Lakshmikumar, S. T. (2007). Gas sensing properties of lead
doped tin oxide thick films. Materials Letters, 61(2), 582-584.
Takahashi, Y., & Matsuoka, Y. (1988). Dip-coating of TiO2 films using a sol derived from
Ti(O-i-Pr)4- diethanolamine-H2O-i-PrOH system. Journal of Materials Science, 23(6),
2259- 2266.
Taurino, A. M., Capone, S., Boschetti, A., Toccoli, T., Verucchi, R., Pallaoro, A., Iannotta, S.
(2004). Titanium dioxide thin films prepared by seeded supersonic beams for gas
sensing applications. Sensors and Actuators B: Chemical, 100(1-2), 177-184.
Yang, T., Lin, H., Wei, B., Wu, C., & Lin, C. (2003). UV enhancement of the gas sensing
properties of nano-TiO2. Rev. Adv. Mater. Sci, 4, 48-54.
Zhang, X., Zhou, M., & Lei, L. (2005). Preparation of anatase TiO2 supported on alumina by
different metal organic chemical vapor deposition methods. Applied Catalysis A:
General, 282(1-2), 285-293.
546
Mengyuan Bi, IJSR, 2019; 3:33
Review Article IJSR (2019) 3:33
International Journal of Social Research

(ISSN:2576-5531)
Multimodal Discourse Analysis of Coca-Cola’s Print Advertising
Mengyuan Bi
Shanxi Normal University.
ABSTRACT
With the development of the Internet, the expression of *Correspondence to Author:

information is not limited to words, but it develops into other Mengyuan Bi
modes, such as images, sounds, actions, etc. This study is Shanxi Normal University.
based on the visual grammar. The visual grammar proposed
by Kress and Van Leeuwen is based on Halliday’s systemic
functional linguistics. Coca-Cola as a popular soft drink brand,
its graphic advertising design can be said to be very mature How to cite this article:
and representative. This study is based on visual grammar Mengyuan Bi. Multimodal Discourse
to carry on the thorough analysis to the American Coca- Analysis of Coca-Cola’s Print Advertising.
Cola classic print advertisement, carries on the analysis to International Journal of Social Research,
each print advertisement from the reproduction meaning, 2019; 3:33
the interactive meaning and the composition meaning. The
following conclusions can be drawn: 1. In the Coca-Cola
print advertisement in the United States, the professional
characteristics of the characters are obvious. 2. Advertising
language can directly express the benefits of the product. 3. The
combination of image mode and language mode in the Coca- eSciPub LLC, Houston, TX USA.
Cola print advertisement can be explained by visual grammar. Website: https://escipub.com/
Keywords: Visual grammar; Coca-Cola; Print advertising
IJSR:Http://escipub.com/international-journal-of-social-research/ 1
1. Background of Coca-Cola and Visual Gra- course linguistics with other fields such as vi-
mmar sion and hearing(Li,2003). The visual grammar
1.1 Background of Coca-Cola proposed by Kress and Van Leeuwen is based
On 8 May 1886, Dr. John Pemberton from At- on Halliday's systemic functional linguistics.
lanta, George, United States, stirred carbonate- Kress and Van Leeuwen think, the significance
d water with soda to make a dark syrup, and it of multimodal research is that it clearly points
had the effect of refreshing, sedation and re- out that the interest in the study of modes other
ducing headache, the drink was Coca-Cola. At than words and speech is inevitable rather than
first, Coca-Cola was sold at a drugstore, with accidental, it is the center of the form of realistic
the first Coca-Cola selling an average of nine communication, not just as a supplement to
bottles a day for 5 cents. language research(Kress,2006). Halliday put
Frank Robinson, an accountant at Coca-Cola, forward three major functions of the language
inspired naming from two ingredients in syrup. in the functional grammar: ideational function,
These two ingredients are the leaves of Coca interpersonal function, textual function. The "id-
and the fruit of Kola. In order to be uniform, eational function" refers to the function of the
Robinson changed the K of Kola to C, then language to describe the "content" and the
added a horizontal painting to the middle of the function of establishing the "logic". The "inter-
two words, and wrote the famous brand logo personal function" refers to the function that
with Spence. In 1887, John Pemberton regis- language has the function of reflecting the rela-
tered the Coca-Cola syrup and concentrate tra- tionship between the speaker and the hearer.
demark at the U.S. Patent Office, acquired its The "textual function" refers to the function that
intellectual property rights, and used coupons language forms a text, and then it can realize
for the first time to promote Coca-Cola. But it's the function of information transmission(Hallid-
a bottling system that made the Coca-Cola ay,1994).
showed up. The Coca-Cola Company sells the Kress and Van Leeuwen extended the three
syrup to the bottling system, and then the syrup meta functions of language proposed by Halli-
is mixed with the bottling system and then sold day to visual texts, thus constructing the theo-
after bottling. The bottle-filling system is man- retical framework of visual grammar and putting
ufactured according to the requirements of Co- forward three meanings in visual grammar: re-
ca-Coca’s company and Coca-Cola allows bot- presentational meaning, interactive meaning,
tling systems to advertise using Coca-Cola's compositional meaning. These three meanings
trademar. With this special bottle-filling system, also construct an effective analytical framework
the factory that produced Coca-Cola has flow- for image text analysis.
ered everywhere. 1.2.1 Representational Meaning
1.2 Visual Grammar According to whether the image has "vector",
With the development of the Internet, the ex- the reproduction meaning is divided into narra-
pression of information is not limited to words, tive representation and conceptual representa-
but it develops into other modes, such as im- tion.
ages, sounds, actions, etc. It is necessary to Kress and Van Leeuwen point out that describ-
combine the unimodal study of language in dis- ing the functions of the existing designing so-
IJSR: https://escipub.com/international-journal-of-social-research/ 2
cial action expresses developing events, chan- As a medium for transmitting information, the
ging processes, or continuous spatial distribu- image can establish the relationship between
tion. According to the different forms of the ve- the image viewer and the image, and create
ctor and the participants, the description and interactive meaning, which affects the psycho-
reproduction is subdivided into action process, logical state of the reader. In the realization of
reaction process, speech and mental process. interactive meaning, contact, social distance,
The action process is considered to be one of perspective, modality. These four aspects are
the most basic forms in narrative reproduction. the main analysis point.
In this kind of narration, the actor and the action In the contact analysis of the image, Kress and
goal complete the narration of the action pro- Van Leeuwen divide the image into demanding
cess together. In such studies, participants, as and giving, according to whether the partici-
the most important part of the image, can be pants in the image and the viewer of the image
analyzed by location, form, size, etc. The reac- have eye contact or not, the producer of the
tion process is a description of the vector stru- image will influence the viewer by a definite eye,
cture through the eyes or eyes, which is dif- thus establishing the relationship between the
ferent from the action process, and the reaction viewer and the image participant. If the partic-
process image is composed of the reactor and ipant in the image has eye contact with the
the phenomenon. The process of speech and viewer from top to bottom, it means seeking
psychology refers to the process of the narra- obedience; If a participant in an image is in
tion of the relationship between the speaker contact with the viewer in a manner that is from
and the sensor through the expression of spe- the bottom to the top, it means that sympathy
ech and the reflection of the vector. This kind of or compassion is obtained. On the contrary, if
narrative process is the most common in car- an image does not determine its intention by
toons. The author often expresses the rela- looking, the viewer must receive meaning from
tionship between the speaker and the senser other information provided in the picture, then
through the bubble of thinking and dialogue. the image is the type given. Social distance
Different from narrative reproduction, concep- refers to the degree of familiarity between the
tual reproduction expresses a more abstract elements in the image and the viewer. Social
narrative process. Kress and Van Leeuwen be- close viewers tend to regard participants as
lieve that conceptual reproduction has the fun- "friends", while social distance tends to make
ction of designing social constructs, and subdi- viewers think of them as "strangers" or "others".
vides it into: classificational process, analytical Kress and Van Leeuwen believe that different
process and symbolic process. Generally spea- lenses can show different degrees of closeness.
king, the classificational process embodies the Generally speaking, if a close-up of an image
relationship between upper and lower levels or presents a participant, then what is revealed is
peers, the analytical process embodies the re- a closer relationship; on the contrary, if the
lationship between the whole and the part, and frame of the image is large, the panoramic view
the symbolic process shows the implicit mean- and the vision of the participants are presented,
ing behind the participants. then the social distance of the image is far
1.2.2 Interactive Meaning away, and the social relations are not so close.
In terms of perspective, different perspectives gree of credibility of the image. Here, the image
reflect the status difference between image reflects the true degree of reality and its pic-
viewers and image participants. The angle of ture's color, detail, depth, lighting, and social
level often gives the viewer a sense of pres- factors
ence and reflects the equal social relations; the 1.2.3 Composition Meaning
perspective of top view shows that the viewer The composition meaning in visual grammar
of the image is in a strong position; if it is a evolved from the "textual function" of the three
bottom view, the participant in the image is in a meta functions of Halliday's language. It syn-
stronger position, and the viewer is in a weaker thesizes the representational meaning and the
position. At the same time, the different per- interactive meaning, and emphasizes how the
spectives also reflect the attitude towards the text embodies the whole thought through stru-
things in the image. The positive and intuitive ctural composition and cohesion, thus express-
image makes the viewer feel close and regards ing the meaning that it wants to express. Spa-
it as part of himself, while the side squint image tial distribution, salience, framing and the use
allows the viewer to define it as "them" rather of color are few levels that can be used to de-
than "us". scribe the meaning of a composition.
In terms of modality, Kress and Van Leeuwen 2. Analysis of Coca-Cola’ s Print Adver-
believe that modality is described by the de- tisement
1 2 3
2.1 The Analysis of Representation Meaning ing the girl coke. The boy's right hand shows
Figure 1 has an obvious vector, which belongs how coke is turned on. The two of them looked
to narrative reproduction. In figure 1, there are at each other and smiled at each other, indi-
four characters at the party. Among them, two cating the joy of the party. In front of the picture
close-up people meet in their eyes. The boy is a Coca-Cola box with the slogan "Drink Co-
holds a bottle of Coca Cola in each hand, and caCola". The advertising language is a direct
the vector of holding coke in the left hand in- indication of attitude. The boy behind the cam-
tersects the vector of the girl holding coke in era holds Coca Cola in his left hand, and the
the right hand, indicating that the boy is hand- vector that produces tells us that the boy is
pouring Coca Cola for the girl. The girl looked 2.2 The Analysis of Interactive Meaning
at the boy, and both of them were happy. This The graphic advertisement in figure 1 adopts a
figure shows the happiness of a friend's party, flat-view shooting technique, giving the viewer
with four of the vectors pointing to Coca-Cola， a sense of equality. The distance between the
highlighting the focus of advertising. character in the picture and the viewer belongs
Figure 2 has an obvious vector, which belongs to the social close distance, which gives the
to narrative reproduction. In figure 2, the soldier viewer the feeling of being there. The four char-
looks at the audience and the vector points to acters in the picture do not intersect with the
the audience. In the front of the figure, one viewer's line of sight, reflecting the attitude of
hand holds the Cola Cup to form a vector. "providing". Images provide viewers with some
There is an advertising language at the bottom information, for example, a happy party be-
of the picture: yes sir, every knows ice cold tween friends can't be separated from Coca-C-
Coca-Cola is delicious and refreshing. .Soldier ola; with Coca-Cola, everyone's party will be
has words on his mouths,: Thirst asks nothing happier. The color saturation in Fig. 1 is high,
more. Behind Asks is the soldier's mouth, showing the high situation, giving the viewer
which point to coke, and forms a vector with the real feeling.
coke. It tells us to drink coke only if we are The print advertisement of figure 2 adopts a
thirsty. head-up shooting method to provide an equal
Figure 3 has an obvious vector, which belongs feeling to the viewer. The distance between the
to narrative reproduction. As can be seen from character in the figure and the viewer is per-
the picture, the background is that on a travel- sonal close, giving the viewer a feeling of clo-
ing plane, the stewardess is providing drinks to seness. The soldier in the picture intersects the
the passengers. In Figure 3, the airline stew- viewer's line of sight and expresses the attitude
ardess looked at the passenger, holding the of asking. According to the analysis of the other
plate to form a vector, on which there was Coke. participants in the print advertisement, we cou-
In addition to the characters, there are other ld know that the information which soldier ask-
participants in the picture. At the bottom of the ed for was that they wanted viewers to drink
picture is the advertising language: “There’s Coca-Cola when they were thirsty. In figure 2,
nothing more refreshing in the blue sky as deli- the color of coke and the color of coke refriger-
cious Coca-Cola. Ice-cold and soaring with the ator are very bright, and the saturation of back-
taste of real in-flight refreshment. Ask for the ground and character color is also very high,
genuine goodness of Coca-Cola. Thirst asks which gives the viewer a feeling of reality.
nothing more.” The advertising language not The plane advertisement of Fig.3 adopts a he-
only expresses the characteristics of the Coke ad-up shooting method to provide an equal
itself, but also the freshness of people after feeling to the viewer. The distance between the
drinking the ice Coke, which proves that the character in the figure and the viewer belongs
Coke is not replaceable. On the left side of the to the close range of the society and gives the
picture is the brand logo of coke, and at the viewer an immersive feeling. The line of sight of
bottom right is the price of coke and the adver- the characters in the picture does not intersect
tising language: quality carries on. with the viewer, reflecting the attitude of "provi-
ding". The image should be provided with some shows the focus of the interactive meaning and
information to the viewer. As can be seen from the narrative meaning. Character occupies on-
the figure, the information that the image wants e-half of the graph. The slogan “There’s noth-
to express is, the passenger wants to drink the ing more refreshing in the blue sky as delicious
coke in the flight of the figure, the airline stew- Coca-Cola. Ice-cold and soaring with the taste
ardess 's dish is only coke, and it shows that of real in-flight refreshment. Ask for the genuine
the coke can't be replaced. In Fig. 3, the satu- goodness of Coca-Cola. Thirst asks nothing
ration of the color in brand logo of Coke, back- more.” located at the bottom of the print adver-
ground and character are also very high, which tisement which is the location of the real in-
gives the viewer a real feeling. formation. And it reflects the reliability and the
2.3 The Analysis of Composition Meaning authenticity of the content of the utterance. The
The male and female of the near-lens in Fig.1 brand identity is located at the location of the
are in the center of the figure with high signifi- known information, Advertising quality carries
cance, and the composition shows the focus of on.'' is located in the area of unknown infor-
the interactive meaning and the narrative mea- mation. It tells us that the brand will continue to
ning. Characters occupy one-half of the graph. improve, so that viewers are full of expectations.
The Coca Cola box in the front of the figure is At the top of the picture is the ideal message,
perfectly combined with the central character "fly...refreshed" shows that the ideal condition
through the vector, The color of coke box is for flying is to drink coke on the way.
bright red, which contrasts sharply with the 3. Conclusion
background color, and the convex dominance In the American Coca-Cola print advertisement,
is high. the character image accounts for about 1 / 2 of
The soldier in FIG.2 is in the center position the whole advertisement picture, the expres-
and the significance is high. The composition sion of the characters on the screen is clearly
shows the focus of the interactive meaning and visible, most of them are Coca-Cola in hand,
the narrative meaning. Character occupies on- their cheeks are ruddy, and they drink happily. it
e-half of the graph. The soldier's words are in is an equal gesture for women to appear inde-
the center of the graph, between known and pendently or at the same time as male images.
unknown information. Such expressions tend to For example, in a Coca-Cola advertisement,
arouse the curiosity of viewers. “yes sir, every there are images of men and women sitting
knows ice cold Coca-Cola is delicious and re- side by side in the picture, and men are pouring
freshing.” This sentence is located in the real- Coca-Cola for the women around them. We
istic position in the picture which reflects the can know that the professional characteristics
reliability and authenticity of the discourse of characters are very obvious, including airline
content. In figure 2, the color of coke and the stewardess soldiers and so on.
color of coke refrigerator are very bright which In Coca-Cola advertising in the United States,
contrasts sharply with the background color, the character's perspective is to look at con-
and the convex dominance is high. sumers and directly show the pleasure of
Fig. 3 the stewardess is in the central position, drinking Coca-Cola. The United States is a
and the significance is high. The composition low-context culture that emphasizes straight-
forward expression and directly tells consum-
ers the benefits of their products. In the early
stage of brand construction, it is necessary to
cultivate and construct the consumer group's
cognition of new products and new culture thr-
ough advertising. The audience will only accept
the loyalty to the brand until they understand
the product first. Coca Cola Advertising in the
United States turned to perceptual appeal in
1935, emphasizing the good feeling of drinking
Coca Cola. As the advertisement says, "There’
s nothing more refreshing in the blue sky as
delicious Coca-Cola. Ice-cold and soaring with
the taste of real in-flight refreshment." The at-
traction of the product is not only its own edible
function, but also the social function of the
product. The satisfaction of socialized needs
becomes more and more important. At this time,
Coca-Cola is not only a kind of soft drink, but
also a link of interpersonal communication,
which is undoubtedly to promote the product
from the perspective of sensibility rather than
rationality.
Reference
1. Li zhanzi. Social semiotic analysis of multi-
modal discourse [J]. Foreign language studies,
2003(5):1-8.
2. Halliday, M. A. K. An Introduction to Functional
Grammar (the second edition) [M]. London:
Edward Arnold, 1994.
3. Halliday, M. A. K. Language as Social Semiotic:
The Social Interpretation of Language and
Meaning [M]. London: Edward Arnold, 1978.
4. Kress, G. ＆ Leeuwen，Theo Van. Reading
Images: The Grammar of Visual Design [M].
London: Routledge，2006.
Home Journals Articles Editors Submit Manuscript Guidelines Contact Us

 Home / International Journal of Social Research /
Journals by Subjects
 Review Article of International Journal of Social Research
Biological and Life Sciences
Chemistry and Materials Sciences
Computer Science and Mathematics
Earth and Environmental Sciences

Economics and Business
Education and Humanities Mengyuan Bi
Engineering Shanxi Normal University

Medicine and Healthcare
Social Sciences
With the development of the Internet, the expression of information is not limited to words, but it
Physical Sciences develops into other modes, such as images, sounds, actions, etc. This study is based on the visual
Others grammar. The visual grammar proposed by Kress and Van Leeuwen is based on Halliday’s systemic
functional linguistics. Coca-Cola as a popular soft drink brand, its graphic advertising design can be
Recent Articles
said to be very mature and representative. This study is based on visual grammar to carry on the
Psychiatric disorders and hepatitis C thorough analysis to the American Coca-Cola classic print advertisement, carries on the analysis to
treated with direct-acting antivirals each print advertisement from the reproduction meaning, the interactive meaning and the composition meaning. The
in a French reference center following conclusions can be drawn: 1. In the Coca-Cola print advertisement in the United States, the professional
The clitoris in the chromosomes, and characteristics of the characters are obvious. 2. Advertising language can directly express the benefits of the product. 3.
immunity – an essay The combination of image mode and language mode in the Coca-Cola print advertisement can be explained by visual
FOOT AND ANKLE CHARACTERISTICS grammar.
IN GOUT: A SYSTEMATIC REVIEW
Hot Articles Keywords: Visual grammar; Coca-Cola; Print advertising
COVID-19 AND AGEING-RELATED

EVENTS
Free Full-text PDF

Telomere Length in Cystic Fibrosis
Patients – Are Patients with CF
How to cite this article:
Ageing Too Quickly?
Psychometric Characteristics of the Mengyuan Bi. Multimodal Discourse Analysis of Coca-Cola’s Print Advertising. International Journal of Social Research,
Mini-Mental State Examination among
2019; 3:33. DOI: 10.28933/ijsr-2019-09-0806
Older Chinese Immigrants
Submit Manuscript
References:
1. Li zhanzi. Social semiotic analysis of multi-modal discourse [J]. Foreign language studies, 2003(5):1-8.
Contact us 2. Halliday, M. A. K. An Introduction to Functional Grammar (the second edition) [M]. London: Edward Arnold, 1994.
3. Halliday, M. A. K. Language as Social Semiotic: The Social Interpretation of Language and Meaning [M]. London:
 TEL: +1-281-656-1158
Edward Arnold, 1978.
 Email: contact_us@escipub.com 4. Kress, G. ＆ Leeuwen，Theo Van. Reading Images: The Grammar of Visual Design [M]. London: Routledge，2006.
 Address: 10700 Richmond Ave,

Houston, TX, 77042 USA
Terms of Use/Privacy Policy/ Disclaimer/ Other Policies:
You agree that by using our site, you have read, understood, and agreed to be bound by all of our terms of use/privacy
policy/ disclaimer/ other policies (click here for details).
This work and its PDF file(s) are licensed under a Creative Commons Attribution-NonCommercial 4.0 International
License.
To use our site, you are required to accept our Terms of Use/Privacy Policy/Disclaimer/Other Policies. eSciPub LLC © 2016-2021 All Rights Reserved.
Open Journal of Social Sciences, 2021, 9, 472-489
https://www.scirp.org/journal/jss
ISSN Online: 2327-5960
ISSN Print: 2327-5952
The Commodification of Cancer: A Multimodal

Discourse Analysis of Cancer Websites in China
Yu Zhang
School of Foreign Studies, Beijing Information Science and Technology, Beijing, China
How to cite this paper: Zhang, Y. (2021). Abstract

The Commodification of Cancer: A Mul-
timodal Discourse Analysis of Cancer With the liberalization of market access, things such as health and healthcare
Websites in China. Open Journal of Social that were once not for trading now have attained an exchange value. The
Sciences, 9, 472-489. technological advancement may continue facilitating such commercialization
https://doi.org/10.4236/jss.2021.910034
and commodification. Against the socioeconomic backdrop, this study inves-
Received: May 4, 2021 tigates how the cause of preventing and fighting cancer is commodified by
Accepted: October 26, 2021 cancer websites in China. Adopting the multimodal discourse analysis ap-
Published: October 29, 2021
proach (Kress & van Leeuwen, 2001), this study identifies five strategies that
cancer websites adopt in the commodification process of cancer preven-
tion/management: 1) oversimplifying cancer prevention; 2) representing ad-
vertising discourse; 3) facilitating the use of cancer websites and their official
account on social media; 4) marginalizing the discourse of cancer-related
scientific knowledge; and 5) creating a sense of solidarity. The present study
suggests that the commodification of cancer is to some degree problematic.
This is because it emphasizes too much on personal efforts, particularly rely-
ing on the consumption of certain things, and it detracts from the importance
of the socio-political responsibility for the welfare of the public.
Keywords
Cancer Website, Commodification, Multimodal Discourse Analysis, Discursive
Strategy, China
1. Introduction
The Internet has become the most commonly used source of cancer information
(Mayer, 2006; National Cancer Institute, 2010), due to its accessibility and ease
of use (Castleton et al., 2011). Web-based health information plays the same im-
portant role as doctors for people seeking health information (Blanch-Hartigan &
Viswanath, 2015) and it tends to influence people’s decision making on main-
DOI: 10.4236/jss.2021.910034 Oct. 29, 2021 472 Open Journal of Social Sciences
Y. Zhang
taining healthy lifestyles (Koteyko, 2009). Given the importance of web-based in-
formation, previous studies examined cancer-related websites in terms of web-
site information or quality improvement and its usefulness (see Clayman et al.,
2008; Snyder et al., 2013; Warren et al., 2014), website design tenets and the ap-
plication of theoretical components (see Ivory & Megraw, 2005; Pendergrass et
al., 2001; Turchin & Lehman, 2000; Whitten et al., 2008; Zhang & von Dran,
2000), websites’ intervening role on patient-doctor relationships (see Haase et
al., 2016; Hou & Shim, 2010; Lewis et al., 2009; McMullan, 2006; Rider et al.,
2014), and the effects of multimodality on users’ perception and communicative
behaviour (see Bol et al., 2013; Nguyen et al., 2017; Spiegelhalter et al., 2011;
Stellamanns et al., 2017).
Few studies, however, have examined the multimodal discourse construction
of cancer websites (see Gibson et al., 2015; Gibson et al., 2016), particularly of
related commercial websites which are characterized with multimodal design
(Martinec & van Leeuwen 2009). Gibson et al. (2015, 2016) identified the mul-
timodal discursive strategies in constructing a “live well with breast cancer”
message and in representing women from various backgrounds on breast cancer
websites that are run by national organisaitons and at the state level in Australia.
Their study provides the insights that meaning is multimodal and that the
commodification of health can be reinforced through the construction of the
multimodal discourse on cancer-related websites. While related studies have
been conducted in western contexts, web-mediated cancer promotion in
non-western contexts is under-researched, especially in the context of China.
In China, cancer is a major health problem (Chen et al., 2016). It has been the
leading cause of death since 2010 (Chen et al., 2016) and about 27% of global
cancer deaths occurred in China (Ferlay et al., 2013). The number of new cancer
cases and deaths for all cancers in China has been increasing from 2002 to 2011
and there is still an increasing trend of cancer incidence (Chen et al., 2016). In
dealing with their health, more and more Chinese people use the Internet for
health and medical information seeking. The number of “netizens” who have
ever used the Internet for medical purposes was more than 0.6 billion, account-
ing for about 70% of the Internet users (BCG & TMI, 2020). Cancer-related
websites probably have an increasingly crucial role to play among Chinese
people who seek cancer-related information online. Catering to the public’s need
of being equipped with cancer-related knowledge, some cancer-related websites
have been established in China (see http://www.globecancer.com/). However,
studies investigating the communicative aspect of these websites are very scarce,
not to mention the exploration of the multimodal discourse construction on
those cancer websites.
The importance of studying cancer websites in the China-specific context may
lie in the contribution to discovering new sites and mechanisms of communica-
tive practice. This is because a fair understanding of discursive practices in spe-
cific contexts is viewed as a prerequisite for improving communication in
DOI: 10.4236/jss.2021.910034 473 Open Journal of Social Sciences

Y. Zhang
health-related settings (Wilce, 2009). While the globally hegemonic ideology of

“communicability” (Briggs, 2005) has privileged the “white” site discourse,
“non-white” discourses are needed for exploring the commonality and variation
in health promotion discourse. Such discourses are also beneficial to investigat-
ing the interactions between global circulation of ideologies and local traditions
that consist of certain relations (Wilce, 2009). Thus, to have a better under-
standing of the communicative value of the cancer websites in a non-western
context, this paper examines the multimodal discourse on cancer websites in
China by the approach of multimodal discourse analysis. In particular, it aims to
identify how the commodification process of cancer prevention and manage-
ment is carried out through the design of different semiotic modes.
2. Data and Methods

2.1. Analytical Framework
This study adopts Kress and van Leeuwen’s (2001) multimodal theory to ex-
amine discourse, design, production and distribution, to understand communi-
cation via cancer websites. Discourse is viewed as social practice which is rea-
lized by design(s) in a communication situation or context. It is socially con-
structed knowledge which is produced or developed in specific social contexts
and for distribution among social actors in the contexts. Designs are the uses of
semiotic resources which involve the selection of resources and the semiotic
modes, such as language and images. Production is the actual articulation of the
semiotic event or artefact (e.g., a website homepage). Distribution, or according
to Koteyko and Nerlich’s (2007) understanding, consumption, refers to “the
technical “re-cording” of semiotic products and events, for purposes of record-
ing and/or distribution” (Kress & van Leeuwen 2001: p. 21). As Kress and van
Leeuwen (2001) state, in the digital context, production and distribution are
technically integrated. For example, as soon as an image is produced or posted
online, it is distributed to web-users. Thus, in this study, the fourth domain, dis-
tribution, is not particularly addressed. In addition, in Kress and van Leeuwen’s
view on multimodal discourse, meaning is being made in those four domains
(i.e., discourse, design, production and distribution). The same meaning in a
particular context tends to be presented in different semiotic modes (see Gibson
et al., 2016). Following their multimodal theory, I propose two analytical catego-
ries for examining cancer website homepages: 1) the meaning that is conveyed
through the multimodal discourse design, and 2) the way that different semiotic
modes are combined to carry out this meaning.
2.2. Cancer Websites

This study first identified 17 cancer-related websites launched in China through
Baidu: the largest and most popular search engine in China, using the keywords:
cancer and tumour in simple Chinese characters. Then, four websites were se-
lected for analysis (see Appendix I). The website selection criteria were: 1) con-

Y. Zhang
taining comprehensive information about cancer and 2) being rich in images

(see Moran & Lee, 2013). Besides, considering the popularity or usability of a
website, websites that do not have any ranking information on Alexa.cn were ex-
cluded. 1 Those four websites are all commercial ones and deal with all cancers.
Only the homepage of each website was analyzed in this study. This is because
there are numerous links on a website which lead to different webpages. Besides,
a website’s value as a “portal” to other useful webpages is viewed as significantly
relying on the perceived value of the homepage (Weinberg, 2000). More specifi-
cally, all images, text, colour, and layout on the homepage of each selected web-
site were considered in the analysis. The homepages are publicly available for
viewing (i.e., they are open access, rather than being password-protected). This
means that conducting research with their content does not require informed
consent from the contributors (Demjén, 2016; Seale et al., 2010).
Screenshots of each website homepage were taken and the non-static images
on each homepage were archived separately in 2020. These screenshots were
archived in PDF format and labelled by number as well as the name of related
websites. The author did a closing reading on each screenshot and made de-
scriptive notes with reference to the use of images, text, colour, and layout (Gib-
son et al., 2015). Then, an initial thematic overview was summarized. After the
initial coding, the research data were analyzed by the two analytical categories
mentioned above. In this way, results were generated, which are presented in the
following section.
3. Findings
The present study identifies the phenomenon of commodification of cancer
prevention and management, a message conveyed through the multimodal de-
sign of the homepages of selected cancer websites. Commodification can be un-
derstood as “the production or conversion of ideas, knowledge, and objects into
items that are considered useful and/or have exchange value” (Lock & Nguyen,
2010: p. 206). Commodification of health is viewed as a process that addresses
health concerns which are taken advantage of by marketing strategies (Nichter,
1989). The phenomenon of commodification of health, medicine or healthcare
has been explored before (see Henderson & Petersen 2002; Nichter, 1989; Ton-
kens et al., 2013). This study shows that the commodification process is also seen
in the online provision of cancer information. This finding is in line with that of
previous studies (see Elliott, 2007; Zavestoski et al., 2004), which present the
“ongoing circulation” of the commodification of a specific cancer in daily sur-
roundings, especially breast cancer (Ehlers & Krupar, 2012: p. 8). It is noted that
breast cancer is commodified by pharmaceutical companies through construct-
ing breasts as problems that are “detached from females and their marginaliza-
tion within the social structure” (Zavestoski et al., 2004: p. 581). Besides, the
awareness of breast cancer is also commodified through cause-related marketi-
zation and the promotion of the pink color (Elliott, 2007). While prior studies

Y. Zhang
highlight the breast cancer-related commodification, this study points out the
extension of the application of commodification to cancer in general.
Nitcher (1989: p. 236) defined health commodification as the “tendency to
treat health as a state which one can obtain through the consumption of com-
modities”. Following this definition, the notion of commodifying cancer in this
study is referred to as the tendency to treat cancer prevention and/or manage-
ment as an ideal which can be obtained through the consumption of certain
commodities that are advertised or promoted on cancer websites. As Hanson
(1999) suggests commodification “is a matter of degree and context” (p. 269),
this paper does not intend to argue that cancer prevention/management as a
whole is commodified in Chinese society; instead, it only suggests that the
examined cancer websites create a sense of commodification of cancer. This
commodification is realized by multimodal discursive strategies, which are
discussed below.
3.1. Oversimplifying Cancer-Prevention/Management

Two types of health promotion discourse are displayed on the website home-
pages: cancer-related and non-cancer-related. The former involves information
on diet/food, exercises and lifestyles. Information in relation to diet and food
constitutes the main content of cancer-related health promotion discourse. The
health promotion discourse presents that (not) eating certain food or (not) hav-
ing a type of diet can help prevent cancers or a specific cancer. The rationale for
displaying this kind of discourse is that the food or diet recommended to pre-
vent cancer or cancer recurrence has particular nutrients that can improve im-
munity. However, the scientific evidence of the correlation between the preven-
tion of cancer and eating particular food is not actually presented, and the rec-
ommended food can be literally good for people to prevent any disease. The
message given to the viewers by such health promotion discourse in general
oversimplifies the cancer prevention cause.
While food/diet is promoted in relation to cancer prevention, it is notewor-
thy that some food promotion discourse is not related to cancer preven-
tion/management. The non-cancer related discourse, however, is positioned within
the “frame” in which there is cancer-related discourse. In this way, it may create
a false association between cancer prevention/management and the benefit of
eating certain food (see Figure 1).
In Figure 1, the heading says “oesophagael cancer” and below this heading
there is a food image. Below the food image it says “Eat more yellow food to im-
prove internal incretion”. Although this statement is not related to the heading,
the image is likely to be associated with the heading by viewers. This is because,
as discussed earlier in this section, the website homepages convey a message:
healthy eating can help prevent cancer. This message may implicate the viewers
that the food in the image of Figure 1 is also one of the healthy foods for cancer
prevention. In addition, the heading “oesophagael cancer” is a text link which

Y. Zhang
Figure 1. Oversimplifying cancer management.
leads to a webpage that talks about oesophagael cancer, but this link is not hig-
hlighted in some way to indicate that it is clickable. However, the text link below
the food image (i.e., “Eat more yellow food to improve internal incretion”) is
highlighted by an underline when the mouse pointer points to it, so it may be
more likely to be clicked by viewers. The food image itself is also a link. Both the
image link and the text link below the image lead to the same webpage, on which
the content does not mention anything related to oesophagael cancer. If viewers
do not click the link to read the content, they may not find out that the state-
ment “Eat more yellow food to improve internal incretion” has nothing to do
with oesophagael cancer. It may result in a potential risk of misleading viewers
into associating this statement with the prevention of oesophagael cancer. That
is, viewers are likely to believe that eating more yellow food can help improve
internal incretion and consequently prevent oesophagael cancer. Thus, the de-
sign of Figure 1 may create a false idea about the relationship between healthy
eating and cancer prevention, which in turn also oversimplifies the cause of
cancer prevention through giving much importance to the benefit of eating cer-
tain food.
In general, the health promotion discourse in these cancer websites mainly
focuses on promoting healthy eating. While such promotion may to some degree
oversimplify cancer prevention efforts, it may also perform the function of in-
structing viewers that they are able to prevent cancer on their own by having a
healthy diet. It implies that the cancer websites convey a message to their viewers
that the efforts of having a healthy diet can play a key role in cancer prevention.
In this way, those cancer websites imply that people who want to keep away
from getting cancer are expected to take some actions to achieve this goal. Such
an implication highlights the importance of self-care, which may be linked to the
advertising discourse that is discussed in the following section. That is, the con-
structed importance of personal efforts may motivate viewers to look for and re-

Y. Zhang
ly on more resources that are marketized on cancer websites, so as to achieve the

goal of avoiding getting cancer. It may in turn provide opportunities for cancer
websites to represent related advertisements that may be of interest to their
viewers. This is illustrated in the next section.
3.2. Representing Commercial Advertisement

This study finds that homepages of the cancer websites represent advertisements
of health products, genetic tests, and herbs that are used in traditional Chinese
medicines. Those advertising discourses may provide viewers a sense of control
and diverse choices which may “perpetuate consumerist ideals” (Gibson et al.,
2015: p. 282), thus facilitating the commodification of cancer preven-
tion/management.
In Figure 2, the health product Brolico is being advertised. This advertising
discourse is made up of different semiotic modes, such as colours, languages,
images, and layout. Those modes construct Brolico as a product that is good to
health. On the left of the image, behind the health product is broccoli which may
be the main component of the health product. The vegetable broccoli is likely to
be associated with the name of the product, Brolico, whose spelling is similar to
that of the word “broccoli”. Just as Nichter (1989: p. 239) states that “medicines
are associated with metamedical values through key symbols and images evoking
networks of semantic meaning”, the name of the health product (i.e., Brolico)
may intend to communicate the meaning that the health product is as healthy
and natural as the vegetable broccoli. This meaning is foregrounded by the
words, “Anti-cancer food”, that are positioned in brackets next to the product’s
name in the middle of Figure 2 [i.e., “Brolico (anti-cancer food)”]. The associa-
tion between broccoli and Brolico is also highlighted by the green colour used in
this image, considering the colour of broccoli is also green. Below “Brolico (an-
ti-cancer food)”, there is a statement with the green-color underpainting which
says “A natural product for enhancing human immunocompetence”. In this de-
finition of the health product, the words “natural product” is highlighted by the
bold black typeface, while other words in this statement are in white color.
Figure 2. The advertisement of a health product.

Y. Zhang
This emphasis may also associate the health product with the vegetable broccoli,
as broccoli is also made of natural ingredients. Below the definition statement,
the function of the product is presented, “Immunity enhancement/an adjuvant
for anti-tumor treatment”. Those words are in italics and the font color is
orange. The information before the forward slash repeats the information that
has been given in the definition statement (i.e., strengthen immune system), and
the information after the slash links to the cancer-related aspect of the health
product. Below this function description, the instruction shows to whom the
health product is applicable, “Applicable to patients who have got an operation,
and have been exposing to chemoradiotherapy and experiencing targeted thera-
py”. This instruction information is displayed in a small font and within a grey
frame. It seems to be separate from other content in the image, because the de-
sign of this instruction communicates the meaning that the targeted consumers
of the health product are cancer patients, which is not in line with the meaning
that is conveyed by other parts of the image. Based on the design of other con-
tent of the image, the product is constructed as a “food” and a “natural product”
which seems to be applicable to all people, not exclusive to cancer patients. This
design may indicate that the health product can be one of the choices for viewers
to eat healthily, since it is advertised as a “food” and “natural product”. Besides,
the pills on the bottom left corner of Figure 2 may imply that it is more conve-
nient to take the pills than eating broccoli since the unprocessed broccoli needs
to be cooked before eating. It thus provides another reason why viewers should
consume this health product in the cause of living healthily. Drawing from the
design of Figure 2 as a whole, the health product is constructed as being safe to
its users. The sense of safety may be the key message this cancer-related adver-
tisement discourse imparts. It can be said that the advertising discourse is de-
signed to show that cancer prevention/management can be achieved through the
consumption of the product. Hence, it commodifies the cause of the cancer pre-
vention/management.
Figure 3 shows an advertisement of the genetic testing technology that is de-
veloped by Caris Life Sciences company which is headquartered in the US. The
genetic testing technology advertisement foregrounds an image in which there
are presumably a father and a son who are smiling and looking in the same di-
rection. Their gazes do not directly address the audience or the viewers. These
are fixed upon the future (see Kress & van Leeuwen, 1996), which may indicate
hope. The gaze may perform the function of encouraging viewers to look toward
the same direction the father and the son are looking toward (see Kress & van
Leeuwen, 1996). It links to a statement “95% of cancer patients can get clinical
medication guides” which is in bold blue typeface and within the father-son im-
age frame. Below the image is another statement which is in bold green typeface,
saying “Caris genetic test”. The “Caris genetic test” statement is positioned in the
centre of Figure 3, which may imply the importance of this information. This
statement is given an important position, because it indicates the ownership

Y. Zhang
Figure 3. The advertisement of genetic testing.
of the genetic testing technology and probably highlights that it is owing to the
Caris genetic testing technology that “95% of cancer patients can get clinical
medication guides”. Below “Caris genetic test” is stated the benefit of the medi-
cation guides: “Generally prolong survival in cancer patients” which is in orange
color). The benefit of doing genetic testing can be associated with the happiness
of the father and the son, because this happiness image may imply that they are
satisfied with or happy about the genetic testing, or they are the beneficiaries of
the genetic testing technology. In a word, those different modes together con-
struct a helpful role of Caris genetic testing service in dealing with cancer. Thus,
it conveys a message to their viewers that cancer survivorship or management
can be handled by having a genetic test.
This study finds that an herb, ginseng, is heavily advertised on one of the can-
cer websites, both verbally and visually. The advertisement is in the form of text
links and an image link (see Figure 4).
In Figure 4, the content that is marked by red circles represents the discourse
that advertising the herb, ginseng. The image link and the two text-links close to
the image link can lead to the same webpage, on which the nature, species and
function of ginseng, and the ways to eat it are introduced. Interestingly, on this
webpage while it says ginseng is not suitable for cancer patients or patients with
tumour, it also says one of the components can be used to prevent cancer. In this
way, ginseng is constructed as an herb that is useful for preventing cancer but
not for coping with cancer. Each of the text links marked by red circles in the
main content section of Figure 4 leads to a different webpage. The webpages
mainly focus on advertising a health product, the main component of which is
ginseng, and these do not mention anything in relation to cancer. Taken together

Y. Zhang
Figure 4. The advertisement of an herb.
the advertising discourse, ginseng is represented as a commodity which can be

consumed to prevent cancer.
The advertising discourse capitalizes on the value of traditional Chinese me-
dicines to promote ginseng. That is, ginseng is highlighted as a valuable herb
being used in traditional Chinese medicines. In the circled image of Figure 4,
there are many elements in relation to ancient China, including the bamboo
book on the left side of this image, the Chinese calligraphy type of writing that is
on the middle of this image [i.e., from left to right: “The King of Plants”,
“TCMs” (Traditional Chinese medicines) and “Ginseng”, and one of the an-
cients holding a bamboo book on the right side. This image indexes to the tradi-
tional culture of China which is made use of by the marketing strategy. The
represented cultural value may shape the succession of health commodities that
are processed to meet the market economy (Nichter, 1989). In this way, the ad-
vertising ginseng discourse reinforces the consumption value of ginseng in terms
of preventing cancer.
3.3. Promoting Cancer Websites and their Account on Social Media

The selected cancer websites advertise their platforms through different semiotic
modes, which are positioned conspicuously on the homepage (see Figure 5 and
Figure 7). These websites also advertise their official account on widely used so-
cial networking sites in China such as WeChat (i.e., a Chinese multi-purpose
messaging, social media and mobile payment app) and Weibo (i.e., a Chinese

Y. Zhang
microblog) (see Figure 6).

In Figure 5, on the right of the navigation section presents an image advertis-
ing the website. In this image, the website’s name, its URL and a slogan are
framed within a shield-like blue line. The design of the shield-like line may serve
the function of indicating that the website can protect the users from cancer.
Within the shield-like frame, the slogan says “In front of cancer we are no longer
helpless”. The use of “we” here may create a sense of solidarity which is dis-
cussed in detail in the following section. This inclusive pronoun may also build a
sense of intimacy, which can link to the people image next to the shield-like blue
line image. The people in this image are presumably a family, including a moth-
er, a father and their daughter. This family image may not detract from the focus
on advertising the website. This is because the gaze of the father is fixed on the
left side of the image where there is the website’s name and slogan, and it may
guide viewers to look toward the same direction (Kress & van Leeuwen 1996).
The promotion of the website may implicate that the cancer website has an im-
portant role to play in the cause of preventing/fighting cancer. Hence, it may fa-
cilitate the use of the website. Such promotion thus reinforces the commodifica-
tion of cancer through highlighting the importance of using the cancer website
to deal with cancer.
In Figure 6, the cancer website’s official account of WeChat and Weibo are
promoted. This advertisement informs the viewers that the cancer website’s
“service” is extended to social networking sites, on which viewers can participate
in social networking activities with other users by such as contributing to the
Figure 5. Promotion of a cancer website.

Y. Zhang
commenting area or group chatting. The design of this advertisement is ac-

tion-oriented. That is, at the bottom right corner of Figure 6 there is a QR code
which has the affordance of scanning. This QR code may facilitate the scanning
action of the viewers, in turn increasing the usage of the mobile app-based busi-
ness of the website. This promotion may forward the commodification of cancer
prevention/management, considering that social networking sites are able to fa-
cilitate social interaction of health consumers and consequently contribute to the
increasement of trust and intention to consume (Hajli, 2014).
3.4. Marginalizing the Discourse of Cancer-Related Scientific

Knowledge
Cancer-related scientific knowledge is presented on the four cancer websites, in-
cluding the definition and/or features of different types of cancer disease, the
difference between tumour and cancer, the symptoms of a specific cancer, the
methods/mechanisms of cancer treatment, and the correction of misleading in-
formation that was disseminated among the public. However, the links led to
such kind of discourses are marginalized on the website homepages. That is, they
are displayed in a form of text links which are in a small font and usually at the
top or top left of the homepage (see Figure 7). The marginalization may be de-
signed with the purpose of saving room for displaying other information.
In Figure 7, the circled text links, which present the name of different types of
cancer, occupy much smaller room than does the image next to them on the
right side. This image advertises the website per se, on which it says “Authorita-
tive international medical platform” in a much larger font. The marginalization
strategy can be viewed as being indirectly associated with the commodification
of cancer prevention/management. This is because it helps to create enough
room to highlight discourses that can link to the commodification process, dis-
courses for advertising the cancer website, as discussed above.
Figure 6. Promotion of the social media of a cancer website.

Y. Zhang
3.5. Building Solidarity

The sense of solidarity is created through the design of language and images. As
shown in the slogan in Figure 5 (i.e., “In front of cancer we are no longer help-
less”), “we” communicates the meaning that the websites align themselves with
their viewers or users. This discursive construction also positions the viewers as
a member in the community that deals with cancer. The construction of solidar-
ity is also seen in an image (see Figure 8).
In Figure 8, the image is a link that leads to a webpage, on which there is an
advertisement for medicines that treat cancer. On the right side of the image,
there is a person with a skull head. This skull head image may communicate a
metaphorical sense, that is, the person symbolizes death. On the left side of the
image, there is a group of people. Drawing from the mouth-muffle that each
Figure 7. Marginalizing the discourse of cancer-related scientific knowledge.
Figure 8. Building solidarity.

Y. Zhang
person wears or holds, they are presumably patients. It seems that their gazes
directly address viewers, which may engage viewers to take certain actions
(Kress & van Leeuwen, 1996). One of the possible actions is clicking the image
link, which consequently leads the viewers to the webpage advertising certain
medicines for dealing with cancer.
The rationale for constructing the sense of solidarity may lie in the collective
anxiety or fear toward the possibility of getting cancer or dying of cancer. It is
suggested that fear is “significantly a key element” in the use of online health
sites or health information seeking (Segal, 2009: p. 363). People’s use of cancer
websites may be also driven by fear. The construction of solidarity may impart a
message to viewers that their fear is shared by others, that the cancer website can
provide a place where people get together to deal with their fear, and that unity
can triumph over fear. In this way, the multimodal design which communicates
solidarity may increase users’ dependence on the cancer website. Consequently,
it may contribute to the commodification of cancer prevention/management,
since the use of a cancer website may potentially increase the consumption of the
promoted commodities as well as the cancer website per se.
4. Discussion and Conclusion

This study shows that the selected cancer websites highlight the notion that can-
cer can be prevented or dealt with through the consumption of certain com-
modities. Cancer websites are seemingly expected to facilitate viewers acquiring
cancer-related knowledge. However, the main focus of the cancer websites in
question is advertising or promoting commodities (including the websites per se
and their account on social media) that can be connected to cancer preven-
tion/management. While cancer-related scientific knowledge is marginalized,
the advertising discourses are highlighted and occupy a relatively large part of
the homepage of each selected cancer website. The overall design of those ho-
mepages performs the function of promoting self-care to prevent cancer and
encouraging viewers to turn to the cancer websites and/or related social net-
working sites for guides or support. This may contribute to the increase of con-
sumption of the websites/their social networking sites, and of the advertised
commodities on those platforms. The multimodal discourse on the homepages is
also produced in a way of increasing the public’s choices, so as to “help” them
find a way to prevent/deal with cancer. In this way, it may facilitate the process
of commodification of cancer prevention and management, since “choice” is the
“language of marketplace” (Rothman, 2008: p. 2). The design of cancer website
homepages may give too much importance to the preventive or controllable role
of the advertised commodities in the cause of cancer prevention/management.
This emphasis may foster “a false sense of security” (Nichter, 1989) among viewers
who follow the “instructions” of those cancer websites with the expectation of
avoiding getting cancer. In addition, when cancer prevention/management is con-
structed as a commodity that can be attainable through personal efforts, there is

Y. Zhang
a risk of depreciating socio-political responsibility for the conditions of public

health, in particular at the time of environmental deterioration and rapid urba-
nization and industrialization (see Nichter, 1989).
The cancer websites in question play upon the public’s eagerness or expecta-
tion of cancer prevention/survivorship in order to get benefits. These link the
cause of preventing/fighting cancer to the consumption of certain things. Broc-
coli that once had its value as a food now is given the exchange value of pre-
venting cancer as an ingredient in health products. Ginseng that once was un-
derstood as part of a plant now is promoted as a valuable ingredient in tradi-
tional Chinese medicines that enhance immune system, which in turn is relied
on to prevent cancer. The genetic testing that was originated in biotechnology
now has been commodified as a product for dealing with cancer or predicting
the cancer risk. Websites and social networking sites that once were for informa-
tion-seeking and peer interaction now have been constructed as an assistant
which people can rely on in order to prevent/fight cancer. The broccoli, ginseng,
genetic testing, and cancer websites/their social networking sites are designed to
be associated with cancer prevention/management. When people consume those
with the purpose of preventing or coping with cancer, they contribute to the
commodification process of cancer prevention/management. In this commodi-
fication process, the major beneficiaries may be the owners of those cancer web-
sites, the pharmaceutical industry, and biotechnology companies. Hence, health
promotion on those cancer websites may not be for the sake of the public’s well-
being, but for those beneficiaries to get more profit in a commercialized society.
The word “cancer” has gained a value that can be exchanged and the potential
for the accumulation of wealth. This is because the cancer disease has its eco-
nomic value. This economic value lies in the controllability of such disease, that
is, cancer can be prevented and coped with; it also lies in the shift from the pro-
duction of things for controlling the disease to its distribution or consumption.
Besides being endowed with the economic value, cancer is also being con-
structed as an opportunity, rather than an illness for people to make efforts to
maintain health or keep away from cancer (Gibson et al., 2015). Such a con-
struction positions users of cancer websites as protentional customers who can
afford to catch the opportunity as well as having the financial capital to do so in
the cause of cancer prevention and management. In this way, while the commo-
dification privileges those who can pay for the opportunity, it marginalises those
who are not able to afford it, or puts financial pressure on them. For future re-
search it is vital to investigate what effects of the commodification of cancer
prevention/management or the manipulation of cancer commercialization may
have on the public, particularly the cancer patients or their family.
The author declares no conflicts of interest regarding the publication of this
paper.

Y. Zhang
References
BCG & TMI (2020). 2020 Digital Healthcare Insight Report.
http://www.ciaba.cn/ueditor/php/upload/file/20200804/1596504415139646.pdf
Blanch-Hartigan, D., & Kasisomayajula, V. (2015). Socioeconomic and Sociodemograph-
ic Predictors of Cancer-Related Information Sources Used by Cancer Survivors. Jour-
nal of Health Communication 20, 204-210.
https://doi.org/10.1080/10810730.2014.921742
Bol, N., Ellen, M. A. S., Mattijs, R., Jacobus, A. B., Hanneke, C. J. M., Eugène, F. L., & Ju-
lia, C. M. (2013). Do Videos Improve Website Satisfaction and Recall of Online Can-
cer-Related Information in Older Lung Cancer Patients? Patient Education and Coun-
seling 92, 404-412. https://doi.org/10.1016/j.pec.2013.06.004
Castleton, K., Thomas, F., Andrea, W. G., Muhammad, A. W., Donna, B. J., Lisa, K.,
Feng, G. and Ramaswamy, G. (2011). A Survey of Internet Utilization among Patients
with Cancer. Supportive Care in Cancer, 19, 1183-1190.
https://doi.org/10.1007/s00520-010-0935-5
Chen, W., Rongshou, Z., Peter, D. B., Siwei, Z., Hongmei, Z., Freddie, B., Ahmedin, J.,
Xue, Q. Y., & Jie, H. (2016). Cancer Statistics in China, 2015. CA: A Cancer Journal for
Clinicians, 66, 115-132. https://doi.org/10.3322/caac.21338
Clayman, M. L., Eric, W. B., & Gregory, M. (2008) The Use of Patient and Provider Pers-
pectives to Develop a Patient-Oriented Website for Women Diagnosed with Breast
Cancer. Patient Education and Counseling, 72, 429-435.
https://doi.org/10.1016/j.pec.2008.05.032
Ehlers, N., & Shiloh, K. (2012). The Body in Breast Cancer. Social Semiotics, 22, 1-11.
https://doi.org/10.1080/10350330.2012.640060
Elliott, C. (2007). Pink!: Community, Contestation and the Colour of Breast Cancer. Ca-
nadian Journal of Communication, 32, 521-536.
https://doi.org/10.22230/cjc.2007v32n3a1762
Ferlay, J., Isabelle, S., Morten, E., Rajesh, D., Sultan, E., Colin, M., Marise, R., Donald, M.
P., David, F., & Freddie, B. (2013). GLOBOCAN 2012 v1. 0, Cancer Incidence and
Mortality Worldwide: IARC Cancer Base No. 11. Lyon, France: International Agency
for Research on Cancer 2016. http://globocan.iarc.fr/
Gibson, A. F., Christina, L., & Shona, C. (2015) Reading between the Lines: Applying
Multimodal Critical Discourse Analysis to Online Constructions of Breast Cancer. Qu-
alitative Research in Psychology, 12, 272-286.
https://doi.org/10.1080/14780887.2015.1008905
Gibson, A. F., Christina, L., & Shona, C. (2016). Representations of Women on Australian
Breast Cancer Websites: Cultural “Inclusivity” and Marginalisation. Journal of Sociol-
ogy, 52, 433-452. https://doi.org/10.1177/1440783314562418
Haase, K. R., Roanne, T., & Wendy, G. (2016) Internet Cancer Information Use by Newly
Diagnosed Individuals and Interactions with the Health System: A Mixed Methods
Study Protocol. International Journal of Qualitative Methods, 15, 1-11.
https://doi.org/10.1177/1609406916650902
Hajli, M. N. (2014). A Study of the Impact of Social Media on Consumers. International
Journal of Market Research, 56, 387-404. https://doi.org/10.2501/IJMR-2014-025
Hanson, M. J. (1999). Biotechnology and Commodification within Health Care. The
Journal of Medicine and Philosophy, 24, 267-287.
https://doi.org/10.1076/jmep.24.3.267.2522
Henderson, S., & Alan, R. P. (Eds.) (2002). Consuming Health: The Commodification of

Y. Zhang
Health Care. Routledge. https://doi.org/10.1046/j.1365-2524.2002.03843.x

Hou, J., & Minsun, S. (2010). The Role of Provider-Patient Communication and Trust in
Online Sources in Internet Use for Health-Related Activities. Journal of Health Com-
munication, 15, 186-199. https://doi.org/10.1080/10810730.2010.522691
Ivory, M. Y., & Rodrick, M. (2005). Evolution of Web Site Design Patterns. ACM Trans-
actions on Information Systems (TOIS), 23, 463-497.
https://doi.org/10.1145/1095872.1095876
Koteyko, N. (2009). I Am a Very Happy, Lucky Lady and I Am Full of Vitality! Analysis
of Promotional Strategies on the Websites of Probiotic Yoghurt Producers. Critical
Discourse Studies, 6, 111-125. https://doi.org/10.1080/17405900902749973
Koteyko, N., & Nerlich, B. (2007). Multimodal Discourse Analysis of Probiotic Web Ad-
vertising. The International Journal of Language Society and Culture, 23, 20-31.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.544.2585andrep=rep1andtyp
e=pdf
Kress, G. R. & Theo, V. L. (2001). Multimodal Discourse: The Modes and Media of Con-
temporary Communication. Bloomsbury.
Kress, G. R., & Theo, V. L. (1996). Reading Images: The Grammar of Visual Design.
Routledge.
Lewis, N., Stacy, W. G., Derek, R. F., & Robert, C. H. (2009). Examining Cross-Source
Engagement with Cancer-Related Information and Its Impact on Doctor-Patient Rela-
tions. Health Communication, 24, 723-734.
https://doi.org/10.1080/10410230903264030
Lock, M., & Vinh-Kim, N. (2010). An Anthropology of Biomedicine. John Wiley and
Sons.
Martinec, R., & Theo, V. L. (2009). The Language of New Media Design: Theory and
Practice. Routledge.
Mayer, D. K. (2006). Information Seeking, Screening and Health Behaviors of Cancer
Survivors. Unpublished Doctoral Dissertation, The University of Utah.
McMullan, M. (2006). Patients Using the Internet to Obtain Health Information: How This
Affects the Patient-Health Professional Relationship. Patient Education and Counseling,
63, 24-28. https://doi.org/10.1016/j.pec.2005.10.006
Moran, C., & Christina, L. (2013). Selling Genital Cosmetic Surgery to Healthy Women:
A Multimodal Discourse Analysis of Australian Surgical Websites. Critical Discourse
Studies, 10, 373-391. https://doi.org/10.1080/17405904.2013.813772
National Cancer Institute (2010). Trends in Cancer Information Seeking (HINTS Briefs
16). https://hints.cancer.gov/docs/Briefs/HINTS_Brief_16.pdf
Nguyen, M. H., Julia, C. M. W., Nadine, B., Eugène, F. L., Kristien, T., Anthony, V., & El-
len, S. (2017) Tailoring the Mode of Information Presentation: Effects on Younger and
Older Adults’ Attention and Recall of Online Information. Human Communication
Research, 43, 102-126. https://doi.org/10.1111/hcre.12097
Nichter, M. (1989). Pharmaceuticals, Health Commodification and Social Relations: Ra-
mifications for Primary Health Care. In M. Nichter & M. Nichter (Eds.), Anthropology
and International Health (pp. 233–277). Kluwer Academic Publishers.
https://doi.org/10.1007/978-94-009-2231-0_10
Pendergrass, S., Margaret, A. N., & David, H. (2001). Design and Evaluation of an Inter-
net Site to Educate Women with Disabilities on Reproductive Health Care. Sexuality
and Disability, 19, 71-83. https://link.springer.com/article/10.1023/A:1010720921585
https://doi.org/10.1023/A:1010720921585

Y. Zhang
Rider, T., Malik, M., & Timothy, C. (2014). Haematology Patients and the Internet: The
Use of On-Line Health Information and the Impact on the Patient-Doctor Relation-
ship. Patient Education and Counselling, 97, 223-238.
Rothman, B. K. (2008). New Breast Milk in Old Bottles. International Breastfeeding
Journal, 3, e1-e4. https://doi.org/10.1186/1746-4358-3-9
https://internationalbreastfeedingjournal.biomedcentral.com/articles/10.1186/1746-435
8-3-9
Segal, J. Z. (2009). Internet Health and the 21st-Century Patient: A Rhetorical View.
Written Communication, 26, 351-369. https://doi.org/10.1177/0741088309342362
Snyder, C. F., Amanda, L. B., Antonio, C. W., Michael, A. C., Joseph, M. H., Albert, W.
W., & Patient Viewpoint Scientific Advisory Board (2013). Feasibility and Value of Pa-
tient Viewpoint: A Web System for Patient-Reported Outcomes Assessment in Clinical
Practice. Psycho-Oncology 22, 895-901. https://doi.org/10.1002/pon.3087
Spiegelhalter, D., Mike, P., & Ian, S. (2011). Visualizing Uncertainty about the Future.
Science, 333, 1393-1400. https://doi.org/10.1126/science.1191181
Stellamanns, J., Dana, R., Keshav, D., Zita, S., & Jutta, H. (2017). Visualizing Risks in
Cancer Communication: A Systematic Review of Computer-Supported Visual Aids.
Patient Education and Counseling, 100, 1421-1431.
Tonkens, E., Christian, B., Nienke, V. S., & Daniël, V. H. (2013). Pretenders and Perfor-
mers: Professional Responses to the Commodification of Health Care. Social Theory &
Health, 11, 368-387. https://doi.org/10.1057/sth.2013.5
Turchin, A., & Lehmann, A. T. (2000). Active Learning Centre: Design and Evaluation of
an Educational World Wide Web Site. Medical Informatics and the Internet in Medi-
cine, 25, 195-206. https://doi.org/10.1080/146392300750019190
Warren, E., Katharine, F., Michela, T., Martin, M., & Cécile, K. (2014). Do Can-
cer-Specific Websites Meet Patient’s Information Needs? Patient Education and Coun-
seling, 95, 126-136. https://doi.org/10.1016/j.pec.2013.12.013
Weinberg, B. D. (2000). Don’t Keep Your Internet Customers Waiting Too Long at the
(Virtual) Front Door. Journal of Interactive Marketing, 14, 30-39.
https://doi.org/10.1002/(SICI)1520-6653(200024)14:1<30::AID-DIR3>3.0.CO;2-M
Whitten, P., Sandi, S., Samantha, M., & Carolyn, L. (2008). Communication Assessment
of the Most Frequented Breast Cancer Websites: Evaluation of Design and Theoretical
Criteria. Journal of Computer-Mediated Communication, 13, 880-911.
https://doi.org/10.1111/j.1083-6101.2008.00423.x
Wilce, J. M. (2009). Medical Discourse. Annual Review of Anthropology, 38, 199-215.
https://doi.org/10.1146/annurev-anthro-091908-164450
Zavestoski, S., Sabrina, M., & Phil, B. (2004). Gender, Embodiment and Disease: Envi-
ronmental Breast Cancer Activists’ Challenges to Science, the Biomedical Model and
Policy. Science as Culture, 13, 563-586. https://doi.org/10.1080/0950543042000311869
Zhang, P., & Gisela, M. (2000). Satisfiers and Dissatisfiers: A Two-Factor Model for Web-
site Design and Evaluation. Journal of the American Society for Information Science,
51, 1253-1268.
https://doi.org/10.1002/1097-4571(2000)9999:9999<::AID-ASI1039>3.0.CO;2-O

Login
Home Articles Journals Books News About Submit
 Home > Journals > Citations Search Title, Keywords, Author, etc.
Journals Menu JSS > Vol.9 No.10, October 2021
 Articles
Open Journal of Social Sciences
 Archive
Volume 9, Issue 10 (October 2021)
 Indexing
ISSN Print: 2327-5952 ISSN Online: 2327-5960
 Aims & Scope Google-based Impact Factor: 0.85 Citations
 Editorial Board
 For Authors
The Commodification of Cancer: A Multimodal Discourse Analysis of Cancer
 Publication Fees
Websites in China
HTML XML Download as PDF (Size: 1609KB) PP. 472-489
Related Articles
DOI: 10.4236/jss.2021.910034 22 Downloads 99 Views
A Multimodal Discourse Analysis
of Second Language Instruction Author(s)
Videos
Yu Zhang
Intraperitoneal Chemotherapy
as a Multimodal Treatment for Affiliation(s)
Gastric Cancer Patients with School of Foreign Studies, Beijing Information Science and Technology, Beijing, China.
Peritoneal Metastasis
Cancer mortality in Inner ABSTRACT
Mongolia of China, 2008-2010 With the liberalization of market access, things such as health and healthcare that were once not for trading now have
Categories of Hegemonic attained an exchange value. The technological advancement may continue facilitating such commercialization and
Discourse in Contemporary commodification. Against the socioeconomic backdrop, this study investigates how the cause of preventing and fighting
China cancer is commodified by cancer websites in China. Adopting the multimodal discourse analysis approach (Kress & van
A Critical Discourse Analysis of
Leeuwen, 2001), this study identifies five strategies that cancer websites adopt in the commodification process of cancer
Reports about China on the
prevention/management: 1) oversimplifying cancer prevention; 2) representing advertising discourse; 3) facilitating the use
COVID-19 Pandemic in The New
of cancer websites and their official account on social media; 4) marginalizing the discourse of cancer-related scientific
York Times
knowledge; and 5) creating a sense of solidarity. The present study suggests that the commodification of cancer is to
some degree problematic. This is because it emphasizes too much on personal efforts, particularly relying on the
Open Special Issues
consumption of certain things, and it detracts from the importance of the socio-political responsibility for the welfare of the
Published Special Issues public.
Special Issues Guideline KEYWORDS
Cancer Website, Commodification, Multimodal Discourse Analysis, Discursive Strategy, China
Share and Cite:

E-Mail Alert
JSS Subscription Zhang, Y. (2021) The Commodification of Cancer: A Multimodal Discourse Analysis of Cancer Websites in China. Open
Journal of Social Sciences, 9, 472-489. doi: 10.4236/jss.2021.910034.

Publication Ethics & OA Statement
Cited by
Frequently Asked Questions
No relevant information.
Recommend to Peers
Recommend to Library
Contact us
Sponsors, Associates,
and Links
Creative Education
Modern Economy
Advances in Applied Sociology
Advances in Historical Studies
Chinese Studies
Follow SCIRP
Contact us
customer@scirp.org
+86 18163351462(WhatsApp)
1655362766
Paper Publishing WeChat
Copyright © 2021 by authors and Scientific Research Publishing Inc.
This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.
Free SCIRP Newsletters Home About SCIRP Service Policies

Add your e-mail address to receive Journals A-Z Publication Fees Manuscript Tracking System Open Access
free newsletters from SCIRP.
Subject For Authors Subscription Publication Ethics
E-mail address
Books Peer-Review Issues Translation & Proofreading Preservation
Select Journal
Sitemap Special Issues FAQ Retraction
Subscribe
Contact Us News Volume & Issue Privacy Policy
Copyright © 2006-2021 Scientific Research Publishing Inc. All Rights Reserved.

Open Linguistics 2020; 6: 267–283
Research Article
Hesham Suleiman Alyousef*

A multimodal discourse analysis of English dentistry
texts written by Saudi undergraduate students: A study
of theme and information structure
https://doi.org/10.1515/opli-2020-0103
received September 12, 2019; accepted March 28, 2020
Abstract: The study of multimodality in discourse reveals the way writers articulate their intended
meanings and intentions. Systemic functional analyses of oral biology discourse have been limited to few
studies; yet, no published study has investigated multimodal textual features. This qualitative study
explored and analyzed the multimodal textual features in undergraduate dentistry texts. The systemic
functional multimodal discourse analysis (SF-MDA) is framed by Halliday’s (Halliday, M. A. K. 2014.
Introduction to Functional Grammar. Revised by Christian M. I. M. Matthiessen. 4th ed. London/New York:
Taylor and Francis) linguistic tools for the analysis of Theme and Kress and van Leeuwen’s (Kress,
Gunther, and Theo van Leeuwen. 2006. Reading Images: The Grammar of Visual Design. London:
Routledge) framework for the analysis of visual designs. Oral biology discourse intertwines two thematic
progression patterns: constant and linear. Although a split-rheme pattern was minimally employed,
disciplinary-specific functions of this pattern emerged. The SF-MDA of the composition of information in
oral biology pictures extends Kress and van Leeuwen’s functional interpretations of the meaning-making
resources of visual artifacts. Finally, the pedagogical implications for science tutors and for undergraduate
nonnative science students are presented.
Keywords: oral biology discourse, dentistry discourse, thematic progression, composition of information
value, systemic functional linguistic, systemic functional multimodal discourse analysis
1 Introduction
The study of multimodality in discourse has increased over the past two decades because it reveals the
way writers articulate their intended meanings and intentions. Multimodality determines “the
combination of different semiotic resources, or modes, in texts and communicative events, such as still
and moving image, speech, writing, layout, gesture, and/or proxemics” (Adami 2016, 451). Discourse
investigations of multimodal biology discourse are limited to a few studies (Hannus and Hyönä 1999, Kress
2003, Guo 2004, Baldry and Thibault 2005, Jaipal, 2010); yet, no published study has explored and
analyzed the way multimodal semiotic forms are organized and presented in this discipline. As a biology
discourse invariably employs pictorial representations (photographs and drawings), it suited the aim of
this study. The theme and information structure contribute significantly to the development of cohesive
and coherent multimodal texts. Halliday’s (1978, 2014) social semiotic approach to language, systemic
functional linguistics (SFL), and Kress and van Leeuwen’s (2006) framework for the analysis of the
grammar of visual design fit with the aim of the present study because they set out the explanation of how
students make meaning of language and the various multimodal semiotic resources.

* Corresponding author: Hesham Suleiman Alyousef, Applied Linguistics, Department of English Language and Literature,
Faculty of Arts, King Saud University, Riyadh, Saudi Arabia, e-mail: hesham@ksu.edu.sa, tel: +966 5 5300 0412
Open Access. © 2020 Hesham Suleiman Alyousef, published by De Gruyter. This work is licensed under the Creative Commons
Attribution 4.0 Public License.
268  Hesham Suleiman Alyousef
The present study conducted a systemic functional multimodal discourse analysis (SF-MDA) to
explore the realization of the textual metafunction of the different semiotic resources in a key topic in the
Oral Biology course, i.e., developmental abnormalities: defects of the face and oral cavity.
The study is pertinent, since it is the first to explore the way Saudi undergraduate dentistry students
produce cohesive and coherent multimodal texts. It is also of interest as the number of Saudi English as a
foreign language (EFL) students enrolled in a dentistry undergraduate program in Saudi Arabia has
increased dramatically during the past 10 years. In 2016, for example, the number of Saudi students
enrolled in the bachelor of science dentistry program increased by 22.34% from 9,883 to 12,091 (Saudi
Ministry of Education 2016). As the Saudi government intends to localize the dental profession, which has
thus far been mostly occupied by foreign expatriates, it is not surprising to see many Saudi students
attracted to this major. The study may provide insights for science tutors and undergraduate EFL/English
as a second language (ESL) science students.
2 Literature review
Investigations of the language of scientific discourse aim to reveal the creative effects of language through
systemic linguistic analyses and descriptions of its peculiar features. Halliday (2004), for example,
investigated scientific English throughout history and showed how it uses grammatical nominalizations
and favors relational clauses (attributive for assigning properties and identifying for definitions), material
or mental. Discourse studies of the textual multimodal features in tertiary contexts have included the
disciplinary fields of business (Alyousef 2013, 2015a, 2016, 2017), science and computing (Drury et al. 2006,
Jones 2006), mathematics (O’Halloran 1998, 1999b, 2005, 2008), journalism and media (Hawes 2015),
history (North 2005), and nursing (Okawa 2008). The majority of research focused on primary or secondary
school contexts (Hsu and Yang 2007, Jaipal 2010, Korani 2012). For example, Hsu and Yang (2007)
explored the effect of science text and image integration on Grade 9 students’ reading comprehension.
A control–experimental design with pre- and posttests, and semistructured interviews was used. The
students were randomly assigned into a focus group (n = 69) that read the traditional textbook and an
experimental group (n = 63) that read an SFL-based textbook which had been created by the researchers.
The two texts contained similar scientific concepts but had major differences related to structure of print,
image modality, and salience and the interaction between the two forms. The quantitative findings
revealed that the experimental group demonstrated better reading comprehension than did the control
group. On the other hand, the interview results indicated that an SFL image has better efficacy. This
indicated that reading comprehension is facilitated when images and print are integrated. Hsu and Yang
(2007) concluded that SFL could serve as a resource for developing and designing scientific textbooks.
Discourse investigations of multimodal biology texts are confined to a few studies (Hannus and Hyönä
1999, Kress 2003, Guo 2004, Baldry and Thibault 2005, Jaipal 2010). For example, Jaipal (2010) developed
a multimodal SFL-based framework for science classroom discourse in order to investigate its potential to
provide insights into how a Grade 11 biology teacher selected, sequenced, and modified semiotic
modalities to support students in constructing a scientific meaning for the concept “chemosynthesis.” The
framework was also based on Lemke’s (2002) semiotics typology of meaning for discourse analysis. Data
were collected through classroom observations, semistructured interviews with students, student artifacts,
and ongoing, informal interviews with the teacher. The findings revealed the usefulness of this framework
for understanding teachers’ explanations and for supporting, scaffolding, extending, and reinforcing
meaning making. Jaipal (2010), however, did not investigate the construal of “theme” and information
value in biology discourse; instead, the researcher identified aspects related to genre, such as the structure
and sequencing of a topic, lessons, concepts, modalities, and words. In visual diagrams, typographical
(e.g., images, arrows) and compositional tools (e.g., texture, color) were identified. Hannus and Hyönä
(1999) investigated the use of pictures in biology elementary-level textbooks using the eye-tracking
methodology to trace the students’ trajectory with precision. The findings showed that high-ability
An MDA of English dentistry texts written by Saudi undergraduate students  269
children performed better at integrating the relevant passages of text and pictures, which was required to
answer the more demanding comprehension questions about the textbook passages. Guo (2004) studied
the use of textbook articles on the molecular biology of the cell by second year bachelor of science majors.
He proposed social semiotic frameworks for analyzing two common types of visual display in biology texts
that interact with each other to make meaning: schematic drawings and statistical graphs. Whereas the
ideational (or representational) meaning of the former is expressed through the topological aspects of
shape, color, size, spatial relation, and action, the latter is understood through the relative numerical
relationships between two sets of variables, or through the distribution of an attribute of some entities
among a sample or a population. Guo (2004), however, did not investigate the construal of thematic
progression (TP) patterns in biology discourse. It is, therefore, pertinent to explore and analyze the
multimodal textual features in undergraduate dentistry students’ responses to the assignments.
To sum up, systemic functional investigations of biology discourse are limited to a few studies, and no
published study has explored and analyzed the multimodal textual features. The present study conducted
an SF-MDA of the textual features of a key topic in multimodal oral biology texts. What follows is a brief
description of the data and method of analysis.
3 Theoretical framework: SF-MDA

The nomenclature SF-MDA has been used since oral biology texts started to include pictorial
representations. The SF-MDA was framed by Halliday’s (2014) SFL social semiotic approach to the
analysis of theme/rheme and TP patterns and Kress and van Leeuwen’s (2006) approach to the grammar of
visual design in order to explore the salient textual features of oral biology texts and the ways in which
undergraduate dentistry students construct cohesive and coherent multimodal texts.
Since Halliday’s (2014) SFL approach considers the functions of language in social interaction, it is
relevant to the context of the present study. SFL is concerned with the interpretation of texts in relation to
the context in which these texts are produced and received. It conceptualizes context into three language
registers: field of discourse, which is concerned with the experiential meanings being discussed; tenor of
discourse, which focuses on the construction of social relations and roles; and mode of discourse, which is
concerned with how semiotic forms are organized and presented. The latter register is represented in texts
by our use of thematic and cohesive structures, which, with the aid of the former two registers, organize
the informational structure of a text into a coherent whole. Due to spatial limitations, I have only
investigated theme and information structure in oral biology texts, since these linguistic resources play an
important role in the organization and cohesive flow and coherence of a text, as is shown next.
Halliday (2014, 64) defines theme as the “point of departure for the message; it is that which locates
and orients the clause within its context.” Theme defines the topic of the clause, while rheme constitutes
the remaining elements of the message that develop the theme by providing additional information. For
example, the phrase “an anomaly” in the sentence “an anomaly is usually something that is abnormal at
birth” is a topical (or experiential) theme, while the rest is the rheme. A sentence can also include other
two theme types which are optional: textual (e.g., furthermore, therefore) and interpersonal (e.g.,
probably, must). Theme conflates with the subject in declarative clauses (e.g., John got up early); the finite
in interrogative clauses (Did [theme] John get up early?); the predicator in imperative clauses (Leave
[Theme] the book here?); or WH in WH interrogatives (Where [Theme] did John go?). The finite carries the
selections for number, tense, and polarity (yes or no). The predicator tells us what process was actually
happening. In such cases, theme is unmarked because this is “the most typical/usual” (Eggins 2004, 318)
choice, whereas it is marked when it conflates with a prepositional or an adverbial group/phrase to
provide circumstantial details about an activity.
Whereas given refers to “what is already known or predictable,” new information, as its name
suggests, refers to “what is new or unpredictable” (Halliday, 2014, 89). Rheme, however, does not
necessarily conflate with new since it is marked off by pitch or tonic prominence. The system of theme/
rheme is typically conflated with the information functions of given/new. It should be pointed out that
theme does not necessarily conflate with what is being discussed (i.e., the subject or given information).
A subject is excluded from being the theme of a clause in the case of a marked topical theme which has a
mood function other than subject, such as the theme “For an older child or adult, [Theme: Topical]” in
“For an older child or adult, [Theme: Topical] tongue-tie can make it hard to sweep debris of food from the
teeth [Rheme]” (Sara’s text). The function of the marked theme in the example above is to guide the reader
through the text by setting the scene for the clause carrying that message.
On the other hand, TP refers to how cohesion and coherence in a text are created by repeating
meanings from the theme of one clause in the theme of subsequent clauses, or by placing elements from
the rheme of one clause into the theme of the next. There are three TP patterns (Eggins 2004): the theme
reiteration (or “constant”) pattern which keeps the same topical theme in focus throughout a sequence of
clauses; the linear (or “zig-zag”) pattern by which information situated in the rheme position moves to the
theme position in the subsequent clause, and the multiple-rheme (or “fan”) pattern by which a theme
introduces a number of various aspects in the rheme position, which are then employed as themes in the
subsequent clauses.
The SF-MDA of pictorial representations was framed by Kress and van Leeuwen’s (2006) framework
for the analysis of the grammar of visual design, which is primarily based on SFL. Kress and van Leeuwen
(2006) assigned representational, interpersonal, and compositional meanings to the analysis of visual
images. First, visual structures, like linguistic structures, include visual representational processes or
activities within and are associated with participant roles and specific circumstances. For example, Kress
and van Leeuwen (2006) argue that when the participants are connected by “vectors” of motion or eyelines,
they are presented as “doing” something. Second, when analyzing the interpersonal metafunction of the
visual modes, one has to take into consideration the relationship between the visual representational
processes and the viewer, which can be revealed through specific visual techniques that build this
relationship, such as facial expressions, gazes, gestures, the angle (is it horizontal or vertical), and distance
of the shots, all contribute to the level of involvement by the viewer and the degree of social distance
between the represented participants and the viewers. The third and final feature is the compositional
metafunction, which helps to determine the extent to which the visual and verbal elements achieve a sense
of coherence to the whole unit which requires the study of the page layout. Kress and van Leeuwen’s (2006)
systems for the analysis of the textual organization in images seemed appropriate for the aims of my study
because they reveal the configurations of the multimodal texts through the representational and interactive
meanings of the image to each other through three “interrelated systems” – composition of information
value (top/bottom, center/margin, left/right); visual salience (size, contrast, color, focus); and visual framing
by dividing lines (or its absence). The absence of framing stresses group identity, whereas its presence
signifies individuality and differentiation. While the composition of information value is similar to Halliday’s
given-new organization in orthographic texts, salience and framing are counterparts of theme/rheme.
4 Data and method of analysis

The corpus used in this qualitative study was written by eight high-achieving Saudi students enrolled in
the bachelor of science dentistry program. Thus, it consisted of eight major assignments (6,085 words)
written in English on a key topic in the Oral Biology course, namely, developmental abnormalities (or
defects) of the face and oral cavity. The eight texts were comparable since they shared the same topic. The
Oral Biology course is one of the foundation courses in the dentistry undergraduate program. The students
were enrolled in the bachelor of science dentistry program and were given the pseudonyms Sara, Ibrahim,
Khalid, Shatha, Hajer, Yara, Zahra, and Noura. A purposive sampling was employed in which the students
were deliberately sought based on gender mix and a high level of achievement. Since the number of
participants cannot be claimed to be a representative sample, the study does not attempt to generalize or
replicate but rather to understand a specific context as it stands.
The SF-MDA included three stages:

• An analysis of thematic choices (topical, textual, and interpersonal) and TP patterns in the orthographic
texts.
• A visual analysis of theme and information value in the pictorial images.
• A description of the meaning-making processes in the pictorial representations and the interplay
between both modes in reaching full meaning.
The unit of analysis in the present study is the T-unit. A T-unit is defined as “a clause complex which
contains one main independent clause together with all the hypotactic clauses which are dependent on it”
(Fries 1995, 318), i.e., an independent clause with one or more dependent clauses. The choice of the T-unit
was prompted by the fact that it is an optimal unit for capturing TP patterns. The identification of theme
depends on the position of the dependent clause in the complex T-unit. If it occurs initially, the entire
clause is deemed to be the theme; conversely, if the independent clause occurs initially, the grammatical
subject is the theme. The frequency of occurrence of each TP pattern was manually annotated. Instances of
elliptical topical themes were included in the analysis as they signal the textual function given. Since the
interpretations of disciplinary-specific pictorial representations are often as important as the language
surrounding them, the SF-MDA also utilized the participants’ intuitive verbal interpretations (or intended
reading path) (van Leeuwen 2005) of these artifacts. These interpretations (or understanding) of the
pictures revealed the processes underlying the construction of conceptual and linguistic knowledge of
theme and information value in the images.
The interpretations were audio-recorded, transcribed, and annotated after placing each interpretation
next to its relevant image. In order to ensure reliability in the annotations of theme, the annotations’ codes
were iteratively cross-checked in addition to being revised by a fellow linguist. In terms of validity, the
percentage for the frequency of each TP type was calculated by dividing the total number of instances by
the total number of occurrences of the overall patterns and then multiplying the result by 100. The use of
quantitative data in the present qualitative study was aimed at making claims such as “most” and
“higher” more precise.
Next I present and discuss the findings of the SF-MDA of oral biology texts.
5 Results and discussion

The findings of the SF-MDA of the textual features in oral biology texts are presented and discussed in this
section, and an overview of the context of the study is included.
5.1 Context
The participants were asked to write an assignment about developmental abnormalities (or defects) of the
face and oral cavity. They were required to refer to the textbook, Oral Histology Development, Structure,
and Function (Nanci 2008). The purpose of the assignment is to understand well the principles of the
development of the oral cavity and the face. Students are expected to be able to:
• have a clear understanding of the congenital and acquired anomalies of the oral cavity and
• know the precise structure and composition of oral tissue.
Each of the eight participants received an “A” for this assignment. Table 1 outlines the key statistics of
the eight students’ texts.
The participants were not constrained by a word limit. With the exception of Yara, the participants’
texts were accompanied with images. It should be noted here that some pictorial representations were
Table 1: A pivot table of the participants’ texts
No. Participant Text word count Number of images Number of tables
1 Sara 865 2 1
2 Ibrahim 966 4 0
3 Khalid 525 6 0
4 Shatha 742 5 0
5 Hajer 305 3 0
6 Yara 1,499 0 0
7 Zahra 656 2 0
8 Noura 527 4 0
Total 6,085 26 1
similar to those in the other students’ assignments since all eight students were required to write on the
topic of defects of the face and oral cavity. As a result, only eight images were analyzed. It is likely that
Yara did not include any visuals in her text because of her preference for a verbal learning style. The eight
texts encompassed 26 images and one table. Sara was the only student who preferred to use a table to
compare the symptoms of two types of macroglossia (or tongue hypertrophy), which includes cases of
apparent tongue enlargement (Vogel et al. 1986).
5.2 SF-MDA findings and discussion
The sociocultural context of this assignment is construed by the three register variables of field, tenor, and
mode (Halliday, 2014):
• Field: the text is on facial and oral cavity defects. Specialized scientific technical terms are related to oral
biology and only known by specialists in the field. These terms are experientially represented in the
written texts through the system of transitivity (relational identifying processes) and logically
represented through lexical sense relations; the terms are represented in pictorial representations
through the system of vectoriality (tilting, distance, and angle) (Kress and van Leeuwen 2006).
• Tenor: type of interaction – monologue, interactive – represented linguistically, in image-text format,
through the declarative mood element. The students formally engaged with the major assignment which
would only be read by a science academic tutor. The text includes interactive pictorial representations,
drawings, and pictures, with some being adjacent to facilitate comparability.
• Mode: written to be assessed by the tutor. Multimodal discourse: texts and multisemiotic images. The
former is linguistically represented through the systems of cohesion, TP, and given/new, whereas the
latter is through the systems of composition, framing, and salience. Medium: print, accompanied by
pictorial printed images: assignment submitted on A4 paper. Frame: informative expository text.
In order to successfully complete the oral biology assignment, students had to manage the expressions
of field, tenor, and mode through experiential, interpersonal, and textual language metafunctions. Due to
space limitations here, I only investigated the way theme and information value were represented by the
eight Saudi EFL participants. The SF-MDA findings revealed that the most frequently occurring TP pattern
was constant theme (81.25%), followed by the rare occurrence of linear pattern (16.25%) and the minimal
use of split-rheme pattern (2.50%). As the students were not constrained by a word limit and were free to
choose the developmental defects about which to write, the frequency of constant theme in the eight
multimodal texts ranged between 73% and 97%. The occurrence of this pattern in the written texts was
higher than that of the images (69.17% and 12.08%, respectively).
The finding related to the extensive use of constant theme is not surprising as this corresponds with
most of the studies (e.g., Li and Fan 2008, Ebrahimi and Ebrahimi 2012, Alyousef 2015a, 2015b). Excerpts of
this pattern are shown below (reiterated experiential themes are italicized).
Tongue-tie can intervene with one’s ability to make certain sounds – such as “t,” “d,” “s,” “z,” “th,” and “l.” It can
particularly be challenging to roll an “r.” (Sara’s text)
Mucoceles are pseudocysts of minor salivary gland origin. They are formed when salivary gland secretions dissect into the
soft tissues surrounding the gland. (Khalid’s text)
These small tumors arise from the epithelium. They have no malignant potential. Most are excised. HEMANGIOMA AND
LYMPHANGIOMA [subheading]. More than half of these angiomas occur in the head and neck. (Shatha’s text)
Two pits develop on the frontal process. They divide the frontal process into three parts. (Hajer’s text)
Congenital pits of the lower lip typically present as bilateral, paramedian depressions in the vermilion border. They
represent small accessory salivary glands. (Yara’s text)
The reconstructive process can be complicated. It often involves multistage procedures (Zahra’s text)
Focal enlargement of the tongue usually is caused by congenital tumors, particularly lymphatic malformations and
hemangiomas (pictures 6 and 7). It also may occur in patients with multiple endocrine neoplasia type 2B. (Noura’s text)
Sara, Zahra, and Noura used the reference item it to refer back to a previously mentioned thematized
participant, while Khalid, Shatha, Hajer, and Yara used the reference item they. This pattern preserves the
theme focus, as new information is presented. The findings also revealed that the students did not use
personal pronouns (we) in the theme position. This finding corresponds with Hyland’s (2005) claim that
while students are inclined to avoid interjections and personal pronouns, “expert writers” employ these
features to build close relationships with readers. Interestingly, the eight students extensively used bullet
points to list the characteristics of some of the developmental defects of the face and oral cavity, the
symptoms, and their causes, as shown below (implicit finites in the rheme position are placed in square
brackets and implicit themes are italicized and placed in square brackets).
The symptoms of macroglossia may be as follows:
• [The first symptom of macroglossia] [is] Dyspnea – labored breathing, cessation of breathing during sleep
(apnea).
• [The second symptom of macroglossia] [is] Dysphagia – difficulty in swallowing certain food or liquids.
• [The third symptom of macroglossia] [is] Dysphonia – disrupted speech, possibly manifest as lisping.
• [The fourth symptom of macroglossia] [is] Sores infecting corners of the mouth.
• [The fifth symptom of macroglossia] [is] Marks on the side border of the tongue due to pressure from the
teeth.
• [The last symptom of macroglossia] [is] A tongue that perpetually extends the mouth may develop
ulceration. (Sara’s text)
The implicit themes and the relational attributive or identifying processes (or verbs) in the above
examples are grammatically truncated (or encoded) through the use of bullet points. Whereas a relational
identifying process identifies an entity (e.g., [The last symptom of macroglossia] [is] a tongue), a relational
attributive process describes the qualities of the entity (e.g., [The third symptom of macroglossia] [is]
Dysphonia-disrupted speech). The rheme is joined with the theme in relational processes through the use
of some form of the verb be, as in “[The first symptom of macroglossia] [is] Dyspnea-labored breathing,
cessation of breathing during sleep (apnea)” (Sara’s text). The use of bullet points makes the constant
theme pattern and other elements implicit. This seems to be a characteristic feature of oral biology texts,
as the main aim is to maintain a reader’s focus on the rheme of a clause, which represents new
information. The topical theme “symptoms of macroglossia” is implicitly repeated six times since the aim
is to concentrate on the main symptoms of macroglossia. Recall of information becomes easier with the
use of bullet points which, in turn, “facilitate the transition from prescription to action” (Chiapello and
Fairclough 2002, 198). The findings also revealed few instances where the students did not benefit from the
powerful means of these tools; instead, they reiterated the topical themes in the subheadings, i.e.,
BREAST-FEEDING PROBLEMS. Breast-feeding requires babies to keep their tongue over the lower gum while sucking.
(Sara’s text)
LIP ANOMALIES: Clefting. Clefting anomalies of the upper lip are more common and more varied than clefting anomalies of
the lower lip. (Ibrahim’s text)
ORAL CAVITY ANOMALIES. Malformations of the oral cavity may result from errors in the embryonic fusion of the anterior
tongue. (Khalid’s text)
LIPOMAS. Intraoral lipomas are rare. (Shatha’s text)
The topical themes in the subheadings were reiterated in the ensuing clauses to provide new
information. Bullet points perform a number of linguistic functions. First, key features of an aspect are
given prominence by being foregrounded and placed in a theme position while peripheral information is
being dispensed (O’Halloran 1999b). Second, the main function of bullet points is to avoid redundancy by
using the constant theme pattern. Finally, they encode textual information and relational processes in the
most economical manner.
The SF-MDA results also showed that most of the students used the topical themes “abnormalities,”
“anomalies,” “disorder,” and “malformations” interchangeably to refer to the same entity. Along similar
lines, Shatha employed the topical themes “facial deformities” and “cleft lips and palates,” while Yara
used the terms “lesion” and “patch.” The use of synonyms in a constant theme pattern makes the text
more cohesive. The frequent use of abstract complex technical terms as topical themes indicates that these
inanimate nominal groups play a major role in the development of theme and information value in oral
biology texts, such as ankyloglossia, dyspnea, dysphagia, dysphonia, dysplasia, epithelial dysplasia,
endocrine frenulum, hemangioma, Idiopathic muscle hypertrophy, lingual thyroid, lymphangioma, macro-
glossia, pseudo macroglossia, and frictional keratosis. This finding is in line with a number of studies
(Iedema 2000, Alyousef, 2013).
Linear (or zig-zag) theme pattern was the second frequent type in the students’ texts (Table 2). This
finding corresponds with the results in Alyousef’s (2015a, 2015b) studies of business discourse. All the
participants employed nondefining relative clauses with the pronouns “that,” “which,” “more,” and “this”
to describe the “thing” being discussed. These elements “serve two functions: as a marker of some special
status of the clause (i.e., textual) and as an element in the experiential structure” (2015a, 10). Excerpts of
linear or zig-zag pattern in the students’ texts, wherein information placed in the rheme position is
repackaged in a subsequent theme are shown below.
Poor oral hygiene. For an older child or adult, [Theme: Topical] tongue-tie can make it hard to sweep debris of food from
the teeth [Rheme]. This [Theme: Topical] can lead to tooth decay and inflammation of the [gums] gingivitis [Rheme].
(Sara’s text)
Some drugs [Theme: Topical] may cause cleft lip and cleft palate [Rheme]. Cleft lip and cleft palate [Theme: Topical] may
be caused by exposure to chemicals or viruses [Rheme]. (Ibrahim’s text)
Thus deformity of the cranium [Theme: Topical] may also be seen as a facial deformity [Rheme]; indeed [Theme:
Interpersonal] this [Theme: Topical] may be more obvious than the skull deformity [Rheme]. Leukoplakia [Theme:
Topical] is a term used to describe a white patch or plaque [Rheme] that [Theme: Textual] cannot be characterized
clinically or pathologically as any other condition [Rheme]. This definition [Theme: Topical] does not imply any specific
histological changes [Rheme]. Lichen planus [Theme: Topical] is the most common cause of persistent white patches in
the mouth [Rheme]. The patches [Theme: Topical] are often striated, forming a lace-like pattern [Rheme]. (Shatha’s text)
The traditional intervention for infants with Robin sequence and airway obstruction [Theme: Topical] has been
tracheotomy [Rheme]. The tracheotomy [Theme: Topical] remains in place until [Theme: Textual] the child and airway
Table 2: Thematic progression patterns in the students’ texts
Thematic progression Semiotic Sara Ibrahim Khalid Shatha Hajer Yara Zahra Noura Sub-total
type resource
Total
Freq. % Freq. % Freq. % Freq. % Freq. % Freq. % Freq. % Freq. % Freq. %
Reiteration (parallel) Text 17 73.91 18 64.28 27 72.97 22 64.71 6 50.00 45 73.77 16 76.19 15 60.00 166 69.17
Visual 2 8.69 4 14.28 9 24.32 5 14.71 3 25.00 0 20.00 2 9.52 4 16.00 29 12.08
Text-Visual 19 82.60 22 78.56 36 97.29 27 79.42 9 81.81 45 73.77 18 85.71 19 76.00 195 81.25
Linear (zig-zag) Text 4 17.40 3 10.72 1 2.71 6 17.64 2 18.19 15 24.59 3 14.29 5 20.00 39 16.25
Visual 0 0.00 0 0.00 0 0.00 0 0.00 0 0.00 0 0.00 0 0.00 0 0.00 0 0.00
Text-Visual 4 17.40 3 10.72 1 2.71 6 17.64 2 18.19 15 24.59 3 14.29 5 20.00 39 16.25
Multiple-Theme Text 0 10.35 3 10.72 0 0.00 1 2.94 0 0.00 1 1.64 0 0.00 1 4.00 6 2.50
Visual 0 10.35 0 0.00 0 0.00 0 0.00 0 0.00 0 0.00 0 0.00 0 0.00 0 0
Text-Visual 0 20.70 3 10.72 0 0.00 1 2.94 0 0.00 1 1.64 0 0.00 1 4.00 6 2.50
Sub-total 23 100 28 100 37 100 34 100 11 100 61 100 21 100 25 100 240 100
An MDA of English dentistry texts written by Saudi undergraduate students

275
[Theme: Topical] are bigger [Rheme] and [Theme: Textual] it [Theme: Topical] is no longer needed [Rheme]. The tongue
[Theme: Topical] is progressively moved forward with the mandible, improving the airway [Rheme]. This [Theme: Topical]
technique has been successful in experienced hands [Rheme]. (Yara’s text)
Ankyloglossia [Theme: Topical] can also prevent the tongue from contacting the anterior palate [Rheme]. This [Theme:
Topical] can then promote an infantile swallow [Rheme]. It [Theme: Topical] can also result in mandibular prognathism
[Rheme]; this [Theme: Topical] happens when the tongue contacts the anterior portion of the mandible with exaggerated
anterior thrusts [Rheme]. (Zahra’s Text)
The thematic complement in Sara’s text is marked: “for an older child or adult.” A marked theme
provides a contextual frame for the ensuing message (Davies 1997). The marked theme in Sara’s text
conflated with the prepositional phrase to provide circumstantial details. Marked themes are thematic
because they are foregrounded as the theme. They could be an adverbial phrase, a prepositional phrase, or
a complement which could potentially be functioning as subject but is not. Marked themes make texts
more coherent through the use of theme predication, which includes thematic and informational choices
(Eggins 2004). Theme predication refers to the case where a constituent is moved from the theme position
and placed in the rheme position in order to be emphasized through intonation, while the real new
information is intact. Thus “John” in the second example below is stressed and became new: John [Theme:
Topical] broke the vase [Rheme]. It [Theme: Topical] was John [Rheme] who [Theme: Topical] broke the
vase [Rheme].
The rare use of split-rheme pattern was found in the texts of four participants: Yara, Ibrahim, Noura,
and Shatha. They employed this pattern by using bullet points in order to list two or three themes in the
rheme position that were picked up and developed in the subsequent themes.
Macroglossia [Theme: Topical] can be focal or generalized [Rheme].

• Focal enlargement of the tongue usually [Theme: Topical] is caused by congenital tumors, particularly lymphatic
malformations and hemangiomas [Rheme][…].
• Generalized macroglossia [Theme: Topical] is seen in Beckwith–Wiedemann syndrome and hypothyroidism [Rheme].
(Yara’s text)
The orbicularis oris muscle [Theme: Topical] is the principal muscle of the lip [Rheme] and [Theme: Textual] [it] is divided
into two parts [Rheme] […] The deep component, in concert with other oropharyngeal muscles, [Theme: Topical] works in
swallowing and serves as a sphincter [Rheme]. The superficial component [Theme: Topical] is a muscle of facial expression
[Rheme] and [Theme: Textual] inserts into the anterior nasal spine, sill, alar base, and skin to make the philtral ridges
[Rheme]. (Ibrahim’s text)
White patches [Theme: Topical] have three main histological features: abnormal keratinization, hyper- or hypoplasia of
the epithelium, and disordered maturation (dysplasia) [Rheme]. Dysplasia [Theme: Topical] is the only significant
histological guide to the possibility of malignant change [Rheme]. (Shatha’s text)
Macroglossia [Theme: Topical] can be focal or generalized [Rheme]. Focal enlargement of the tongue usually [Theme:
Topical] is caused by congenital tumors, particularly lymphatic malformations and hemangiomas [Rheme][…].
Generalized macroglossia [Theme: Topical] is seen in Beckwith-Wiedemann syndrome [32] and hypothyroidism
[Rheme]. (Noura’s text)
Ibrahim employed this pattern to describe the two parts of the primary muscle of the lip,
orbicularis oris: the deep and superficial components. Likewise, Yara and Noura used this pattern to
introduce the two subdivisions of macroglossia. Although Shatha used this pattern to name three main
histological features of white patches in the oral cavity, she only elaborated on dysplasia because of
its ability to turn into a malignant lesion. These results showed some of the functions of this pattern in
oral biology texts: describing the types or key features related to facial or oral cavity defects. The
second topical theme “it” in Ibrahim’s excerpt is ellipsed and was recovered from the first sentence.
The split-rheme pattern is considered the most difficult for students as different pieces of information
are packed or listed in the rheme position and then picked up and used as themes in the following
clauses. This finding is in keeping with Alyousef’s (2015a) study of finance texts (<2.50%). The absence
of this pattern in the other four participants’ texts may suggest their limited proficiency level and
probably their limited writing opportunities. However, the good results the students achieved indicate
that they did not have difficulties that hindered their lexico-grammatical choices and the multimodal
meaning-making processes.
The data lacked instances of objective interpersonal themes with anticipatory “it” in the subject
position with (be to +) infinitive. Writers employ this theme to extend their viewpoints (“it is expected
that…”) when presenting key features of an aspect. This finding seems expected as oral biology texts deal
with facts rather than presumptions (Abd-El-Khalick et al. 2008). Martı́nez (2003) investigated theme in
biology journals and found a lower proportion of interpersonal themes in the discussion section. This
indicates the students’ influence by the dominant ideologies of scientific oral biology texts. What follows is
the SF-MDA of the pictorial representations.
5.2.1 SF-MDA of the pictorial representations
Pictorial representations (photographs, drawings, paintings, movies) support students’ learning, as they
help them to construct scientific learning when attempting to recall a message. Image–text relations are
metafunctionally integrated across the experiential, textual, and logical meanings at the discourse stratum
(Liu and O’Halloran 2009). As stated earlier, the SF-MDA of oral biology pictorial representations (or the
visual semiotic mode) and the surrounding text aimed to reveal the underlying processes through which
students constructed knowledge of theme and information value. The pictorial representations
(fabrications) in oral biology discourse include realist photographs and abstract drawings.
Oral biology images function intersemiotically (across different semiotic resources) through the
interaction of the images and the accompanying text, as readers shift their attention from one semiotic
mode to another. The pictorial representations provide spatial information that clarifies the meaning in the
accompanying text, as students shuttle between the visual and verbal modes. O’Halloran (1999a, 317)
refers to “meaning arising from interaction and interdependence between these semiotic codes in joint
construction” as a “semiotic metaphor.” The primary semiotic code that formed the “semiotic metaphor” is
the text accompanying the picture, as it served as the point of departure and the anchor for the message.
Concepts, such as ankyloglossia and dyspnea, are transformed into another format. The meaning-making
processes of pictorial representations encompass multimodal explanations, as they explain the key
features of an aspect. As these representations clarify the text, a logico-intersemiotic semantic relation of
elaboration exists. The accompanying text provides a strong topical focus by presenting further
explanations that are not present in the photographic image. Ideologically, the text plays the prime (or
lead) role while the images a subservient role. Oral biology images contain visually realized implicit
processes and participants that can be recovered from the prior verbal text (van Leeuwen 2006). The two
semiotic resources, however, construct a similar meaning as they share the same topical theme. As van
Leeuwen (2005, 79) states, “the two are not concatenated in linear fashion. They fuse, like elements in a
chemical reaction.” This finding exemplifies the ideational complementarity relation of distribution
mentioned by Unsworth (2006) and Daly and Unsworth (2011), as the two semiotic modes jointly construct
similar content.
The SF-MDA findings revealed that oral biology pictorial representations included instances of
constant theme (Table 2), as they aim to illustrate and, thereby, complement verbal texts.
For example, the implicit conceptual knowledge underlying the photographic picture of ankyloglossia
(or tongue-tie) in Zahra’s text (Figure 1) constitutes an important part of meaning-making. It can be made
explicit through Zahra’s verbal interpretation (or reading path) of the picture as follows:
Ankyloglossia or tongue-tie [Theme] is a congenital oral anomaly [Rheme] that [Theme: Textual] may decrease the
movement of the tongue tip [Rheme] and [Theme: Textual] [it] is caused by a short, thick lingual frenulum [Rheme].
(Zahra’s text)
Figure 1: A photographic picture of ankyloglossia (or tongue-tie) from Zahra’s text.
This interpretation included two instances of constant theme. Thus, ankyloglossia is the theme and what
follows is the rheme. Salience and framing are the main principles of information value in images. Salience
can be realized through size, color contrasts, movement, or anything that distinguishes a word from others
(i.e., through different weight, font, or set). Ideas are expressed by means of adjacency and color. They are
expressed in the oral biology texts by the degree of difference between the adjacent pictures. Color is a
mode of representation, which is organized in oral biology images through splashes with high color
saturation, especially Figures 1 and 2, which are brighter and more illuminated. The high color saturation
in Figure 1 reveals the accumulation of saliva on both sides, which represents the patient’s suffering from
ankyloglossia. In fact, a tongue-tie can lead to the accumulation of saliva or even drooling of saliva.
Different shades of the orange color are being used in the two images to emphasize the lip. As Kress and
van Leeuwen (2006, 145) noted, color is conceived of as “a combinatory system with five elementary,
‘abstract’ colors (‘red’ in general, rather than a specific red, and so on) from which all other colors could be
mixed.” For example, the ankyloglossia visual image used different colors (for lips, teeth, mouth, etc.) to
represent the real-world (or bodily) elements and distinguish the different parts of the human oral cavity,
while creating topical unity and coherence. The mouth in the ankyloglossia image (Figure 1) is given
greater salience, as it stands out from its immediate textual environment, i.e., the human face, since the
aim of this semiotic mode is to illustrate the attachment of the lingual frenulum to the lower part of the
tongue. The lingual frenulum represents the nucleus of information to which all the other elements are in
some sense subservient. The tongue represents the ideal (given) and the lingual frenulum represents the
real (or new) information. The angle of the shot is at an extreme close-up distance which draws the
reader’s focus and involvement. Thus, the topical given theme ankyloglossia anomaly is foregrounded in
Figure 2: A photographic picture of macroglossia/maxillary hypoplasia from Sara’s text.

the zoomed image, while new information is elicited from the students’ interpretations of images, or from
the text accompanying the picture. In other words, the topical theme in the text accompanying the oral
biology picture is saliently illustrated in Figure 1. This indicates that the structure of the two semiotic
modes is in complete intersemiotic parallelism. Intersemiotic parallelism refers to “a cohesive relation
which interconnects both language and images when the two semiotic components share a similar form”
(Liu and O’Halloran 2009, 10). The two tongues in the macroglossia/maxillary hypoplasia adjacent
photographs (Figure 2) stand out as distinct (or salient) entities because they are not equal in size. They
are the mediators since they represent the most salient entities in the two images. The angle here is at a
close-up distance. Likewise, the mouth and the nose (Figure 3) are the most salient entities (or topical
themes) in Ibrahim’s two adjacent illustrative drawings of lip anomalies. Over 25% of the images in the
students’ texts included adjacent pictures or drawings.
A reader attempts to compare the two adjacent pictures by initiating the gaze at the left picture. The
topical themes in the two adjacent images function intrasemiotically since they facilitate comparability. By
intrasemiotically, I mean moving your eyes from one component to another within the same semiotic
mode. This contrasts with the intersemiotic shifts explained earlier. As the image encompasses two
adjacent pictures, the reading path (or a viewer’s interpersonal function of gaze) is linear, i.e., from left to
right and from top to bottom. The SF-MDA of informational content in Figure 1 revealed, however, that
new visual information is at the center of the zoomed picture, rather than to the right. Kress and van
Leeuwen’s (2006) left-hand and right-hand English-language spatial dimensions, therefore, do not
correspond with the compositions of given-new/ideal-real in zoomed images of congenital and acquired
anomalies of the oral cavity. This corresponds to Alyousef’s (2015a, 2015b) and Jones’ (2006) findings that
new visual information does not always occur to the right side. O’Halloran (1999b, 27) states that while
“verbal discourse functions to describe commonsense reality, visual display connects our physiological
perceptions to this reality and in combination with metaphorical shifts, creates new entities which are
intuitively accessible.” Fei (2004) refers to the element that marks the beginning of the reading path of the
dominant visual semiotic element as the “center of visual impact” or, using van Leeuwen’s (1993) earlier
term, a brief “scanning” of a page to peruse visually salient elements. Whereas Fei (2004) equates image
with language in terms of visual perception, van Leeuwen (1993) prioritizes image over language, and
Kress (2003) proposes that scanning determines the dominant semiotic resource. In his analysis of the eye
image in biology texts in secondary schools, Kress (2003) argues that salience is captured by the bolding of
captions. Higher level relations are marked by proximity, while the lower levels use connecting lines. The
visuals aided students in building up their taxonomy of oral biology terms.
Framing refers to the demarcation of the elements of a text (verbal or visual). It can be realized
through a wide range of semiotic modes (or resources) and, within each mode, by a number of different
means.
Figure 3: An illustrative drawing of cleft lip types from Ibrahim’s text.

For example, the participants in Figure 3 are demarcated by means of an actual frame line that divides
the two images, which refer to two distinct classifications of the same concept. The process is expressed
here in each drawing by means of the labeled lines (or leaders) and the borderlines circumventing the
mouth and the nose.
6 Conclusion and implications

The students have understood well the principles of the development of the oral cavity and the face, as
evidenced by their verbal interpretations of oral biology pictorial representations and the high scores they
received for this assignment. The students have understood the congenital and acquired anomalies of the
oral cavity and also identified the structure and composition of the oral and dental tissues. This study
contributes to our understanding of the way in which theme, the composition of information value, and
the logico-semantic relations are constructed in the oral biology multimodal texts. These texts intertwined
two TP patterns: constant and linear theme patterns. The findings also revealed disciplinary-specific
functions of the split-rheme pattern, which was minimally used.
The SFL provides the theoretical basis for future development of a powerful framework for
systematically describing the lexico-grammatical aspects of all types of visual artifacts. As Unsworth
(2006, 57) stated, “the strength of SFL in contributing to frameworks for the development of intersemiotic
theory emanates from its conceptualization of language as one of many different interrelated semiotic
systems,” such as mathematics, music, painting, and so forth. This framework needs to be capable of
expansion/modification as new forms of communication emerge (Unsworth 2006). The SF-MDA of the
composition of information in oral biology images extends Kress and van Leeuwen’s (2006) functional
description of meaning-making resources of visual artifacts in terms of compositional zones. New visual
information in oral biology texts is at the center of a zoomed picture. The students’ explanations (or
interpretations) contributed to our understanding of the meaning-making processes and the way they
processed and integrated print and visual images in scientific English. The participants’ interpretations
and the SF-MDA revealed the processes underlying the construction of conceptual and linguistic
knowledge of theme and information value. However, we should keep in mind that the abstract aspect of
visual representations can yield different interpretations; therefore, only a subset of the full range of the
writers’ communicative potential has been presented. This observation contradicts the view that the
structuring of the reading path is a linear, staged, goal-oriented process. Readers of multimodal texts
construct meaning from these texts by “design[ing] the way the text is read, its reading path, what is
attended to and, in the process, construct a unique experience during their transaction with a text”
(Serafini 2012, 157). In addition, the SF-MDA revealed the construction of the intra- and intersemiotic
meanings in the multimodal oral biology texts. Meanings were construed by the students through the
intersemiotic shifts (or the resemiotization processes) from textual form to visual and vice versa. The
reiteration of a theme in the text surrounding the images provides a strong topical focus by presenting
further explanations (elaboration).
The following key linguistic features are drawn from the results of the SF-MDA of oral biology texts:
• The development of theme and information value in oral biology texts is based on extensive use of
inanimate, abstract, complex, and topical themes not known by the general community but by
specialists in the field.
• The extensive use of textual themes.
• The use of nondefining relative clauses with pronouns to describe a “thing.”
• Key features of an aspect are foregrounded through the use of bullet points in order to facilitate recall
and to avoid repetition of the same information through the use of a constant theme pattern.
• Image–text relations in oral biology discourse aim to illustrate and, thereby, complement verbal texts.
The two modes jointly construct similar content.
• A logico-intersemiotic semantic relation of elaboration exists between the images and the text
surrounding the images since the former provide spatial information.
The present study is limited to eight oral biology texts and, therefore, the findings are not based on a
representative sample of the discipline’s academia. A number of pedagogical implications of the study are
suggested for science tutors as well as undergraduate EFL/ESL science students, especially those whose
first language has a different information structuring pattern from English, such as Arabic and Chinese.
Students can employ the split-rheme pattern in longer essays where different pieces of information are
listed in the rheme position and then picked up and introduced in the theme position in subsequent
clauses. This pattern can also be employed through the use of the powerful means of bullet points where
the key features of an aspect (e.g., characteristics of some of the developmental defects of the face and oral
cavity, the symptoms, and their causes) are listed. Thus, the key features are foregrounded and placed in
the theme position and peripheral information is dispensed with instead of being reiterated. Bullet points
facilitate recall of information. TP patterning plays a major role in facilitating the comprehension of a text
and in producing a well-structured text. Science tutors can introduce students into types of TP patterns
and in particular, the split-rheme pattern, which is regarded as the most difficult for students. They need
to focus on the process of writing and not only the product by presenting a number of class activities on
the use of theme. Tutors should introduce different strategies for creating a cohesive text through the
implementation of activities which could include exercises that require students to identify and analyze TP
patterns. Subsequently, they can be asked to write an essay to practice organizing theme in their writing.
This will aid students to easily control the flow of their texts and to organize their texts more effectively.
Abbreviations
EFL English as a foreign language
ESL English as a second language
SFL Systemic functional linguistic
SF-MDA Systemic functional multimodal discourse analysis
TP Thematic progression
Acknowledgments: The author is indebted to the two anonymous reviewers for their insightful and helpful
comments. The author also expresses his gratitude to both the Deanship of Scientific Research at King
Saud University and the Research Centre at the Faculty of Arts for funding the present study. He also
thanks RSSU at King Saud University for their technical support.
References
Abd-El-Khalick, Fouad, Mindy Waters, and An-Phong Le. 2008. “Representations of nature of science in high school chemistry
textbooks over the past four decades.” Journal of Research in Science Teaching 45(7): 835-55. doi: 10.1002/tea.20226.
Adami, Elisabetta. 2016. “Multimodality.” In Oxford Handbook of Language and Society, edited by O. Garcìa, N. Flores, and
M. Spotti, 451–72. Oxford: Oxford University Press.
Alyousef, H. S. 2013. “An investigation of postgraduate business students’ multimodal literacy and numeracy practices in
finance: a multidimensional exploration.” Social Semiotics 23(1): 18–46. doi: 10.1080/10350330.2012.740204.
Alyousef, H. S. 2015a. “A multimodal discourse analysis of international postgraduate business students’ finance texts: an
investigation of theme and information value.” Social Semiotics 26(5): 486–504. doi: 10.1080/10350330.2015.1124518.
Alyousef, H. S. 2015b. A study of theme and information structure in postgraduate business students’ multimodal written texts:
a SF-MDA of management accounting texts. Paper presented at the 2015 Asian Conference on Language Learning (ACLL
2015): Proceedings of the International Academic Forum, Kobe, Japan.
Alyousef, H. S. 2016. “A multimodal discourse analysis of the textual and logical relations in marketing texts written by
international undergraduate students.” Functional Linguistics (A Springer Open Journal) 3(3): 1–29. doi: 10.1186/s40554-
016-0025-1.
Alyousef, H. S. 2017. “A multimodal discourse analysis of textual cohesion in tertiary marketing texts written by international
undergraduate students.” In Semiotics 2016: Archaeology of Concepts (Yearbook of the Semiotic Society of America), ed.
J. Pelkey, 99–122. Charlottesville, VA, US: Philosophy Documentation Center.
Baldry, Anthony, and Paul J. Thibault. 2005. Multimodal Transcription and Text Analysis: A Multimedia Toolkit and Coursebook.
London: Equinox.
Chiapello, Eve, and Norman Fairclough. 2002. “Understanding the new management ideology: a transdisciplinary contribution
from critical discourse analysis and new sociology of capitalism.” Discourse and Society 13(2): 185–208. doi: 10.1177/
0957926502013002406.
Daly, Ann, and Len Unsworth. 2011. “Analysis and comprehension of multimodal texts.” Australian Journal of Language and
Literacy 34(1): 61–80. Retrieved from http://hdl.handle.net/10072/48262.
Davies, Florence. 1997. “Marked Theme as a heuristic for analysing text-type, text and genre.” In Applied Languages: Theory
and Practice in ESP, ed. J. Piqué, and D. Viera, 45–79. València, Spain: Universitat de València.
Drury, Helen, Peter O’Carroll, and Tim Langrish. 2006. “Online approach to teaching report writing in chemical engineering:
Implementation and evaluation.” International Journal of Engineering Education 22(4): 858–67. Retrieved from http://
www.ijee.ie/articles/Vol22-4/16_ijee1747.pdf.
Ebrahimi, Seyed Foad, and Seyed Jamal Ebrahimi. 2012. “Information development in EFL students composition writing.”
Advances in Asian Social Sciences 1(2): 212–7.
Eggins, Suzanne. 2004. An Introduction to Systemic Functional Linguistics. 2nd ed. London/New York: Continuum.
Fei, Victor L. 2004. “Developing an integrative multi-semiotic model.” In Multimodal Discourse Analysis: Systemic-Functional
Perspectives, ed. K. L. O’Halloran, 220–46. London: Continuum.
Fries, Peter. 1995. “Themes, methods of development, and texts.” In On Subject and Theme: A Discourse Functional
Perspective, ed. R. Hasan, and P. Fries, 317–60. Amsterdam: John Benjamins.
Guo, Libo. 2004. “Multimodality in a biology textbook.” In Multimodal Discourse Analysis: Systemic-Functional Perspectives,
ed. K. O’Halloran, 196–219. London/New York: Continuum.
Halliday, M. A. K. 1978. Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward
Arnold.
Halliday, M. A. K. 2004. “The language of science.” In Collected Works of M. A. K. Halliday, vol. 5, ed. J. Webster, 162–78.
London: Continuum.
Halliday, M. A. K. 2014. Introduction to Functional Grammar. Revised by Christian M. I. M. Matthiessen. 4th ed. London/
New York: Taylor and Francis.
Hannus, Matti, and Jukka Hyönä. 1999. “Utilization of illustrations during learning of science textbook passages among low-
and high-ability children.” Contemporary Educational Psychology 24(2): 95–123. doi: 10.1006/ceps.1998.0987.
Hawes, Thomas. 2015. “Thematic progression in the writing of students and professionals.” Ampersand 2: 93–100.
doi: 10.1016/j.amper.2015.06.002.
Hsu, Pei-Ling, and Wen-Gin Yang. 2007. “Print and image integration of science texts and reading comprehension: a systemic
functional linguistics perspective.” International Journal of Science and Mathematics Education 5(4): 639–59.
doi: 10.1007/s10763-007-9091-x.
Hyland, Ken. 2005. “Patterns of engagement: dialogic features and L2 undergraduate writing.” In Analysing Academic Writing:
Contextualized Frameworks, ed. L. Ravelli, and R. Ellis, 5–23. London: Continuum.
Iedema, Rick. 2000. “Bureaucratic planning and resemiotisation.” In Discourse and Community: Doing Functional Linguistics,
ed. E. Ventola, 47–69. Tubingen: Gunter Narr Verlag Tubingen.
Jaipal, Kamini. 2010. “Meaning making through multiple modalities in a biology classroom: a multimodal semiotics discourse
analysis.” Science Education 94(1): 48–72. doi: 10.1002/sce.20359.
Jones, Janet. 2006. Multiliteracies for academic purposes: a metafunctional exploration of intersemiosis and multimodality in
university textbook and computer-based learning resources in science. Unpublished doctoral thesis. Sydney, Australia:
University of Sydney. Retrieved from http://hdl.handle.net/2123/2259.
Korani, Akram. 2012. “A survey of the cohesive ties – reference and lexical cohesion – in history books of the second and third
grades in guidance school in Iran.” Procedia-Social and Behavioral Sciences 47: 240–3. doi: 10.1016/
j.sbspro.2012.06.645.
Kress, Gunther. 2003. Literacy in the New Media Age. London: Routledge.
Kress, Gunther, and Theo van Leeuwen. 2006. Reading Images: The Grammar of Visual Design. London: Routledge.
Lemke, Jay L. 2002. “Multimedia semiotics: Genres for science education and scientific literacy.” In Developing Advanced
Literacy in First and Second Languages. Meaning with Power, ed. M. J. Schleppegrell, and M. C. Colombi, 21–44. Mahwah,
NJ: Erlbaum.
Li, Jian, and Xiang-Tao Fan. 2008. “Application of patterns of thematic progression to literary text analysis.” Journal of Dalian
University 29(4): 59–62. Retrieved from http://en.cnki.com.cn/Article_en/CJFDTOTAL-DALI200804013.htm.
Liu, Yu, and Kay L. O’Halloran, 2009. “Intersemiotic texture: analyzing cohesive devices between language and images.” Social
Semiotics 19(4): 367–88. doi: 10.1080/10350330903361059.
Martı́nez, Iliana A. 2003. “Aspects of theme in the method and discussion sections of biology journal articles in English.”
Journal of English for Academic Purposes 2(2): 103–23. doi: 10.1016/S1475-1585(03)00003-1.
Nanci, Arnold. 2008. Ten Cate’s Oral Histology Development, Structure, and Function. India: Elsevier.
North, Sarah. 2005. “Disciplinary variation in the use of theme in undergraduate essays.” Applied Linguistics 26(3): 431–52.
doi: 10.1093/applin/ami023.
O’Halloran, Kay. 1998. “Classroom discourse in mathematics: a multisemiotic analysis.” Linguistics and Education 10(3):
359–88. doi: 10.1016/S0898-5898(99)00013-3.
O’Halloran, Kay. 1999a. “Interdependence, interaction and metaphor in multisemiotic texts.” Social Semiotics 9(3): 317–54.
doi: 10.1080/10350339909360442.
O’Halloran, Kay. 1999b. “Towards a systemic functional analysis of multisemiotic mathematics texts.” Semiotica 124(1–2):
1–30. doi: 10.1515/semi.1999.124.1-2.1.
O’Halloran, Kay. 2005. Mathematical Discourse: Language, Symbolism and Visual Images. London: Continuum.
O’Halloran, Kay. 2008. “Mathematical and scientific forms of knowledge: a systemic functional multimodal grammatical
approach.” In Language, Knowledge and Pedagogy: Functional Linguistic and Sociological Perspectives ed. F. Christie,
and J. R. Martin, 205–36. London: Continuum.
Okawa, Toshikazu. 2008. Academic literacies in the discipline of nursing: Grammar as a resource for producing texts.
Unpublished master’s thesis. Adelaide: University of Adelaide.
Saudi Ministry of Education. 2016. Number of Students Registered in Dentistry Colleges per Sector, Nationality, and Gender.
Riyadh, Saudi Arabia: Ministry of Education.
Serafini, Frank. 2012. “Expanding the four resources model: reading visual and multi-modal texts.” Pedagogies:
An International Journal 7(2): 150–64. doi: 10.1080/1554480X.2012.656347.
Unsworth, Len. 2006. “Towards a metalanguage for multiliteracies education: describing the meaning making resources of
language-image interaction.” English Teaching: Practice and Critique 5(1): 55–76. Retrieved from www.education.waikato.
ac.nz.
van Leeuwen, Theo. 1993. “Genre and field in critical discourse analysis: a synopsis.” Discourse and Society 4(2): 193–223.
van Leeuwen, Theo. 2005. “Multimodality, genre and design.” In Discourse in Action: Introducing Mediated Discourse Analysis,
ed. S. Norris, and R. Jones, 73–94. London: Routledge.
van Leeuwen, Theo. 2006. “Towards a semiotics of typography.” Information Design Journal 14(2): 139–55. doi: 10.1075/
idj.14.2.06lee.
Vogel, James, John Mulliken, and Leonard Kaban. 1986. “Macroglossia: a review of the condition and a new classification.”
Plastic and Reconstructive Surgery 78(6): 715–23.

Multimodal Discourse Analysis

Uploaded by

Copyright:

Available Formats

You might also like

Multimodal Discourse Analysis

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Multimodal Discourse Analysis

Uploaded by

Copyright:

Available Formats

JOURNAL OF ADVANCES IN LINGUISTICS Vol 10 (2019) ISSN: 2348-3024 https://rajpub.com/index.

A Multimodal Discourse Analysis of the Interactive Meaning in Public Service Advertisement

School of Interpreting and Translation Studies, Guangdong University of Foreign Studies,

No.2 Baiyun Dadao Bei, Baiyun District, Guangzhou, China

2 SFG, VG, MDA and PSA

2.1 Systemic Functional Grammar (SFG)

2.2 Visual Grammar (VG)

Table 1 Three strands of meta-functions in SFL and VG

2.3 Multimodal Discourse Analysis (MDA)

2.4 Public Service Advertisement (PSA)

Q1: How is the interactive meaning constructed in visual mode?

Q2: How is interactive meaning facilitated in verbal mode?

4.1 Analysis on Example 1 “Everyone Pays”

Figure 1 “Everyone Pays”

4.1.1 Analysis on the Interactive Meaning in the Visual Image

(1) Visual contact

(2) Social distance

4.1.2 Analysis on the Interactive Meaning in the Verbal Language

4.1.3 Analysis on the Visual-Verbal Inter-Semiotic Interaction

4.2 Analysis on Example 2 “Broken Child”

Figure 2 “Broken Child”

4.2.1 Analysis on the Interactive Meaning in the Visual Image

(1) Visual contact

(2) Social distance

4.2.2 Analysis on the Interactive Meaning in the Verbal Language

4.2.3 Analysis on the Visual-Verbal Inter-Semiotic Interaction

4.3 Analysis on Example 3 “Cigarettes Smoke People”

Figure 3 “Cigarettes Smoke People”

4.3.1 Analysis on the Interactive Meaning in the Visual Image

(1) Visual contact

(2) Social distance

4.3.2 Analysis on the Interactive Meaning in the Verbal Language

4.3.3 Analysis on the Visual-Verbal Inter-Semiotic Interaction

4. Royce, T. Synergy on page: exploring inter-semiotic complementarity in page-based multimodal text[A].

6. O 'Halloran, K. Towards a systemic-functional analysis of multi-semiotic mathematics texts[J]. Social

11. Eggins, S. An introduction to systemic functional linguistics[M]. London: Continuum. 2004.

13. Martinec, R. Cohesion in action[J]. Semiotic, 1998:120(1/2), 161-180.

14. Halliday, M. An introduction to functional grammar[M]. London: Edward Arnold. 1994.

25. Hu Zhuanglin, Dong Jia. Multimodal Construction of Meaning-Discourse Analysis of One-on-One

28. Leech, G. English in advertising[M]. London: Longman. 1966.

29. Cook, G. The discourse of advertising[M]. London: Routledge. 1992.

A Multimodal Discourse Analysis of Tmall’s Double Eleven

3.1 Representational Meaning

Figure 1. Wall with shaped holes

Figure 2. Wall filled with commodities

Figure 3. Man surrounded by document piles

Figure 4. Wine glass

Figure 5. Man riding a wrecking ball

Figure 6. Woman running through a door

Multimodal Discourse Analysis of the Movie Argo

Scene one: Burning of American national flag

Scene two: Six diplomats in security office

RESEARCH Open Access

A multimodal discourse analysis of the

Table 1 Theme types

Fig. 1 Linear (or ‘sequential’/‘zig-zag’) Theme pattern

Fig. 2 Multiple-Theme or split Rheme progression pattern

A Centred composition is perceived by some element(s) appearing in the centre,

The SF-MDA of the composition of information value in terms of Given-New and

[The first objective] [is] Continuing to focus on consumer research.

Increase overall dealer ratings 7 to 75 % by period 8.

[The first strength] [is] technical competence of the proprietors

Extensive use of Theme reiteration pattern, followed by the linear Theme

Discuss key features of items