See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/235897890

The Visual Representation of Metaphor: A Systemic Functional Approach

Article  in  Annual Review of Cognitive Linguistics · December 2013

CITATIONS READS

0 830

2 authors, including:

William Dezheng Feng

The Hong Kong Polytechnic University
33 PUBLICATIONS   56 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Call for paper: Session 10 " " View project

Multiliteracies pedagogy in English language teaching View project

All content following this page was uploaded by William Dezheng Feng on 21 May 2014.

The user has requested enhancement of the downloaded file.

 y
pan
The visual representation of metaphor

Com
A social semiotic approach

ing
Dezheng Feng and Kay L. O’Halloran

lish
The Hong Kong Polytechnic University / National University of Singapore

Pub
Complementing cognitive theories which attribute the understanding of visual
metaphors to situational and cultural contexts, this study adopts a social semi-
otic perspective to investigate how visual images themselves are constructed
ins
to cue conceptual metaphors. The visual realization of metaphors in represen-
tational, interactive and compositional meaning structures is elucidated based
am

on Kress and van Leeuwen’s (2006) visual grammar. It is found that most types
of visual metaphor identified by cognitive linguists can be explained within
the framework. Instances of visual metaphor in advertisements are analyzed in
enj

terms of their persuasive effects. It is concluded that the social semiotic frame-
work is able to provide a comprehensive account of the visual realization of
nB

metaphor, and in addition, the study also offers a cognitive explanation of how
resources like camera positioning and composition acquire meanings.
Joh

Keywords: visual metaphor, conceptual metaphor theory, social semiotics,

visual grammar, metafunctions
-

1. Introduction
ofs

The central argument of conceptual metaphor theory is that metaphor is a con-

pro

ceptual phenomenon (Lakoff & Johnson, 1980), realized in both language and
other communication modes, such as visual image, gesture and architecture (e.g.
Forceville, 2009; Goatly, 2007; Kovecses, 2002). Recently, the study of non-linguis-
ted

tic realization of metaphor has attracted much attention, following the pioneering
works of Forceville (1994, 1996), Carroll (1996), Morris (1993), and others. How-
rec

ever, these early attempts define visual metaphors in terms of “their surface real-
ization or formal characteristics” (El Refaie, 2003, p. 78). El Refaie (2003) argues
cor

that visual metaphors should be seen as the pictorial expression of metaphori-

cal thinking. While agreeing with this definition, we further ask, as semioticians,
Un

Review of Cognitive Linguistics 11:2 (2013), 320–335. doi 10.1075/rcl.11.2.07fen

issn 1877–9751 / e-issn 1877–976x © John Benjamins Publishing Company
 The visual representation of metaphor 321

y
how metaphors are visually expressed. In other words, we are interested in the

pan
visual mechanisms which are used to construct metaphors. From this perspective,
Carroll (1996) and Forceville (1996) can be viewed as efforts to describe the visual

Com
realization of conceptual metaphors, which accord with El Refaie’s (2003) cogni-
tive definition.
However, the descriptions are inadequate, as “there seems to be a whole range
of different forms through which metaphorical concepts can be expressed visu-

ing
ally” (El Refaie, 2003, p. 80). We argue that the inadequacy is due to the lack
of understanding and systematic description of meaning making mechanisms in

lish
visual images. As a result, cognitive linguists attribute the understanding of vi-
sual metaphors to the situational/cultural context, but pay less attention to the
text-internal mechanisms of visual images. From a semiotic point of view, while

Pub
acknowledging the role of context and human cognition, we argue that visual
images themselves are constructed in certain ways to cue metaphors. Therefore,
our aim is to provide a systematic account of the visual mechanisms for the re-
ins
alization of metaphor, based on Kress and van Leeuwen’s (2006) social semiotic
visual grammar. This endeavor complements Feng’s (2011a) explanation of visual
am

grammar with conceptual metaphor theory.

The data analyzed in this study is comprised of 100 car advertisements, which
enj

are chosen for their creative use of visual images (cf. Forceville, 1996). The main
conceptual framework of the social semiotic approach is presented in Section 2.
nB

Then we discuss how metaphors are visually realized in representational mean-

ing structures in Section 3. The metaphorical meaning of interactive resources is
Joh

investigated in Section 4, and in Section 5 we examine how conceptual metaphors

are realized in the composition of visual images. Finally, we describe how a social
semiotic visual grammar can provide a comprehensive account of the visual con-
-

struction of metaphors and how conceptual metaphor theory lends epistemologi-

cal status to such a grammar in Section 6.
ofs

2. Visual metaphor from a social semiotic perspective

pro

Alongside Forceville’s (1994, 1996) theory of pictorial metaphor, the field of mul-
ted

timodal semiotics emerged, building on Halliday’s (1994) social semiotic theory

of language (known as systemic functional linguistics). In this approach, language
rec

is modeled as sets of inter-related systems of choices which are metafunctionally

organized. The “systemic” principle regards grammar as systems of paradigmatic
cor

choices which are represented as system networks. The “functional” principle

states that language simultaneously provides resources for constructing three
Un

metafunctions: ideational meaning, interpersonal meaning and textual meaning.

322 Dezheng Feng and Kay L. O’Halloran

y
Social semioticians argue that these principles are applicable to non-linguistic

pan
resources as well, which results in the development of metafunctional frame-
works for semiotic resources such as visual image, architecture and mathematical

Com
symbols (e.g. Kress & van Leeuwen, 2006; O’Toole, 2010; O’Halloran, 2005). Ac-
cording to Kress and van Leeuwen (2006), visual images, like language, fulfill the
metafunctions of representing the experiential world (representational meaning),
interacting with viewers (interactive meaning), and arranging the visual resources

ing
(compositional meaning).
Representational meaning is realized by the configuration of processes (e.g.

lish
actions), participants (e.g. actors), and circumstances (e.g. locations). Kress and
van Leeuwen (2006, pp. 45–113) further identify two types of structure in terms of
representation: narrative and conceptual. These structures are defined in terms of

Pub
the relationship between the image participants, that is, whether it is based on the
“unfolding of actions and events, processes of change” (i.e. narrative), or based on
“generalized, stable and timeless essence” (i.e. conceptual). Interactive meaning
ins
involves the four parameters of symbolic contact, social distance, power relations,
and involvement between viewers and visual participants. Contact is constructed
am

by the nature of the visual participants’ gaze at viewers; social distance is con-
structed by shot distance (e.g. close or long shot); power relation is constructed
enj

by vertical camera angle (i.e. high or low angles); involvement is constructed by

horizontal camera angle (i.e. frontal or oblique angles). Compositional meaning
nB

relates the representational and interactive meanings into a meaningful whole

through three interrelated systems: information value, salience, and framing
Joh

(Kress & van Leeuwen, 2006, p. 177). Information value is realized by the place-
ment of visual elements (e.g. top or bottom, left or right); salience deals with the
prominence of visual elements, through size, sharpness of focus, color contrast,
-

and so on; framing is concerned with the connection between visual elements.
In what follows, we discuss how this social semiotic framework can explain the
ofs

construction of different types of visual metaphors.

According to Lakoff and Johnson (1980), many concepts (e.g. ‘a’) are under-
stood in terms of other concepts (e.g. ‘b’), the process of which constitutes a con-
pro

ceptual metaphor a is b. ‘A’ is termed the target domain and ‘b’ the source domain.
In cognitive metaphor theory, metaphor is classified into two broad categories:
ted

conventional and creative (Lakoff & Johnson, 1980), or inactive and active (Goatly,
1997). Conventional metaphors are those that structure the ordinary conceptual
rec

system of our culture, while creative metaphors are those which give us new un-
derstandings of our experience (Lakoff & Johnson, 1980, p. 139). For example,
cor

happy is up is a conventional metaphor based on correlations in our bodily ex-

perience and shoe is tie is a creative metaphor which offers a new perspective
Un

of conceptualizing shoes (Forceville, 1996, p. 110). Conventional metaphors, as

 The visual representation of metaphor 323

y
well as many creative metaphors, serve the purpose of understanding through

pan
domestication, a process in which abstract ideas and unfamiliar persons or events
are converted into something close, familiar, and concrete (Morris, 1993, p. 201).

Com
However, creative metaphors may also defamiliarize the target domain for rhe-
torical or decorational purposes, especially in poetry and art. In visual images,
both the target and the source of the metaphor are usually concrete objects, thus
constituting the concrete is concrete metaphor (Forceville, 2009, p. 27).

ing
Forceville’s (1996) examples mostly belong to this type (e.g. shoe is tie, popcorn
is wine, ticket is deck chair, etc.) (see also El Refaie, 2009, p. 175). However,

lish
abstract concepts can also be metaphorically represented in visual images, and
in this study, the visual realization of both defamiliarization and domestication
metaphors is investigated.

Pub
In terms of visual realization, Forceville (1996) distinguishes three kinds of
pictorial metaphors: MP1 (only the source or the target is present), MP2 (the
source and the target are present and integrated) and pictorial simile (the source
ins
and the target are juxtaposed). Forceville’s (1996) three types of pictorial meta-
phor are based on the systemic choices of spatial relations between the “meta-
am

phorical subject” (typically the target domain, that is, the primary subject) and
the “pictorial context”. From the social semiotic perspective, Forceville’s “meta-
enj

phorical subject” and “pictorial context” belong to one unified grammatical unit
in the representational meaning structure. Meanwhile, aside from representa-
nB

tional meaning, visual images also have interactive and compositional meanings,
which are important resources for the visualization of abstract concepts. In this
Joh

paper, the metafunctional resources are seen as metaphor potential and we shall
explore how they realize visual metaphors, building on Feng (2011b).
The role of context (e.g. linguistic context, discourse purpose, cultural back-
-

ground, etc.) is also acknowledged in the social semiotic approach, in this case
for the identification of the source and target domains. However, countering
ofs

Forceville’s (1994, p. 7) claim that “invoking the pictorial context helps little to
determine the order of the terms,” we argue that the structural features of repre-
sentation provide essential cues for the determination of visual metaphors. There-
pro

fore, the aim of the present study is to see what the social semiotic visual grammar
can offer in modeling the representation of metaphor.
ted
rec

3. Representational meaning and the visual construction of metaphor

cor

In this section, a framework is proposed to model how both defamiliarization

metaphors and domestication metaphors are realized in representational struc-
Un

tures.
324 Dezheng Feng and Kay L. O’Halloran

y
3.1 Defamiliarization metaphors

pan
In this sub-section, we propose a social semiotic model for the “object is object”

Com
metaphor which is the focus of Forceville (1994, 1996). As mentioned above,
Forceville’s “metaphorical subject” and “pictorial context” are seen as belonging to
one unified grammatical unit in representational meaning structures. Represen-
tational meaning in visual images is modeled in terms of processes, participants

ing
and circumstances, and each image is a configuration of choices from these three
categories. In narrative structures, the metaphorical subject relates to other ele-

lish
ments through actional, verbal, or mental processes; in conceptual structures, it
relates to other elements through relational processes in the form of taxonomic
relations (classification processes), part-whole relations (analytical processes) or

Pub
identifying relations (symbolic processes).
Defamiliarization metaphors are mainly constructed by anomaly, or un-
conventionality, of visual elements in the representational structure, in a similar
ins
manner to the colligational interpretation of metaphor in language (Goatly, 1997,
p. 111). As there is variation in the conventions associated with different process
am

structures, it is important to examine the different types of anomaly. In what fol-

lows, we shall investigate how metaphors are realized in actional, classificational
enj

and analytical processes.

In actional processes, conventional participants (i.e. actors or goals) or cir-
nB

cumstantial elements associated with certain actions are substituted by uncon-

ventional ones, with the former as the source domain and the latter as the target
Joh

domain. For example, in a car advertisement in Feng (2011a, p. 63), the car is
worn on a man’s wrist like a watch. Apparently, the car takes the place of a watch,
which results in colligational anomaly. By taking the place of a watch, the car
-

adopts its attributes, constituting the metaphor car is watch. The medium of
an action (e.g. the tools which are used to perform the action, see Halliday, 1994,
p. 154) can also be substituted. In Forceville’s (1994, p. 10) example, a person is
ofs

killing himself by pointing a gas nozzle on his head. The metaphor gas nozzle
is gun is constructed because the gas nozzle adopts the role of a gun. El Refaie’s
pro

(2003, p. 79) example in which a group of Kurdistan refugees are holding the flag
with its inscription “New Kurdistan” can also be explained with participant sub-
ted

stitution. Conventionally, the army carries the flag and claims sovereignty after
conquering a place. Here, the conventional actor is substituted by refugees, which
rec

results in the metaphor refugees are army/invaders.

In classificational processes, metaphor is constructed in two ways. First, en-
cor

tity a is an unconventional member of a category whose conventional member is

entity b. As a result, a borrows the salient features of b and the metaphor a is b is
formed. For example, in a car advertisement from Qilu Evening Paper on 19 July
Un
 The visual representation of metaphor 325

y
2008, the image shows five athletes ready to run a 100-meter race, but the middle

pan
track is occupied by a car. As a result, the car adopts the most salient feature of the
athletes, that is, fast. In Teng’s (2009, p. 198) example, where an American news-

Com
paper is put among horror books on a bookshelf labeled “horror”, the resultant
metaphor American news is like horror novels is another case of this type of
realization. Second, two entities may be put together unconventionally to form a
covert category (Kress & van Leeuwen, 2006). The formation of covert categories

ing
requires a crucial visual feature – that is, symmetry in composition, such as equal-
ity in size, framing and arrangement (Kress & van Leeuwen, 2006, p. 79). This

lish
process is similar to the visual simile in Forceville’s (1996) categorization, but the
conceptualization of two juxtaposing entities as forming an unconventional co-
vert category helps to explain the metaphorical mapping – the unconventionality

Pub
of the category alerts us to the metaphor and being members of the same category
makes the mapping of attributes possible. However, the source and target do-
mains cannot be structurally determined in this case because they are represented
ins
on the premise that they are equal, and we have to draw upon other cues like the
linguistic context and the discourse purpose. The advertisement in Plate 1 is a
am

good case in point. The minivans are juxtaposed with weight-lifting champions.
They form a covert category by being identical in number and arrangement. Since
enj

it is an advertisement for the minivan, the minivan is the target and the metaphor
thus formed is minivans are weight-lifting champions. The salient feature of
nB

the athletes, that is, strength, is mapped onto the minivans.

Anomaly in analytical processes occurs when there is an unconventional part
Joh

in the whole. This can happen in two ways. First, the unconventional part a takes
the place of the conventional part b and hence inherits its salient features. The
-
ofs
pro
ted
rec
cor

Plate 1.  Wuling minivan, from Qilu Evening Paper, July, 19th, 2008, A10
Un

(reproduced with permission)

326 Dezheng Feng and Kay L. O’Halloran

y
well-known example from Forceville (1996, p. 110), which shows a man’s torso

pan
with a suit but with the tie substituted by a shoe, illustrates this type of metaphor.
By taking the place of the tie, the shoe inherits the salient features of the tie and

Com
the metaphor shoe is tie is formed (see Forceville, 1996, p. 10 for detailed analy-
sis). However, there are also rare cases in which the substituted part is the target,
as in Forceville’s (1996, p. 123) car advertisement in which the life buoys take the
place of car tires. The metaphor formed is car tires are life buoys, in which

ing
the unconventional part is the source. This is where the structural cue of realiza-
tion contradicts the contextual cue, and we have to resort to the latter to identify

lish
the metaphor.
Second, an entity (or part of it) is superimposed on another entity (or part
of it). The superimposition may or may not change the conventional identity of

Pub
the entity. If it doesn’t, the superimposed entity becomes an unconventional part
of the whole, and as in the case of substitution, the unconventional part is the
target. However, this case differs from substitution, because the superimposed
ins
entity inherits the attributes of the whole which it forms a part. We can call this
type Superimposition 1 (S1). An example is found in Yus (2009, p. 162), where a
am

saucepan has an image of the continents of the earth superimposed upon it. The
superimposition doesn’t change the identity of the saucepan. As part of the sauce-
enj

pan, the earth inherits one of its attributes, that is, warms up gradually.
The superimposed entity may also change the identity of the original image
nB

and they together form an unconventional whole, or a hybrid, similar to the for-
mation of covert categories in classificational processes. In this case, the superim-
Joh

posing part is the source and its salient features are added to the whole. We shall
call this type Superimposition 2 (S2). For example, in an advertisement in which
a pair of butterfly wings is added to a motorbike, the salient features of butterflies
-

such as beauty and lightness are projected onto the motorbike, which produces
the metaphor motorbike is butterfly. However, this example can also be seen
ofs

as the motorbike substituting the body of the butterfly, which results in the same
metaphor. Yus (2009, p. 164) provides a similar example in which dice dots are
superimposed on a ballot box. The superimposing part is the source which lends
pro

attributes to the entity it is superimposed upon. It can also be interpreted as the

ballot box substituting the body of the dice. Either way, the ballot box borrows
ted

the features of the dice and results in the metaphor ballot box is dice, which
further stands for election is gambling.
rec

To summarize, we have examined the visual mechanisms for realizing meta-

phors in representational structures, as illustrated in Figure 1. The slanted arrow
cor

denotes realization. For example, visual anomaly in narrative structures is “real-

ized” as participant substitution and circumstance substitution. The analysis shows
Un

that the source and target domains of defamiliarization metaphors can mostly
 The visual representation of metaphor 327

y
Narrative anomaly

pan
Participant substitution
Circumstance substitution

Com
Visual anomaly Classificational anomaly

Member substitution
Unconventional covert category

ing
Analytical anomaly

lish
Part substitution
Part superimposition

Pub
Figure 1.  Visual anomaly in the representational structure
ins
be identified by examining the way they are represented (i.e., anomaly in dif-
ferent process structures). However, since metaphors are not constructed by de-
am

contextualized visual components, representational resources alone may not be

able to specify the source and the target. Moreover, the cues from the process of
enj

construction may contradict with the more explicit contextual cues of interpre-
tation. Such awareness of context makes our approach social semiotic, whereby
nB

representational anomalies are seen as resources for metaphor realization, rather

than as rigid semiotic codes. In this sense, our framework only describes the met-
aphor potential in the representational structure, without claiming that structural
Joh

anomalies are able to determine all visual metaphors independently.

-

3.2 Domestication metaphors

In this sub-section, we examine the visual realization of creative and conventional

ofs

metaphors which serve the purpose of understanding abstract concepts. We shall

call them domestication metaphors without distinguishing between creative and
pro

conventional metaphors, because the working mechanism remains the same, re-
gardless of how conventionalized the mappings are. Domestication metaphors
ted

play a significant role in the representation of abstract meaning because it is not

possible to represent abstract meaning visually without recourse to symbols, me-
rec

tonymies or metaphors (El Refaie, 2009, p. 177). However, visual images seldom
work alone in the process of domestication, and as our main focus is on visual
metaphor, we only discuss the process of visual domestication briefly.
cor
Un
328 Dezheng Feng and Kay L. O’Halloran

y
A typical strategy of domestication metaphors is that the image shows the

pan
source domain and the linguistic context specifies the target by labeling the im-
age. In social semiotic terms, they constitute a symbolic attributive process (Kress

Com
& van Leeuwen, 2006, p. 105), in which the image is the token and the verbal label
endows it with a value. In cognitive terms, the “value” is understood in terms of
the “token”. This kind of verbal-visual metaphor works in the same way as Super-
imposition 1 (S1) metaphor in analytical processes, except in this case, it is the

ing
verbal text that is superimposed. Most researchers talk about multimodal meta-
phor in this sense (e.g. El Refaie, 2003; Forceville, 2009). For example, in a cartoon

lish
in El Refaie (2003, p. 83), the image shows a fortress with the word “EUROPA” on
it. The token (i.e. the fortress) only refers to Europe because the value (EUROPA)
is superimposed on it. The metaphor thus formed is Europe is fortress.

Pub
The “value” of a visual “token” may not be explicitly labeled by linguistic text,
but is sometimes implicit in the cultural context. This is the case with conven-
tional metaphors, such as orientational metaphors in which spatial orientations
ins
(e.g. up, down) are endowed with metaphorical meanings based on our embodied
experience (e.g. happy is up) (see Lakoff & Johnson, 1980). For example, the
am

windows with lights on form an up-pointing vector in Plate 2, which echoes the
cultural practice that a person’s office, salary and benefits are adjusted to a higher
enj

level when he/she gets promoted. The visual process of “moving up” is also rep-
resented in the verbal text. The metaphorical target, or the symbolic meaning,
nB

of this visual process is clearly “becoming more powerful or wealthy”, based on

cultural knowledge. In this way, the advertisement works by implicitly linking the
Joh

car with gaining power.

-
ofs
pro
ted
rec

Plate 2.  Toyota Camry, from The Straits Times, 4th October, 2008, C2
cor

(reproduced with permission)

Un
 The visual representation of metaphor 329

y
4. Interactive meaning and the visual representation of metaphor

pan
Cognitive studies of visual metaphor mostly focus on what is in the image, instead

Com
of how the image is represented. In social semiotic terms, only representational
resources are investigated, while interactive and compositional resources remain
largely implicit. As an exception, El Refaie (2009) discusses the visual realization of
orientational metaphors in political cartoons, associating spatial orientations with

ing
concepts like power and time. However, a systematic account of the metaphori-
cal meaning of spatial orientations in visual images is not yet available. Kress and

lish
van Leeuwen’s (2006) semiotic model of interactive and compositional meaning
resources provides a comprehensive framework for systemizing such metaphors.
Building on Feng (2011a, 2011b), we discuss the visual realization of metaphor in

Pub
interactive and compositional meaning structures in Sections 4 and 5.
Interactive and compositional meaning resources construct conventional
metaphors. The mappings between the source and target domains in conventional
ins
metaphors are not based on similarities, but on correlations derived from our ba-
sic experience of the world (Lakoff & Johnson, 1980, p. 155). The interpretation
am

of such metaphors does not depend on immediate context, but on physical and
cultural experiences that are common to human beings in general or to specific
enj

cultural communities (Lakoff & Johnson, 1980, p. 14). Therefore, to prove the
validity of the conventional metaphors realized by interactive and compositional
nB

resources, we need to provide their experiential bases.

Interactive meanings include contact, social distance and subjectivity. Accord-
Joh

ing to Kress and van Leeuwen (2006), contact is realized by gaze, social distance
by shot distance and subjectivity by camera angle (see also Dyer, 1989; Messaris,
1994). Since gaze and camera angle converge in most cases (i.e. gaze normally
-

converges with front angle and absence of gaze with oblique angle), we shall only
discuss the resources for social distance and subjectivity, under the term camera
positioning.
ofs

From a cognitive perspective, the relation between camera positioning and

interactive meaning is the metaphorical mapping between the source domain and
pro

the target domain (Feng, 2011a). This mapping can be considered as a master
metaphor which entails all the sub-mappings between camera positioning and in-
ted

teractive meaning. In this way, Kress and van Leeuwen’s (2006) descriptive gram-
mar is reformulated as a conceptual metaphor system that is visually realized, as
rec

shown in Figure 2.
To prove the validity of the metaphor system, we need to provide experiential
cor

bases for the mappings. The metaphorical meaning of camera positioning is pre-
mised on the iconic nature of visual images. That is, shot distance reproduces the
Un
330 Dezheng Feng and Kay L. O’Halloran

y
Image-viewer relation is camera positioning

pan
Com
Social distance is Power relation is Involvement is
shot distance vertical angle horizontal angle

ing
Close relation is close shot Image power is low angle Involvement is frontal view
Equality is eye-level angle
Distant relation is long shot Viewer power is high angle Detachment is back view

lish
Figure 2.  The visual realization of metaphor in interactive resources (Feng, 2011b, p. 27)

Pub
structural features of physical distance in real life and camera angle reproduces
features of the ways we look at and interact with people. The basis of the mapping
ins
between physical distance (hence shot distance) and social distance is well estab-
lished in the study of proxemics (e.g. Hall, 1969) and will not be elaborated here.
am

The mapping between image-viewer power relation and vertical camera angle is
based on the structural features of real-life situations in which we “look up” to
enj

powerful people and “look down” upon weak people (Messaris, 1994, p. 9). The
mapping between involvement and horizontal camera angle is based on real life
nB

situations where we face the person we want to interact with and gaze at him/her,
and turn our face (gaze) away if we don’t want to interact.
Through these experiential bases, it can be argued that these metaphors do
Joh

exist and are conventionalized in our ordinary conceptual system. However, these
conventional or default interpretations of camera positioning may be overridden
-

by other factors in specific contexts. For example, Dick (2005, p. 53) points out
that sometimes film scripts require a high or low angle shot for the sake of consis-
tency rather than for symbolism. For this reason, social semiotic interpretations
ofs

are often criticized for being too rigid, while in reality the connections are fluid
and subject to change. From the cognitive perspective, this is because certain se-
pro

miotic choices (e.g. low angle) are not motivated by the default experiential basis,
but by other factors (e.g. intertextual and discursive consistency). In such cases,
the overriding factors are usually more salient and point to one specific interpre-
ted

tation. In the social semiotic approach, we do not consider camera positioning, as

well as composition, as rigid semiotic rules, but as resources for making meaning.
rec

Ambiguity may arise as a result, particularly as metaphors are by their very nature
open to more than one interpretation (El Refaie, 2009, p. 182).
cor

In the corpus of 100 print car advertisements, 67% uses high camera angle,
which suggests the advertisers’ intention to build consumer power (Feng, 2011a).
Un
 The visual representation of metaphor 331

y
However, low camera angle is sometimes used to construct product superior-

pan
ity, for example, in the case of expensive cars such as Mercedes-Benz and BMW.
Viewers look up to the cars (and the characters), as if they are superior. Such ad-

Com
vertisements thus persuade viewers by subliminally guide their interpretation of
the car as a symbol of high status.

ing
5. Compositional meaning and the visual representation of metaphor

lish
Lakoff and Johnson (1980, p. 126) point out that linguistic forms are endowed
with content by virtue of spatial metaphors. This observation is certainly appli-
cable to visual images where space plays an even more important role across a

Pub
larger number of dimensions. For example, visual semiotic resources include
the spatial positioning of different elements, their relative size, and the distance
between them. These semiotic resources construct compositional meanings of
ins
information value, salience and framing (Kress & van Leeuwen, 2006). The infor-
mation values of given/new, ideal/real and important/unimportant are realized by
am

the spatial orientations of left/right, up/down and central/marginal respectively.

Salience and framing are not abstract concepts and will not be discussed, but the
enj

size and the distance between elements are included in the visual metaphor sys-
tem, as shown in Figure 3.
nB

Similar to interactive meanings, visual compositional meanings are also de-

rived from our embodied experience. Given is left/new is right is based on
Joh

the experience that in most cultures, people write and read from left to right, so
we take the left as given information and the right as new. In ideal is up, “ideal”
has two different but related entailments, that is, desirable and unrealistic (Feng,
-

2011a, p. 59). Desirable is up is synonymous with the well-established metaphor

good is up (Lakoff & Johnson, 1980) and will not be further explained. Unreal-
ofs

istic is up uses a different sense of “up” – that is, high. It is difficult or unrealistic
to get things that are too high (e.g. stars). Therefore, ideal things, while desirable,
pro

Information value is Time is Importance is Social closeness is

ted

spatial position space size physical closeness

rec

Given is left Ideal is up Important is central Important is foreground

cor

New is right Real is down Unimportant is marginal Unimportant is background

Un

Figure 3.  The visual realization of metaphor in compositional resources

332 Dezheng Feng and Kay L. O’Halloran

y
may be unrealistic. Real is down is based on the same experience as unrealis-

pan
tic is up. The association between central and important is so conventionalized
that “important” has become a lexical meaning of “central”. It may arise from our

Com
biological composition whereby the most vital organs (e.g. heart and lungs) are
located near the center of our bodies (Goatly, 2007, p. 40). In terms of foreground/
background, Feng (2011a, p. 70) explains their meanings in relation to the notion
of “depth”, which is “the distance between the viewers’ eyes and any point in the

ing
visual field” (Messaris, 1994, p. 51). The foreground is perceived as nearer to the
viewer than the background. Our biological feature of vision results in different

lish
visual impacts of the objects at different distances: we notice what is in the fore-
ground first (most likely for reasons of survival) and take it as more important
than that which appears in the background.

Pub
Aside from information value, spatial orientations may also construct other
abstract concepts. Left/right and foreground/background orientation can also
represent the concept of time, resulting in the general time is space metaphor
ins
(El Refaie, 2009, p. 179; Lakoff & Johnson, 1980). Taking left/right orientation as
an example, it includes the sub-mappings of past is left/present is right or
am

present is left/future is right based on our experience that the information

to the left is processed before the information to the right in many cultures. How-
enj

ever, if human beings are in the image, then their front represents future and their
back represents past (El Refaie, 2009, p. 179).
nB

The other two conceptual metaphors we identify are importance is size and
social closeness is physical closeness. El Refaie (2009, p. 176) notices that
Joh

the size of objects in an image does not just construct salience, but also impor-
tance. The mapping is based on the association between physical power and social
power. Goatly (2007, pp. 35–39) provides a detailed study on its realization in both
-

language and buildings. The social closeness is physical closeness metaphor

states that the physical distance between visual participants is perceived as an in-
ofs

dex of their social distance. It is similar to the mapping between shot distance and
image-viewer closeness, and it reproduces real life situations more directly.
Aside from conceptualizing abstract concepts, in visual media such as adver-
pro

tisements, compositional resources also construct implicit meaning which influ-

ences readers’ attitudes. For example, in Plate 3, the car image is salient because
ted

of its large size and central position, and is therefore perceived as the most im-
portant object. The price, in contrast, is positioned at the bottom left with a small
rec

font size and thus is interpreted as real, given and unimportant (i.e. not the focus
of attention). The price is strategically downplayed because it is high, and is not
cor

the selling point. This is manipulative because for many people, price may be the
most important information.
Un
 The visual representation of metaphor 333

y
pan
Com
ing
lish
Plate 3.  Alfa-Romeo, from The Straits Times, 4th October, 2008, C8
(reproduced with permission)

6. Conclusion Pub
ins
As a new development of the conceptual metaphor theory, the working mecha-
am

nism of visual metaphor needs further exploration. Complementing current ap-

proaches which attribute the understanding of visual metaphor to our cognitive
enj

capacity and situational/cultural context, this study argues that the structural
features of visual images themselves play an essential role in the construction of
nB

visual metaphor. Visual representations of metaphor are modeled with respect to

the metafunctions of visual images, namely, the representational, interactive and
compositional meaning structures. It is found that defamiliarization metaphors,
Joh

which include those in both Carroll’s (1996) and Forceville’s (1996) definitions,
can be explained by colligational anomalies in representational structures. Vi-
-

sual representational resources may also serve to domesticate abstract concepts,

either in a creative or conventional way, where the metaphor is constructed by
superimposing a verbal “value” to a visual “token”. Interactive and compositional
ofs

resources, whose meanings are derived from correlations in our physical and cul-
tural experience, realize conventional metaphors. Through the analysis of car ad-
pro

vertisements, this study also demonstrates how the visual resources which invoke
metaphorical interpretations are exploited as tools of persuasion.
We conclude that the social semiotic framework can provide a comprehen-
ted

sive account of the visual realization of both creative and conventional meta-
phors. Meanwhile, the approach also offers a cognitive explanation of how visual
rec

resources like camera positioning and composition acquire meanings. However,

as the first step toward the application of semiotic theory in cognitive studies, the
cor

analysis is limited in depth and comprehensiveness. More convincing conclusions

should be based on the systematic analysis of a large corpus of data. Nonetheless,
Un
334 Dezheng Feng and Kay L. O’Halloran

y
this study has demonstrated that the integration of social semiotics and cognitive

pan
metaphor theory is significant for the understanding and explanation of visual
semiosis. Therefore, we conclude with the hope that these two theoretical ap-

Com
proaches will be combined in further explorations of multimodal discourse.

Acknowledgement

ing
The research for this article was supported by the Interactive Digital Media Pro-

lish
gram Office (IDMPO) in Singapore under the National Research Foundation’s
(NRF) Interactive Digital Media R&D Program (Grant Number: NRF2007IDM-
IDM002-066).

References
Pub
ins
Carroll, N. (1996). A note on film metaphor. Journal of Pragmatics, 26(6), 809–822.
am

Dick, B. F. (2005). Anatomy of film. Boston, Mass: Bedford/St. Martins.

Dyer, G. (1989). Advertising as communication. London: Routledge.
enj

El Refaie, E. (2003). Understanding visual metaphor. Visual Communication, 2(1), 75–95.

El Refaie, E. (2009). Metaphor in political cartoons: Exploring audience responses. In
nB

C. Forceville & E. Urios-Aparisi (Eds.), Multimodal metaphor (pp. 173–196). Berlin:

Mouton de Gruyter.
Feng, D. (2011a). Visual space and ideology: A critical cognitive analysis of spatial orientations
Joh

in advertising. In K. L. O’Halloran & B. Smith (Eds.), Multimodal studies: Exploring issues

and domains (pp. 55–75). London: Routledge.
Feng, D. (2011b). The construction and categorization of multimodal metaphor: A systemic
-

functional approach. Foreign Language Research, 1, 24–29.

Forceville, C. (1994). Pictorial metaphor in advertisements. Metaphor and Symbolic Activity,
9(1), 1–29.
Forceville, C. (1996). Pictorial metaphor in advertising. London: Routledge.
ofs

Forceville, C. (2009). Nonverbal and multimodal metaphor in a cognitivist framework: agendas

for research. In C. Forceville & E. Urios-Aparisi (Eds.), Multimodal metaphor (pp. 19–42).
pro

Berlin: Mouton de Gruyter.

Goatly, A. (1997). The language of metaphors. London: Routledge.
Goatly, A. (2007). Washing the brain: Metaphor and hidden ideology. Amsterdam: John
ted

Benjamins.
Hall, E. (1969). The hidden dimension. London: Bodley Head.
Halliday, M. A. K. (1994). An introduction to functional grammar. London: Arnold.
rec

Kövecses, Z. (2002). Metaphor: A practical introduction. Oxford: Oxford University Press.

Kress, G., & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd ed).
cor

London: Routledge.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press.
Un

Messaris, P. (1994). Visual literacy: Image, mind and reality. Boulder: Westview Press.
 The visual representation of metaphor 335

y
Morris, R. (1993). Visual rhetoric in political cartoons: A structuralist approach. Metaphor and

pan
Symbolic Activity, 8(3), 195–210.
O’Halloran, K. L. (2005). Mathematical discourse: Language, symbolism and visual images.
London: Continuum.

Com
O’Toole, M. (2010). The language of displayed art. (2nd ed). London: Routledge.
Yus, F. (2009). Visual metaphor versus verbal metaphor: a unified account. In C. Forceville &
E. Urios-Aparisi (Eds.), Multimodal metaphor (pp. 147–172). Berlin: Mouton de Gruyter.

ing
Authors’ address
Dezheng Feng

lish
Room AG 422, Department of English
The Hong Kong Polytechnic University

Pub
Hung Hom, Kowlong, Hong Kong
Phone: +852 6580 8797
will.feng@polyu.edu.hk
ins
Kay O’Halloran
Multimodal Analysis Lab, Interactive & Digital Media Institute
am

9 Prince George’s Park, National University of Singapore

Singapore 118408
enj

Phone: +65 6516 6550

kay.ohalloran@nus.edu.sg
nB

About the authors

Joh

Dezheng Feng, PhD, is Research Assistant Professor in the Department of English, The Hong
Kong Polytechnic University. His research interests include the critical analysis of multimodal
discourse, social semiotic theory and cognitive linguistics. His recent publications include
-

“Representing emotion in visual images: A social semiotic approach” in Journal of Pragmatics

and “Intertextual voices and engagement in TV advertisements” in the journal Visual Com-
munication.
ofs

Kay O’Halloran is Director of the Multimodal Analysis Lab in the Interactive Digital Media In-
stitute (IDMI) and Associate Professor in the Department of English Language and Literature
pro

at the National University of Singapore. Her main research areas include a social semiotic ap-
proach to multimodal discourse analysis with a particular interest in mathematics and scientific
texts, and the development of interactive digital media technologies for multimodal analysis of
ted

(multimedia) data. Further information is available at http://multimodal-analysis-lab.org/.

rec
cor
Un

View publication stats