(2000)1462

Visual semiotics
The feasibility of such a domain as visual semiotics, a speciality purportedly
concerned with the investigation of all kinds of meaning conveyed by means of
the visual senses, may well be doubted: following one common interpretation, it
should be excluded by the structuralist conception according to which form, not
substance, is relevant to meaning. Yet, Saussure’s preoccupation with the
linearity of speech, as against the multidimensionality of pictures, Jakobson’s
interest in the differences between auditory and visual signs, and, more generally,
the Prague school model, according to which the receiver of the work of art is
called upon to transform a sign into a concrete perceptual object, certainly
suggest a different conception. Indeed, Lessing already was essentially
concerned with the location of visual signs in space, as opposed to the projection
of auditory signs into time. According to the Greimas school, however, all
meaning is of a kind, so not only do pictures and literature manifest the same
organising principles, but, more broadly, visuality and aurality must be taken to
be identical on a deeper level.
In recent semiotics, Kümmel (1969) has written a rather haphazard
catalogue of visually conveyed signs. The first to propound a division of all signs
according to the sense modalities which they address was Thomas Sebeok (1976).
Preziosi (1983) then ranged architecture among the several devices of visual
communication, opposing them in some respects to verbal language. According
to Fernande Saint-Martin (1987), visual semiotics comprises the study of pictorial
art, sculpture, and architecture. This means that she ignores all visual signs which
are not, in our culture, considered to be artistic (in spite of some passing remarks
on children’s drawings in other publications). In actual fact, pictures, sculpture
and architecture also are the only domains, with an emphasis on the first one,
which are covered by Groupe µ’s (1992) recent book the title of which announces
that it is concerned with visual signs.
Like Preziosi, Saint-Martin opposes visual semiosis to the verbal kind,
forgetting that verbal language may alternatively be conveyed by writing, which
can be rudimentary, as in the case of alphabetical script, or more elaborate, as in
the various kinds of ideographic writing, in some varieties of logotypes and
pictograms, and in gestural emblems. The issue is further complicated by the fact
that many types of semiosis are partly visual, and partly conveyed by other
senses, as, for instance, theatre, the cinema, dance, and even food and clothing.
Indeed, if the gaze, as has often been suggested, is really privileged in Occidental
(or even pan-human) culture, the wider semiotic domain of the “natural world”
will be largely dissolved into visual semiotics.
 To the extent that pictorial semiotics has been well-advised to turn
recently to perceptual psychology in search of its foundations, we must suppose
there to be some general organising principles of pictorial and other visual signs
which are relevant to their transmission of meaning. If so, however, it will be
necessary to distinguish those domains which are intrinsically visual in
organisation, from those in which meanings which are differently constituted are
merely secondarily conveyed by visual means.
On the other hand, from the point of view of Hjelmslevean semiotics, we
would normally not expect visuality, being a mere “substance” or even “matter”,
to determine any relevant categorisations of semiotic means. In their dictionary,
Greimas & Courtés actually claims that sense modalities, identified with the
expression substance, are not pertinent for semiotics, and this is no doubt the
reason for visuality being one of the many layers between the unique picture and
signification per se being left out of consideration in Floch’s analyses.
This type of argument is based on a confusion of the terms “substance” and
“matière”, as employed by Hjelmslev, and in their ordinary usage. In fact, the
term “matière”, to Hjelmslev, is that which is unknowable, and, as a consequence,
not susceptible of being analysed; that is, it is the residue of the analysis; and
“substance”, which, in the earlier texts, is the term used for “matière” in the
above-mentioned sense, stands, in the later works, for the combination of
“matière” and “form”. Thus, “substance”, in the early works, and “matter” later,
simply means “that which is not pertinent relative to the other plane of the sign”
(see discussion in Sonesson 1989,II.4. and 1988); it does not necessarily stand for
matter in the sense of ordinary language, that is, the material of which something
is made, or the sense modality. If the material or the sense modality turns out to
be relevant in relation to the other plane of signification, it becomes form (from
Hjelmslev’s standpoint, this is what happens in connotational language).
More importantly, the psychology of perception certainly seems to suggest
the existence of some common organisation which puts all or most visually
conveyed meanings on the same level. If all signs must also be objects of
perception, there is every reason to believe that the modality according to which
they are perceived determine at least part of their nature. This is indeed the
position taken by Groupe µ (1992: 58f), who goes on to compare this conception
to the one favoured by such linguists as Saussure, Martinet, and Bloomfield,
according to which the vocal character of language is one of its defining
characteristics. More to the point, they observe that the linearity of verbal
language is a constraint imposed on linguistic form by the characteristics of the
vocal channel by which it was once exclusively conveyed. That is, the qualities of
the visual sense modality are of interest to semiotics, to the extent that they
specify formal properties embodied in each system addressed to that particular
sense. Although Groupe µ seems unaware of it, Hjelmslev (1953) does not reason
differently when he posits different “forms” for written and spoken language.
If, however, properties imposed by their mode of communication are only
some among several traits defining these signs, as is the case of the linearity of
verbal language, one may well wonder whether we are really justified in making
visuality into a subdivision of semiotics. There may, moreover, be other, perhaps
more fundamental division blocks of semiosis, of which pictures and some other
visual signs form a part, such as, for instance, that of iconicity. On the other hand,
it is conceivable that some more decisive argument could be advanced for
privileging the domain of visuality over other possible divisions.
But to the extent that there is a legitimate domain of visual semiotics, it
should comprehend much more the sacred trinity of art history, painting (to
which drawing, photography, and so on, have been assimilated), sculpture, and
architecture, of which Groupe µ, just like Saint-Martin (1987) seems to be the
victim, in spite of the promise made in the introduction (p.12ff) to ignore received
categories such as art. As soon as we leave the traditional divisions of art history
behind, this analysis has a very limited value. Thus, sculpture should be
compared semiotically similar objects like the tailor’s dummy, and the like. At one
point (p.405f), the authors are suddenly reminded of marionettes, considered as a
kind of sculpture to which movement has been added. But why not also add the
ballet dancer, whose art is certainly visual?
There are also significations which are only partly visual, such as those of
theatre communication. Others might be considered not to have an intrinsically
visual organisation, such as writing, the conformation of which depends in part
on spoken language. But all kinds of gestures and bodily postures, objects,
dummies, logotypes, clothing, and many other phenomena must be counted as
visual signs and significations. In fact, even visual perception per se supposes a
pick-up of meaning of sorts. Not only should we therefore have to face the
arduous task of determining the ways in which the various kinds of visual
semiosis, beyond those of pictures, architecture, and sculpture differ, but it also
remains to be shown that they all have sufficient properties in common to be
considered “visual signs” And we have to determine in which way pictures differ
from other, intrinsically visual signs, not only from sculpture and architecture.
The domain of visual semiotics, to the extent that it exists, remains to be
constituted.
The real issues of visual semiotics may then turn out to be still, or rather
again, those characterised by Lessing (cf. Sonesson 1988): if language is better
adapted to the rendering of temporal succession, while pictures lend themselves
more readily to deployment in space, then how is that visuality and narrativity,
as many critics of television and more recent media have suggested, concurrently
invade our culture? The interrelations of visual and auditory signs (which are not
simply verbal either) certainly deserves more serious consideration than has been
given them by those propounding the domain of visual semiotics.
Göran Sonesson

Bibliography:
Kümmel, Peter, Struktur und Funktion sichtbarer Zeichen Quickborn. Schnelle 1969
Greimas, A.J. & Corutés, J., Sémiotique. Dictionnaire raiaonné de la théorie du langage.
Paris: Hachette 1979.
Groupe µ, Traité du signe visuel. Paris: Seuil 1992.
Hjelmslev, Louis, Introduction à la discusssion générale des problèmes relatifs à
la phonologie de langues mortes. In Acta Congressus Madvigiani: Proceedings of
the Second International Congress of Classical Studies, Copenhagen 1954,
Copenhagen 1954, 101-113.
Jakobson, Roman, On the relation between visual and auditory signs. In Selected
Writings II, 338-344. Paris & The Hague: Mouton 1967.
Preziosi, Donald, Advantages and limitations of visual communication, in
Krampen, M., ed. Visuelle Kommunikation und/oder verbale Kommunikation.
Berlin: Olms Verlag, Hildesheim/Hochschule der Künste 1983, 1-34.
Saint-Martin, Fernande, Sémiologie du langage visuel. Québec: Presse de l
´Université du Québec 1987.
Sebeok, Thomas A., Contributions to the doctrine of signs. Bloomington: Indiana
University Press.
Sonesson, Göran, Methods and models in pictorial semiotics. Lund: Institute of Art
History 1988.
Sonesson, Göran, Pictorial concepts. Lund: Lund University Press 1989.

See also Icon, Iconicity, Image/Picture