J Multimodal User Interfaces

DOI 10.1007/s12193-011-0074-4

O R I G I N A L PA P E R

Art at the edges of materiality

Ina Conradi

Received: 14 April 2011 / Accepted: 2 November 2011

© OpenInterface Association 2011

Abstract This paper discusses digital artwork that through

the act of creation investigates and integrates emotive sub-
jective abstract imagery in digital and traditional forms.
The areas covered are the art of experimental non-
narrative animation, 3D stereo and large-scale algorithmic-
based painting and prototyping for interactive and digitally
fabricated art installations and performances. The paper’s in-
tention is to interpret a complex body of digitally generated
artwork that attempts to integrate animation, fine art paint-
ing and scientific inquiry through artist practice as pursued
through the process of academic research. The outcome of
is collaborative works done at the Nanyang Technological
University, School of Art Design and Media. Fig. 1 Envisioning creating new pictorial space using 3D stereoscopy,
The overall aim is in better understanding of the fine experimental animation and traditional painting methodology
art process and mutual influences that various and often
opposed disciplines have in collaborative projects where
Following thematic progression from the last few years
emotion-based interaction is important.
until recent all the art works presented in this paper have in
Keywords Abstract · Art · Experimental · Painting · common digital component with the very same goal to ex-
Pictorial · 3D stereo · Animation · Installation plore alternative methods of artistic expression, using avail-
able and emerging synthetic image creation technologies
and their evocative possibilities for vivid, real and stimu-
1 Introduction lating pictorial spaces.
The works continue to pose “the questions of material
In the art work under discussion, animation and digital paint- and immaterial, concrete and digital states” in art practice,
ing methodologies investigate tangible edges, borders and and are focusing on how to translate virtual properties of
interfaces of technology, materiality and space. The tradi- digital media into tangible art outcome. This new art resides
tional concept of painted canvas is replaced by proposed im- at the edges of virtual, digital, electronic materiality empha-
mersive transformational media experiences (Fig. 1). sizing psychological, invisible and poetical experiences [1].

1.1 Origins and inspirations: about abstract art and painting

I. Conradi ()
School of Art Design and Media, Nanyang Technological Regarding his creative work, the poet and art critic, Baude-
University, Singapore, Singapore
e-mail: inaconradi@ntu.edu.sg
laire stated, “I want to illuminate things with my mind and
url: www.inaconradi.com to project their reflection upon other minds, the sensation

Fig. 2 Jackson Pollock painting expressing motion and energy of his
‘inner world’
of spiritual and physical bliss, of isolation, of the contem-
plation of something infinitely great and infinitely beautiful,
of an intensity of light which rejoices the eyes and the soul
until they swoon, to the sensation of space reaching to the
furthest conceivable limits” [2].
Abstract art remains misunderstood by the majority of the
viewing public. Most people, in fact consider it meaningless.
Yet around 1910, when groups of artists moved away from
representational art towards abstraction, preferring evoca-
tive symbolic colors to natural color, signs to perceived re-
ality, ideas to direct observation, there was never an outright
dismissal of meaning. Instead, artists made an effort to draw
upon deeper and more varied levels of meaning, the most
pervasive of which was that of spiritual [3].
The early American painting style of late 1940s and
1950s, have no uniform stylistic traits on the surface. What
brought them together is the same feverish energy and pref-
erence for the monumental scale and highly personal ex-
pression. They did not represent their emotions, but enacted
them before the canvas [4], such that for some artists, the
process of painting became the very subject of the paint-
ing [5].
Of particular inspiration are the drip paintings by Jackson
Pollock (Fig. 2). Pollock created his very own unique style
of painting by dripping, pouring, splattering, and flinging
paint onto a huge canvas on the ground. This method en-
abled him to create very dynamic paintings from which the
viewer can almost feel his action and force when looking
at them. When the viewer encounters his large scale highly
textured painting; they feel almost engulfed by it. Despite
all the frenzy in his work, Pollock described that when he is
painting, he has a general notion as to what he is doing. He
J Multimodal User Interfaces
Fig. 3 Viewer immersing oneself into animated projected painting
can control the flow of paint. “There is no accident, just as
there is no beginning and no end.”
Today the same concept of ‘being and becoming part of
the painting’ can be experienced through existing technol-
ogy on an immersive level. New emotive and tangible con-
templative space-awareness is created though phenomenal
nature of the space formed between the observer and the pic-
ture plane, between the viewer and the viewed, using multi-
ple disciplines and interactive scenarios (Fig. 3).
1.2 Experimental animation
The evolution of abstract painting is simultaneous with that
of moving pictures. In many ways, developments in film
and digital technologies provided new approaches and op-
portunities to achieve the goals of the 20th Century Abstract
Painters. The essence of which is not for the viewer to un-
derstand it, but to experience it.
Russian painter Wassily Kandinsky believed that colors
and lines can be used like music to create moods without
a subject. He discussed the problem of de-materializing the
picture plane, urging artists to transform it into a sensation
of indefinable space so that the spectator might experience
the extension of “the dimension of time” [6].
With the use of video and computers, it becomes possi-
ble to directly link abstract forms and colors with their own
musical orchestration [7]. Technology has provided oppor-
tunities to imagine entirely new approaches to non-objective
forms and to liberate the imagination of artist. While many
of the Abstract Expressionists and Action painters painted
large canvases in order to encompass the viewer, film and
animation projection makes it possible to engulf the viewer
in cinematic fields.
Abstract painting and the moving image are very much
related and the conceptual approach used in painting when
applied to computer animation creates new imaginative aes-
thetics out of non-objective moving forms. Unlike still paint-
ings, moving images have the added dimension of time. The
painted composition is constantly changing, thus requiring
additional planning and consideration on how the image will
J Multimodal User Interfaces
Fig. 4 Digital planning and visualization for installation featuring
moving animated paintings done as a large scale simultaneous pro-
jections. Titled Emotion Study, installation consists of separate screens
each corresponding to a particular emotion sequence from the film
Fig. 5 ‘Emotion Study’ Large scale animated projection on the side
of the National Gallery of Modern Art Museum Building, Inaugural
Opening titled ‘Corporeal Kaleidoscope’, Attakkalari India Biennial
2011, Bangalore, India, 2011
evolve from start until end. At any point in time, the paused
frame should be able to stand on its own as a complete and
resolved painted composition [8] (Fig. 4).
With the further introduction of other disciplines into fine
arts, painting can create new kind of pictorial space through
use of large scale projection spectacle, incorporating archi-
tectural elements as a backdrop canvas. Recently screened
at the Attakkalari India Biennial, ‘Emotion Study’ film was
projected on the side of the National Gallery of Modern Art
Museum Building as a large format artistic electric canvas
(Fig. 5).
2 About the art work
“The works will be structures but not sculptures. They
would be painted but not paintings. They would be
decorative but not decorations” [9].
Drawing on the ideas of the merging of digital painting,
avant-garde film-making and sound cultures, the work un-
der discussion aims to craft immersive, interactive and 3D
animated installations using digital images, seeking innova-
tive convergence of art and technology to transform spaces
into novel experiences. Developed works aimed to be pre-
sented in a way that echoes the forms of traditional paint-
ing: transformation of a 3-dimensional space (‘canvas’) via
the projected film (‘brush strokes and colors’). The result is
a unique “scape” within which the viewer would be actively
reconsidering preconceived idea of two dimensional ‘paint-
ing’. The very somewhat participatory, sensorial, individu-
alized engagement that involves viewer is goal for the arts
created. Works explored the following oversized image cre-
ation using algorithmic paint strokes, high resolution com-
puter rendering techniques, (Fig. 7), digital prototyping and
printing; imagery integrating animation and 3D stereo.
2.1 Internal External installation (Post Museum, Singapore
2008, 1000 sqm)
The following compositions were done in 3D environment
of Autodesk Maya using custom-made algorithm brushes.
(Fig. 6). Auto Desk Maya is a software commonly used
in Feature Animation for modeling and animating as well
as creating visual effects. The algorithm custom brush is
used to “paint” in 3D space and was rearranged to create
an aesthetic composition. During the working process, low
resolution renders have to be made periodically to observe
the progress and changes made to the brushes as it cannot
be fully previewed real time within Maya’s viewport. The
strokes appear in the forms of lines, which serve as are refer-
ence of its position and trajectory as opposed to an accurate
representation of the final product. Due to software limita-
tions, images generated with Maya paint effects is incom-
patible with Mental Ray renderer, and hence cannot benefit
from lighting solutions such as Global Illumination & Final
gathering.
The series of works began as a virtual planning and digi-
tal pre-visualization for public art proposals for site specific
locations in Singapore (Fig. 7). Pre-visualization of artwork
in both internal and external spaces is explored through the
construction of 3-dimensional sets and photo space manip-
ulation. Realistic pre-visualizations are achieved with 3D
package Maya and integrated renderer Mental Ray, utilizing
advanced lighting solutions and materials to achieve close to
life renders. Accurate Spatial reconstruction of the selected
locale is essential in planning and visualization of the art-
work in space. The space/artwork relationship has to pass
both aesthetic and practical considerations.
Based on these visualizations the first actual art installa-
tion was a site work titled Internal External and it was exhib-
ited in the Post-Museum Gallery, Singapore (Fig. 8). A nor-
mal method of rendering was not possible due to extreme
 J Multimodal User Interfaces

Fig. 6 Autonomous and

independent expressions of
algorithm had enabled
development of complex
dynamic patterns. (a) algorithm
preset brush, titled funMesh,
changed the look (b) with
altering attributes such as
creation, segment length, width
and texture and growth and
behaviour, displacement and
forces

Fig. 7 Public art proposal for Esplanade Exterior public sculpture

‘Natural Systems: Elixir of Redness’ tempered glass, exposed U-chan-
nel with concealed ground mounting 14 panels 1100 × 227 × 200 cm
high resolution renders 160 cm × 227 cm, at 300 dpi solution for each
panel would equate to 18899 × 26780 pixels, file size 1.41 GB

size of each composition, would exceed the limitations of

Maya. Thus a batch render script is created in notepad to in-
struct Maya to slice up the render into multiple smaller tiles;
these individual tiles would then be composited with Adobe
Photoshop in post production.
‘Internal External’ is a multilayered environment com-
bining video projection, 3D computer animation, and space
modifiable backlit digital painting. Through an integrated
design and fabrication practice, aim was in trying to cre-
ate experimental, built environments to enhance and cele-
brate the potential for social interaction through sensation
and physical engagement. Enhanced by light and augmented
through complex patterns the exhibition space was recon-
structed again in Japan into an intricate spatial environ-
ment consisting of tall backlit compositions one can travel
through and explore (Fig. 9).
In these two art gallery installations, the goal of the work
was to express an enthusiasm for emotive animation and tan-
gible fabrication processes as they relate to the built installa-
tion both physically and poetically. For the exhibit purposes,
the animation is in most cases followed with the actually
fabricated digital painting and print prototype. The aim was
to merge generated physical output based on the animated
dynamics and moving images. Depending on the available
resources, the outcomes are ranging from commercial and
Fig. 8 Internal External installation Post-Museum, Singapore, 2008.
This is an example of materializing virtual painting into print on a
tangible substrate. Installation consists of tall backlit compositions, 14
panels each 100 × 227 cm, digitally printed on UV VUTEk ® Q-S2000
back-lit polycarbonate using AVS technologies. Size of installation is
100 sg m
custom-coated 2D fabricated surfaces and or 3D sculptures.
In that manner, the works explore properties, limitations,
and site-specific scenarios associated with an image process.
While developing methods to extend the intertwined bound-
aries of image aesthetics, animation and digital code tactile
quality, an immersive experience remains main component.
Using dreams as a conceptual motif, and adopting simulated
natural phenomena such as flow and growth, the work ex-
plored organic and natural design leading the viewer thor-
ough the aesthetic impression of a dream- like state sensi-
bilities, feelings and experiences that go beyond language.
Inundated with light, color, and motion, paintings are visu-
ally translating invisible realities, the ephemeral and ever-
changing appearances of nature in flux. The overlapping and
mirrored layers of complex imagery are engaging in the rep-
etition of marking the void: experiential, spiritual and infi-
nite.
J Multimodal User Interfaces
Fig. 9 Art installation Kyoto International Textile Center, Gallery Ex,
Natural Systems: Elixir of Gold, Digitally generated still images on
UV VUTEk ® Q-S2000 back-lit polycarbonate AVS technologies,
300 × 300 × 227 cm, Kyoto, Japan, 2009
2.2 On ‘Emotion Study’, experimental animation (running
time 9.27 min, HD DVD, color, June 2010)
The idea of fundamental study on perceptions of emotion
provoking images originated within Co-Space research into
cinematic storytelling with the real-time and rendered im-
agery. At the time, Emotion Study Survey was to assist
in planning visual and behavioral representation of ani-
mated form. Production of digitally generated stills, pho-
tographs, and paintings depicting various emotional states
proceeded. The study addressed the audience’s emotional
perception towards abstract images in order to understand
viewer’s reaction and emotion when they are exposed to
non-representational images. It is based on Rosenberg and
Ekman’s studies [10], and empirical tests designed to estab-
lish a coherence between expressive and experiential sys-
tems in emotion, in which they found that there was a high
degree of temporal linkage and categorical agreement be-
tween facial expressions and self-report and an even stronger
agreement for more intense emotional events. Thus, emo-
tion theorists have utilized these results as solid evidence to
introduce the self-report and cue-review to the subsequent
emotion tests.
In another related study, the self-report method was used
to collect the viewer’s data. Forty six out of one hundred
participants fully scored all images [11]. Within these six
basic emotions, the category of Happiness gains the highest
consistency between normal viewers and creators, while the
category of Disgust follows as the second highest constant
emotion. It is inferred that the audience is able to discrimi-
nate between two polarized emotions even when they are ex-
posed to purely abstract images without any textual instruc-
tion. Researchers also run a correlation test to disclose con-
nections among different emotions, which (Fig. 10) shows a
strong correlation between Fear and Disgust as well as Fear
Fig. 10 A circumplex model of affect. A circumplex model of af-
fect illustrates of relations of feeling-related concepts that are places
in a circular order and are contrasted as: pleasure-displeasure and
arousal-sleepiness [12]
and Anger, while another pair of emotions having a corre-
lated connection are Happiness and Surprise [12].
The result indicates that similar emotions within either
category of positive or negative have a significant correla-
tion. That is, a viewer is very likely to feel frightened when
exposed to disgust images; likewise, the viewer may feel
happy when something surprises him/her in a positive way.
Although the sample size was targeting a pilot result, the
gender differences still can be seen as being among positive
and negative categories.
While male viewers are more aware of the pair of Happi-
ness and Surprise, female viewers are more sensitive about
Disgust and Fear [13].
Over a hundred still images pertaining to the six different
emotion categories: Happiness, Sadness, Surprise, Anger,
Fear, and Disgust plus neutral were used putting the results
achieved into the animations created.
For image generation research the diversity of stylistic
methods of abstract painting, painterly automatism with in-
tuitive authoring of imaginary forms, action painting, and
color field painting with a highly articulated, psycholog-
ically based use of color. Kandinsky’s theories about the
relationships between colors and emotions were also ap-
plied [14]. The resulting still frames rendered in Maya were
then further layered to achieve more expressive abstract im-
agery reminiscent of Pollock’s ‘all-over’ drip paintings and
works from the French Art Informel movement that em-
ployed a form of unconscious calligraphy to trace out the
emotions the artist wishes to express (Fig. 11).
From these still images, experimental short sequences
were developed and pieced together into a painted animated
setting. A unique style of experimental animation is devel-
oped that aims to provide the viewer with a new aesthetic
experience that is immersive and imaginative.

Fig. 11 Still images depicting
the following emotions: Fear
(on the left), Happiness (on the
right)
Fig. 12 Part 1 Gloom from Emotion. Still frames from the ‘Gloom’,
Animated paint slowly flowing down, from the top of the screen, while
its details are constantly moving, then flowing down and dissolving
away
The experimentation used Next Limit Real Flow, a fluid
dynamics simulation graphics technology that gives the ap-
pearance of streaks of light and paint (Fig. 12). Some limi-
tations of this method was also discovered, such as it could
better mimic the appearance of paint when darker colors are
used and the polygonal mesh must have more complexity
to render a convincing form. The methodology developed in
this project work around these limitations to create anima-
tion clips of a lush quality that may be put together into an
abstract composition meant for large-scale projection.
The emotion ‘sadness’ was chosen to be the starting
theme for exploration because it is associated with dark
heavy colors and slow movements which can bring out the
beauty in the evolving fluid forms. Rather than simply an-
imating the splashing of fluid or ‘paint’ that is achievable
to a limited extent with live-action movie photography, the
focus was on the creation of a main sculptural form in the
animation that would depict the targeted affective state [8].
These animated sequences are further developed to elicit
the affectation of Gloom, Torment, Fear, Loneliness, and
Lust (Fig. 12).
J Multimodal User Interfaces
3 Conclusion
The challenge for the artist today is in presenting an artwork
compatible to painting that is non-representational that de-
fines itself within a physical space, and in doing so chal-
lenges the concept of pictorial space in twentieth century.
“An effective painting of today should present its space in
such a way as to include both viewer and maker each with
its own space intact. It is not that this experience should be
literal it is simply that the sense of space projected by the
painting should seem expansive enough to include the view-
ing and the creation of that space. Concerning contempo-
rary painting today, the artist should strive to encourage a
response to the totality of pictorial space: the space within
and outside of the action of making the painting. The act of
looking at a painting should automatically expand the sense
of that painting’s space, both literally and imaginatively. In
other words, the spatial experience of the painting should
not seem to end at the framed edges or contained in box that
is the picture plane” [15].
Further networking with interactive media, science and
wide range of disciplines, painting can reassess its tools
to be reinvented again. Study and art works described in
this paper could be used both in emotion induction exper-
iments and in emotion visualization and expression in mul-
timodal interfaces. For example, the author has a collabora-
tion project where the user’s emotions such as fear, frus-
trated, sad, happy, pleasant, and satisfied could be recog-
nized from EEG in real time [13], and interpreted with art
animations depicting the corresponding emotional states.
Acknowledgements
– School of Art Design and Media, Nanyang Technological Univer-
sity, Singapore.
– Academic Research Fund Tier 1, Reference Number RG105/10,
Funding by Ministry of Education, Singapore, 2007–2010.
– National Research Foundation, Singapore, Project Title: ‘Cinemat-
ics and Narratives: creating stories within real-time visual toolsets’,
Asst. Prof. Mark Chavez, 2008–2010.
– Undergraduate Research Experience Program, URECA, Nanyang
Technological University, Singapore.
J Multimodal User Interfaces
References
1. Gramelsberger G (2010) Heavy Matter: a laboratory study view.
In: Cologne AOMA (ed) Heavy Matter ISEA2010 Ruhr. Academy
of Media Arts Cologne, Dortmund, p 29
2. Baudelaire C, Mayne J (1995) The painter of modern life and other
essays. Phaidon, London
3. Tuchman M et al (1986) The spiritual in art: abstract painting
1890–1985. Abbeville Press, New York
4. Honour H, Fleming J (2005) A world history of art. Laurence King
Publishing, London
5. Brown J, Frankenthaler H (1998) After mountains and sea:
Frankenthaler 1956–1959. Solomon R. Guggenheim Foundation,
Bilbao
6. Long RCW (1890) Expressionism, abstraction, and the search for
utopia in Germany
7. Clearwater B (2009) Abstract cinema and technology. Available
from: http://www.mocanomi.org/archive/images/abstractcinema_
onlinecatalogue.pdf
8. Conradi I, Khoo YH (2010) Reassessment of painting: fine art
painting and experimental computer animation ADM09036. In:
Proceedings of NTU URECA
9. Harrison C (1977) On color and abstract art: David Batchelor’s
recent work. In: Sculpture: contemporary form and theory. Art &
design, vol 55, p 39
10. Rosenberg EL, Ekman P (1994) Coherence between expressive
and experiential systems in emotion. Cogn Emot 8(3):201–229
11. Mark C (2010) Access survey. Available from: http://www.
visual-analysis.com/survey.html
12. Russell JA (1980) A circumplex model of affect. J Pers Soc Psy-
chol 39(6):1161
13. Liu Y, Sourina O, Nguyen MK (2010) Real-time EEG- based hu-
man emotion recognition and visualization. In: 2010 int conf on
cyberworlds, Singapore, pp 262–269
14. Kandinsky W (1977) Concerning the spiritual in art. Dover, New
York
15. Stella F (1986) Working space. Harvard University Press, Cam-
bridge