AFFECTIVE INTERACTION AS EXPERIENCE

Affective Interaction as experience: Employing Immersive

Interaction Design to influence a User-Directed, audiovisual

Dionysian experience.

by Moira Zahra

MA Digital Arts Online (Visual Arts)

23 January 2011
AFFECTIVE INTERACTION AS EXPERIENCE

Aims and Objectives

“ To argue for experience as a horizon amounts to defining an attitude whose relationship

with reality is open ended.”

(Ardenne, 2004: 10)

“Changed, mind includes even itself. Unchanged, nothing gets in or out.”

John Cage

For my project and research, I will be delving into subjects that I had never explored

previously, such as certain aspects of human psychology, experimental sound, as well as

Processing. Needless to say, I will be learning as I go along, which is why I want to approach

this project with a fresh mind. I will not be looking for an answer to a specific question; I will

merely be creating an interactive experience for the user, with the starting point being the

subject of dreams, the unconscious, intoxication; a distorted reality. As much as possible, I

want to try and take a coexisting role as an artist rather than a controlling one, in order to have

a project that is more open to that which is unintentional and unexpected.

According to Harpers Magazine, the Dionysian experience is “Governed by emotions and

particularly passions, sometimes whipped to a self-destructive frenzy of excess.” Harpers

Magazine (2009)

My Aim at the moment is to create a Dionysian-inspired interactive space, through an

application and immersive installation, which will actively engage the user. In essence, I want

the audience to be able to experience visuals and sound that can typically be felt when under

the influence of hallucinatory substances, for instance. This would be done through a

continuous Human-computer interaction where audio and visuals (animated illustrations and
AFFECTIVE INTERACTION AS EXPERIENCE

video) change unpredictably according to what the user chooses.

My theoretical research will therefore, explore to what extent (if at all) an affective interactive

installation can transport the viewer into this type of experience. This will be done through an

evaluation of the installation and the program. In addition to this, through my theoretical

research, I will also be investigating affective and emotional interactive design, art as an

experience and the influences and effects of sound in interactive installations.

Objectives:

Objective 1: Develop strong contrasting visual concepts and ideas through illustrations and

video.

Objective 2: Become familiar with Processing (through tests and research) in order to create

digitized visuals and effects that will create a sharp contrast with traditional techniques.

Objective 3: Carry out extensive contextual research to assist in the creation of visuals and

examine the response of the audience to other similar installations.

Objective 4: Based on research and resources; effectively combine experimental audio pieces

with visuals

Objective 5: Develop an interface that will unexpectedly and continuously transform

according to the user’s decision.

Objective 6: Put together an installation.

AFFECTIVE INTERACTION AS EXPERIENCE

Objective 7: Record the response of the audience’s experience through continuous testing, the

installation and application.

Context [Including Historical, Contemporary and Theoretical Contexts]

My context will be based on the Projective theory, since my research will guide the project I

will undertake.

I will be looking at Immersive Installation Arts by Bonnie Mitchell, Elaine Lillios and Greg

Cornelius. Their installations are put together using audio and visuals, and focus on

introspection, tranquility, transformation, confrontation, enlightenment and self-actualisation.

The audience is bound to feel uncomfortable at first, but eventually finds the experience

‘strangely compelling’.

Toshio Iwai invites the audience to play and choose own visuals and music. In one of his

projects he created a virtual score to trigger keys of a piano, which produce projections of

computer-generated images.

In Purbeck Light years, Jeremy Gardiner combines traditional with digital art, as well as

immersive virtual reality.

Lori Varga is an experimental video artist. Her live work, Beyond the Frames of Light and

Strange Sounds is transmitted onto four projectors, which is something I’m looking into

creating for my installation.

Bill Viola is considered to be one of the most important New Media artists of our time. His

works focus on human experiences such as birth, death and aspects of consciousness.
AFFECTIVE INTERACTION AS EXPERIENCE

I will be looking into film directors such as Michel Gondry and David Lynch, for the

unexpected structure and surreal experience of their films. Michel Gondry includes several

aspects of traditional media particularly in the film ‘Science of Sleep’. David Lynch on the

other hand, presents the audience with darker, gloomier films; these are features I’d like to

make use of in certain parts of my project.

For historical contexts, I will be looking at art and literary movements such as surrealism,

dada, preliminary video art, abstract expressionism and the beat generation. The main Key

Figures I will be researching are Franz Kline, Adolph Gottlieb, Frida Kahlo, Brion Gysin and

Egon Schiele.
AFFECTIVE INTERACTION AS EXPERIENCE

I will be taking a closer look at the beat generation even for my theoretical context. This

short-lasting movement was based on the idea of a Dionysian notion. One of the key members

of the Beat Generation, Jack Kerouac expressed that the only counterculture there is, is indeed

a Dionysian Movement.

Chris Cunningham will be an essential key figure for my contemporary context. I have been

following his works with Aphex Twin for quite some time and the elements of fascination and

surprise are never missing in his work. Like David Lynch, his works are mostly on the dark

side, therefore more associated with the concept of a Dionysian culture.

As inspiration for my traditional illustrations, I will be looking at works of Leigh J. Mc

Closkey, whose art works are based on history, religion, mythology, esotericism as well as

quantum physics and the nature of consciousness. Leigh J. Mc Closkey illustrated a cover for

musician Steven Ellison (known as Flying Lotus in the music industry) which is how I came

across his fascinating work. I will be exploring works of Mtendre (known as Teebs) who is

also a vinyl cover illustrator, and tends to combine a mixture of traditional and digital art. The
AFFECTIVE INTERACTION AS EXPERIENCE

music of Steven Ellison as well as the works of Leigh J. Mc Closkey are typically inspired

from the drug, DMT, which again ties with the whole idea of a Dionysian experience.

I have recently come across the works of Christian Boltanski, who, amongst other things, is

an installation artist. His installations focus on death, memory and loss and are constructed

using a vast selection of media, such as found objects, photography, projections and video.

The heart of his work lies in the creation of a spatial dimension.

I will be investigating a number of experimental audio artists and musicians such as La Monte

Young, Charlemagne Palestine, Erik Satie and John Cage, as well as less experimental

musicians like Eluvium and John Hopkins. The latter’s music generally offers a very

bittersweet experience to the listener, therefore it might be an interesting addition to the

installation. John Cage, on the other hand, will not be researched solely for his music, but also

for his philosophies and theories.

For the Processing part, I will be looking at a number of artists on

http://processing.org/exhibition/

Methodology
AFFECTIVE INTERACTION AS EXPERIENCE

Research methods

Overview of approach. My research will be based on a combination of both

qualitative and quantitative methods and case studies, however my core method will be a

qualitative flexible structure that is susceptible to change and evolve depending on the results.

As Maxwell states, “Design in qualitative research is an ongoing process that involves

“tracking” back and forth between the different components of the design, assessing the

implications of goals, theories, research questions, methods, and validity threats for one

another.” (2005, p.3). There will not be a list of steps that I will follow to get to a particular

goal, but the focus will be mostly on “interconnection and interaction” Maxwell (2005, p.3).

The reason I want to incorporate quantitative research methods, is so that the weaknesses of

the qualitative methods are “compensated by the counter-balancing strengths” of the

quantitative.” Jick (1979, p.604) This process has been identified as Triangulation by Denzin

(1978, p.291) who describes it as “the combination of methodologies in the study of the same

phenomenon.”

Jick(1979, pp.603,204) continues to say that the use of triangulation may not be limited to just

researching the same subject from different perspectives, “buy also to enrich our

understanding by allowing for new or deeper dimensions to emerge.” Since my project is

multi-faceted and is based on the idea of experience, I believe that using both research

methods will provide the best results.

Data Collection.

Qualitative Tests. Most of the data collection will be taking place through a series of

case studies at the college where I teach at; which is the Malta College of Art, Science and

Technology in Mosta, Malta. I will try to be allocated a small room with minimal light, where

I can carry out my tests.

AFFECTIVE INTERACTION AS EXPERIENCE

The first tests will be carried out using projected visuals and audio with no interaction. I will

invite students and lecturers to stand in a specific area of the room for about fifteen minutes

(one person at a time). Certain persons will be given a recorder to document what he or she is

experiencing during this time. Additionally, there will also be a hidden camera aimed at the

area where the person is standing; recording their movement, facial expressions and general

reactions. The reason I want to use a hidden camera is that I do not want the viewer to be

aware that they are being filmed, since they will not act natural. Handing them the recorder

will also alter their natural reactions, but will give me an idea of what the visuals are

transmitting to each and every person. Therefore I will only allocate recorders to some of the

viewers.

I will be repeating the same tests when I include interaction with the visuals.

Quantitative Tests. I will be constantly uploading the visuals on my blog and sharing

the post link on social network sites and by email to gather reponses from a wider number of

individuals. These will be accompanied by a number of short questions in the blog posts or in

the email.

Data Analysis. My project is meant to be an experience so if the emotional

response of the viewers who are being tested is one of disinterest, then it will be ineffective

and therefore must change or be updated. I’m looking for strong emotional responses from the

viewers; immersion and engagement. The inclusion of interaction aims to further enhance the

experience; hence it needs to be smooth and effortless. If the viewers find problems with

interacting, it will have to be modified. The idea of interaction is meant to add curiosity,

unexpectedness and surprise to the project; these are all keywords that might lead to

frustration if not utilized appropriately, however through continuous testing I hope to achieve
AFFECTIVE INTERACTION AS EXPERIENCE

just the right balance.

Interpretation. I will know if the tests are working when my viewers start to find the

experience appealing and addictive. Total success will be achieved if the viewers also start to

experience a rise of adrenaline. As I said earlier however, my project is open to an

unintentional outcome, so if unexpected emotions and responses are gathered through the

experience, they will not be listed as unsuccessful, but will be recorded and investigated.

Expected result. I am aware that it will be quite difficult to evoke strong emotional

responses from the people that I will be testing. I am expecting quite unmoved reactions in the

first few tests, however I am hoping to be able to update my project, find the areas that need

improvement, and finally achieve successful results.

Practical methods

My project will consist of quite a large variety of practical methods. For the first part of my

project, I will be scanning traditional illustrations and digitising them in After Effects. The

reason I want to start off by using traditional illustration is that they present no particular

restrictions. I want the first ideas to be raw and hand-drawn. Through the digitisation process

I aim to detach the user from the real world by using depth on otherwise flat scanned

illustrations. A variety of other effects will be researched and put to use, to enhance the user

experience. I will be using After Effects because it allows for ample flexibility for the artist,

and yet provides means of further experimentation through the use of expressions.
AFFECTIVE INTERACTION AS EXPERIENCE

Recent Experiment carried out in After Effects using scanned Water colour and 3D Layers

I shall also be including some short video clips to break apart the consistency of drawn and

generated images. Moreover I shall be including mathematically created visuals by using

Processing. The combination of mixed traditional and digital media is all part of an aim to

bring together subjects that are familiar to the user, with elements that are unexpected. This is

what will expectantly lead the user into an immersive dream state. I plan to generate computer

interaction using Flash, since it is one of the best programs that can support both visuals as

well as interaction. The implementation of interaction is one of the last phases of my project,

as well as one of the most important stages of my project. It is therefore open to change and

develop.

My theoretical research will assist my practical experimentation by forming a solid

foundation based on outcomes and reflections of previous artist works. I want to see how key

figures attempted to captivate the audience through euphoric experiences, and investigate

what methods they have used; which ones were successful, and why they were successful.

Even though previous works of art will play an imperative role in my research, I want to try to

keep my mind open to new ideas, in order for my project to evolve.

AFFECTIVE INTERACTION AS EXPERIENCE

I will also be conducting research on affective interaction and emotional design. I consider

this research to be an essential part of my theoretical research, since I would not be able to

produce an installation which has a real effect on the human emotions, without first

understanding the human psychological reactions to certain visuals and sounds.

Both my practical experiments as well as my theoretical research will be recorded on my

blog.

Outcomes

I want my project to be unanimously available, for that reason I want to share it online in the

form of an application. However the true outcome of the project is the experience of the

affective interaction, which will be in the form of an installation. Presently, for my final

presentation I am aiming to have a dark room with a computer and a projector where a

member of the audience can play and lose his/herself in the interactive space. The projections

might also be on 3D surfaces.

The enlarged screen (projected) would hopefully give the individual a more authentic and

surreal experience. I want the space to cater for as many human senses as possible in order for

the user to feel more enthralled.

Work Plan
AFFECTIVE INTERACTION AS EXPERIENCE

The major stages of my work plan are highlighted in yellow above.

I will dedicate the first year to creating visuals, learning Processing and conducting contextual

research. In the second year I will be collecting audio material, putting together visual and

audio and developing the interface. The final task will be the preparation of an installation to

evaluate the results of my project. Continuous testing of the installation will also be carried

out.

The visuals will be anything from scanned illustrations and paintings to digitally generated

pieces. These might continue in the second year if I am not happy enough with the ones I

produce in the first year. For my theoretical research, I am aiming to finish most of the

reading and findings in the first year, and dedicate the second year to writing and formulating

arguments and analysis.

AFFECTIVE INTERACTION AS EXPERIENCE

As shown above, most stages will blend into one another, especially when it comes to the

creation and testing of material; this will be an ongoing process that will last until the

execution of the installation.

Bibliography

Rush, M. (2007). Video Art, Second Edition. London: Thames & Hudson.

Colson, R. (2007). The Fundamentals of Digital Art. Switzerland: AVA Publishing.

Ardenne, P., Beausse, P., Goumarre. (2004). Contemporary Practices: Art as Experience.

Paris: Dis Voir.

Dewey, J. (2009). Art as Experience. Perigee Books.

Knight, B. (1996). Women of the Beat Generation: The Writers, Artists and Muses at the

Heart of a Revolution. U.S: Conari Press.

Shultis, C. (1998). Silencing the Sounded Self: John Cage and the American Experimental

Tradition. York, Pennsylvania: Maple Press.

AFFECTIVE INTERACTION AS EXPERIENCE

Beatrice, L., Campo, A. (2007). Sound and Vision. Italy: Damiani.

Rush, M. (2005). New Media in Art (World of Art). London: Thames & Hudson.

Greenberg, I. (2007). Processing: Creative Coding and Computational Art. New York:

Springer-Verlag.

Wands, B. (2006). Art of the Digital Age. New York: Thames & Hudson.

Taylor, T. (2001). Strange Sounds: Music, Technology and Culture. New York: Routledge.

Kotsopoulos, N. (2009). Krautrock: Cosmic Rock and Its Legacy. UK: Black Dog Publishing.

Moggridge, B. (2007). Designing Interactions. US: The MIT Press.

Web:

http://immersiveinstallationart.com/

http://museum.doorsofperception.com/doors1/content.html

http://www.independent.co.uk/arts-entertainment/art/features/nam-june-paik-first-video-artist-

in-retrospect-2159047.html

http://www.designinginteractions.com/

http://www.pixelsurgeon.com/interviews/
AFFECTIVE INTERACTION AS EXPERIENCE

http://harpers.org/archive/2009/07/hbc-90005412

http://www.biothing.org/

http://theprobotics.blogspot.com/

http://www.co-de-it.com/wordpress/

http://www.playusdesign.com/

Magazines:

Vale, V. (1982). Re/search no.4/5 – William S Burroughs, Throbbing Gristle, Brion Gysin.

San Francisco: V/Search

Coyne, P. (2003). Communication Arts Interactive Annual 9. Menlo Park, CA:

Communication Arts

Coyne, P. (2006). Communication Arts Interactive Annual 12. Menlo Park, CA:

Communication Arts

Coyne, P. (2007). Communication Arts Interactive Annual 13. Menlo Park, CA:

Communication Arts

Paper and Workshops:

AFFECTIVE INTERACTION AS EXPERIENCE

Wood Holman, D. (2005). The Death of Dionysos: Formative Experience and Human

Anatomy in Wilhelm Meisters Lehrjahre. Retrieved from:

http://etd.library.vanderbilt.edu/available/etd-04232005-

033746/unrestricted/DWHolman.PhD.pdf

Sundström, P., Ståhl, A. & Höök, K (2006). A User-Centered Approach to Affective

Interaction. Retrieved from: http://www.sics.se/~kia/papers/kina.pdf

Reynolds, C. & Picard, R. (n.d.). Designing for Affective Interactions. Retrieved from:

http://hd.media.mit.edu/tech-reports/TR-541.pdf

McGooking, D. & Brewster, S. (2006). Haptic and Audio Interaction Design – First

International Workshop, HAID 2006 Glasgow, UK, August/September 2006 Proceedings.

Germany: Springer.

Oakley, I. & Brewster, S. (2007). Haptic and Audio Interaction Design – Second

International Workshop, HAID 2007 Seoul, South Korea, November 2007 Proceedings.

Germany: Springer.

Pirhonen, A. (2008). Haptic and Audio Interaction Design – Third International Workshop,

HAID 2008 Jyväskylä, Finland, September 2008 Proceedings. Germany: Springer.

Interview:

New Media Artist, Saygin Soher http://www.sayginsoher.com/

AFFECTIVE INTERACTION AS EXPERIENCE

References

Ardenne, P., Beausse, P., Goumarre. (2004). Contemporary Practices: Art as Experience.

Paris: Dis Voir.

Maxwell, J.(2005). Qualitative research design - An interactive approach – second Edition.

Sage Publications.

Denzin, N.(1978). The Research Act, 2D ed. New York: McGraw-HI.

Jick, T.(1979). Administrative Science Quarterly, Vol. 24, No. 4, Qualitative Methodology.