FACILITATING

INTERSPECIES PLAY
Physical Computing, DIGF-1006-001
com/2009/03/nick-veasey.html
http://panopticamx.blogspot.

Mandeep Mangat
mandeep.mangat.s@gmail.com
mandeepmangat.format.com
 Introduction
FOR THIS PROJECT I WAS TASKED WITH BUILDING
AN EMPATHY MACHINE - AN INTERACTIVE
INSTALLATION THAT GENERATES THE EXPERIENCE
OF BOTH EMOTIONAL EXCHANGE AND EMOTIONAL
UNDERSTANDING OF ANOTHER IN THE PRIMARY USER.

APPLYING A MULTI-SPECIES PERSPECTIVE, I

INTERPRETED THIS PROJECT THROUGH CREATING
A CO-DESIGNED, FLUCTUATING AND PLAYFUL
EXPERIENCE BETWEEN THE USER AND ANOTHER
SPECIES.

AS WE WERE REQUIRED TO USE INPUT (SENSORS) AND

ney-at-david-richard-gallery/article_4718ff7e-aba6-53cb-ad59-b525317c6728.html
http://www.santafenewmexican.com/pasatiempo/art/exhibitionism/bryan-whit-

OUTPUT (ACTUATORS) DATA TO MAKE THE “MACHINE”

COME TO LIFE, I DECIDED ON A TOUCH SENSOR (TO
SENSE THE INTERSPECIES INTERACTIONS) AND MIDI
OUTPUT TO A DIGITAL SYNTHESIZER AS MY ACTUATOR
(ENABLING A SENSE OF PLAY THROUGH MUSIC).
 02

TABLE OF CONTENTS

Content Page

Introduction 01

Concept 04

The Final Piece 05

Prototype Development 07
Shelf 07
Electronics 13

Software Development 15
Circuit Diagram 17
Digital Synthesizer 18

Conclusion 25

References 27
 “IN THE ‘JOINT DANCE OF BEING’ CRITTERS
ARE ‘BOUNDED IN SIGNIFICANT OTHERNESS’
- A RELATIONAL CONSTITUTION OF IDENTITY
http://allthatsinteresting.com/bryan-whitney-x-ray-art

WHERE BEINGS ‘ARE NEITHER WHOLE NOR

PARTS’ BUT RATHER ARE MADE IN THE
FLESHLY SPACE OF ENCOUNTER.”
(Katherine Wright)
 04

Concept Statement

The empathy machine that I designed is inspired by the

emergence of a multi-species perspective - a discourse
that re-examines the role between Homo sapiens and non-
human lifeforms.

In the past three decades, there has been an increasing

accumulation of scientific evidence that often re-frames
non-human species as having human-like qualities, such
as: intelligence; living within social fabrics, and complex
emotional capabilities.

A multi-species perspective challenges the boundaries

between the concept of the human and the non-human
condition. It reconsiders the framework of the existing
ethics within our modern day society.

This piece encourages a playfully symbiotic interaction

between species. The application of touch (from the user)
stimulates the otherwise inhibited senses of the indoor
plants, and in response (through the touch capacitor), the
user is rewarded with audio sounds.
The Empathy Machine

The frosted white acrylic houses the plants along with the
arduino leonardo, and capacitive touch shield.
 06

Parts Used:
Arduino Leonardo,
Capacitive Touch Shield,
22 AWG Hookup Wire,
4 x Biomedical Pads,
3/8” Frosted Acrylic
Prototype Development: Shelf

In order to both avoid short circuits and allow the touch
sensor to play different sounds, I needed to keep the plants
separate from one another.
 08

Measuring the size of the pots that the plants came in, I
was able to determine a safe distance between the plants.
By using these measurements, I then sketched out the
design for the shelving unit and purchased acrylic sheeting.
Prototype Development: Shelf

In order to make the acrylic sheeting more manageable, I

reduced the size of it using the table-saw, followed by the
band-saw.
 10

Afterwards, I finalized the measurements of the cut pieces

using the wet sander.

To make the holes (that would contain the planter) in the

acrylic sheeting, I made a file on adobe illustrator and had
the sheeting cut with a laser cutter.
Prototype Development: Shelf

To prepare, I secured the corresponding joints together

using masking tape and then secured them the solvent
“res-n-bond”. Once dried, I would move onto another edge.
 12

Once the exterior of the shelf was constructed, I placed

two small acrylic panels inside the shelf to help stabilize
the micro-controller and sensor board.
In order to acquire accurate
data from the plants and avoid
damaging them, I wrapped
wire around biomedical pads
which would then stick onto
individual plant leaves.
 14

Prototype Development: Electronics

The wires would be connected to biomedical pads on one

end, and on another, the capacitive touch shield.
Software Development: Circuit Diagram

The header pins allowed me to snap the capacitive touch

shied directly on top of the arduino leonardo micro-controller.
The output sent from the plants > touch sensor > micro-
controller > laptop is in the form of MIDI data (Musical
Instrument Digital Interface). This MIDI data is read and
responded to by a digital synthesizer software.
16
Software Development: Digital Synthesizer

Leveraging collaborative reviews from

various media artists, Ableton was
frequently cited as a preferred digital
synthesizer software.

I downloaded a free trial and began by

exploring the interface. The version
I installed came with a step-by-step
instructions that introduced the
different features of the software.
18
Software Development: Digital Synthesizer

This required me to manually map the

MIDI input from each biosensor to my
chosen tracks.

While Ableton is compatible with MIDI

input, the software does not recognize
the Arduino platforms input channels.
20
Software Development: Digital Synthesizer

To set-up the software to respond to

the MIDI input, I created a Drum Rack
Track (see right side of screen shot).

Next, I selected the Drum Rack, track

8 for more information to appear at
the bottom of the screen.

I searched the pre-existing audio

samples that the software comes with
(top-left of screen shot) for gentle,
meditative sounds.

Once I found a sound file that I liked,

I would click on it and drag the file
into an empty slot into the drum rack
below.
22
Software Development: Digital Synthesizer

Once I had my audio files that I

wanted to play, I had to match them
up with the appropriate plant sensor.

Selecting the “MIDI” option on the top-

right of Ableton’s interface allowed
me to enter a new screen for manually
mapping each sensor to an audio
sound on my drum rack.

To do this, I would trigger the sensor

by touching the plant. Once done,
the touched sensor would appear as
a channel on the “MIDI Mappings” tab.
From this tab, I would select the note
that I would want to be triggered in
the Drum Rack.

For instance, the first biosensor

appeared as Channel 1, correlates to
Note D1 (Note D1 is selected in the
Drum Rack, as “Freeze, 6-FM Sitar”).
24
Conclusion

This interactive installation is intended as an inquiry into

interspecies relations, with an emphasis on technological
interventions to augment human to plant communication
through both a tactile interface and audio feedback loop. In
consideration of this, the experiences that emerge through
the designed interactions are considered a success, but as an
iterative process there is always room for further development.

While touching the plants, the users immediately developed

a greater awareness of their hands and movements in relation
to the plants, due to the real-time feedback of the digital
synthesizer. For instance: touch triggered a track; continued
touch would result in the volume of the particular track
escalating; and ending the tactile interaction would result in
the track fading out and ending. The interactions and sounds
selected resulted in a feeling of wonder. Users reported feeling
connected to the plants through the co-designed process of live-
music creation.
 26

Unfortunately, the implementation of technology was overly

apparent - and deterring from my intention of an organic
moment between species. Particularly, the reliance on an
external digital synthesizer on a laptop drew the users attention
away from the plant installation, and served as a reminder
of the technosphere. The next prototype should emphasize
greater integration of electronics, and remove the dependence
on a laptop altogether. For instance, an audio board (that is
compatible with arduino and accepts MIDI input to trigger audio
output) and speakers could be included in the plant shelf.

Through my research, I discovered that different species of plants
respond differently to touch (IE: stimulated, indifferent, or even
harmed). In future iterations, species of plants that respond well
to touch to include are: Potentilla recta, Arabidsopsis thaliana,
and Linaria vulgaris.

Lastly, larger themes to consider are that of the role of

technology to intervene in experiences and to prompt
synesthesia. As well, throughout this project I have been
reflecting on the ethical responsibilities a designer must navigate
in the process of representing and granting a “voice” to a species
unable to participate in the dialogue.
References

Due to the scope of this project within a limited time-frame, I

relied on existing tutorials to inform my process and development.
Adding electronics to plants, especially in audio form (IE. Giving
non-speaking species a voice) turned out to be a popular topic.
As well, within the same theme, I found some very interesting
resources on: fauna bio-electricity; biosensing for human to
computer interactions; creating electronic actuators that respond
to galvanic skin response; and touch sensors that create MIDI
output.

“Moss Power.” Tumblr, 2013-2016, mosspower.tumblr.com/.

Accessed on 3 Feb. 2018.

“When Plants Jam with Synths: Leslie Garcia’s Open Project

Lets Plants Talk with Sound.” Create Digital Media, 27 March
2013, cdm.link/2013/03/when-plants-jam-with-synths-leslie-
garcias-open-project-lets-plants-talk-with-sound/. Accessed
on 3 Feb. 2018.

Christie, Thomas. “Plant Sounds.” Youtube, 21 April 2015,

www.youtube.com/watch?v=VvWPT4VhKTk. Accessed on 5
Feb. 2018.

DangerousTim. “Arduino Tutorial: Capacitive Touch Sensors.”

Instructables, 13 Sep. 2014, www.instructables.com/id/How-To-
Use-Touch-Sensors-With-Arduino/. Accessed on 12 Feb. 2018.

Electronics, A2D. “Easy DIY Arduino Touch Sensor Piano.”

Instructables, 25 Jul. 2017, www.instructables.com/id/Easy-
DIY-Arduino-Touch-Sensor-Mini-Piano/. Accessed on 11 Feb.
2018.
 28
03

Hess, Der. “Biosensing for Human Computer Interactions.”

Der Hess, am medialen werkeln, 14 June 2014, blog.
derhess.de/2014/06/14/biosensing-for-human-computer-
interactions/. Accessed on 7 Feb. 2018.

Hess, Der. “Can Plants Be A Musical Instrument.” Der

Hess, am medialen werkeln, 14 June 2014, blog.derhess.
de/2013/10/08/can-plants-be-a-music-instrument/.
Accessed on 7 Feb. 2018.

Howse, Martin. “Biologic Sensing/Systems Research.”

Micro_Research, 28 Mar. 2013, www.1010.co.uk/org/
biologic.html. Accessed on 12 Feb. 2018.

Madshobye. “Singing Plant. Make Your Plant Sing With

Arduino, Touche and A Gameduino.” Instructables, 4 Oct.
2012, www.instructables.com/id/Singing-plant-Make-your-
plant-sing-with-Arduino-/. Accessed on 5 Feb. 2018.

Patel, Abhilash. “DIY: Arduino Based Continuous Touch

Piano.” Instructables, 11 Feb. 2017, www.instructables.
com/id/DIY-Arduino-Based-Continuous-Touch-Piano/.
Accessed on 11 Feb. 2018.

Straaten, Bas. “Ardunio Touch Capacitive MIDI Keyboard.”

Instructables, 9 Feb. 2014, www.instructables.com/id/
Arduino-Touch-capacitive-Midi-keyboard/. Accessed on 11
Feb 2018.

Straaten, Bas. “How to Make a Cheap Arduino MIDI

Controller.” Instructables, 7 Dec. 2016, www.instructables.
com/id/How-to-Make-a-Cheap-Arduino-MIDI-Controller/.
Accessed on 11 Feb. 2018.