Towards the Concept of
“Digital Dance and Music Instrument”
João Tragtenberg
ISI-TICs
Rua Frei Cassimiro, 88
Recife, Brazil
joao.tragtenberg@pe.senai.br
Giordano Cabral
CIn-UFPE
Av. Jornalista Anibal
Fernandes, s/n
Recife, Brazil
grec@cin.ufpe.br
ABSTRACT
This paper discusses the creation of instruments in which
music is intentionally generated by dance. We introduce
the conceptual framework of Digital Dance and Music In-
struments (DDMI). Several DDMI have already been cre-
ated, but they have been developed isolatedly, and there is
still a lack of common process of ideation and development.
Knowledge about Digital Musical Instruments (DMIs) and
Interactive Dance Systems (IDSs) can contribute to the de-
sign of DDMI, but the former brings few contributions to
the body’s expressiveness and the latter brings few refer-
ences to an instrumental relationship with music. Because
of those different premises, the integration between both
paradigms can be an arduous task for the designer of DDMI.
The concept of DDMI can also serve as a bridge between
DMIs and IDSs, serving as a lingua franca between both
communities and facilitating the exchange of knowledge. Fi-
nally, we describe four new DDMI we created during this
research. The conceptual framework has shown to be a
promising analytical tool for the design, development and
evaluation of new dance and music instrument.
Author Keywords
digital musical instrument, interactive dance systems
CCS Concepts
•Applied computing → Sound and music comput-
ing; Performing arts; •Human-centered computing →
Interaction devices;
1. INTRODUCTION
A great advantage of digital technology for artistic expres-
sion is related to the unprecedented freedom of integration
between diverse expressive modalities. With interfaces that
allow the conversion of human movement into information,
Licensed under a Creative Commons Attribution
4.0 International License (CC BY 4.0). Copyright
remains with the author(s).
NIME’19, June 3-6, 2019, Federal University of Rio Grande do Sul,
Porto Alegre, Brazil.
Filipe Calegario
ISI-TICs
Rua Frei Cassimiro, 88
Recife, Brazil
filipe.calegario@pe.senai.br
Geber Ramalho
CIn-UFPE
Av. Jornalista Anibal
Fernandes, s/n
Recife, Brazil
glr@cin.ufpe.br
it is possible that a wide variety of gesture can control any
sound, visuals, light or robotic media [6].
The field of artistic creation with digital technology is at
least 56 years old [27] and presented considerable advances
[19]. The area of Digital Musical Instruments (DMI) has
been well delimited for a long time[29]. Unlike acoustic in-
struments, in which the same energy of the gestural control
is responsible for the sound production, the DMI have open
possibilities, since the production of sound is independent
from gesture control, being only connected by digital means.
This freedom has allowed a much wider range of gestures to
control sound production. For instance, there are DMI that
are controlled by the positioning of several people in space
[13], by eye movement [39], by muscle tension, [20] and by
brain signals [12] just to cite a few.
The area of Interactive Dance Systems (IDS) is another
that puts digital technology in service to artistic expres-
sion. In academic literature, there hasn’t been such a clear
definition of the area, referred by different terms that vary
in broadness like: Interactive Dance/Music Systems [6], In-
teractive Music/Dance/Video Systems [8], or Multimodal
Interactive System (MIS) [9]. Other terms have been used,
such as Interfaces for Dance Performances [23], Interfaces
for Dancers [33], Sensor System for Interactive Dance [3] or
Multisensory Integrated Expressive Environments [7]. Even
though there isn’t a consensus on terminology, we will con-
sider all these as parts of the area of IDS, which are digital
systems used in interactive dance performances developed
to enhance the expressive possibilities of dancers with sen-
sors that capture their movements and produce sounds, vi-
suals or movement through robotic actuators.
The creation of digital instruments for symbiotic expres-
sion of music and dance have been tackled by several re-
searchers, but rarely refer one another, share conceptual
frameworks, or join efforts in tackling similar challenges of
a broader area. To contribute in this direction, we propose a
conceptual framework for Digital Instruments of Dance and
Music (DDMI). The DDMI stand out from the DMI for
the explicit concern about body expressiveness and present
different characteristics from the majority of IDS given the
instrumental relation with the musical production.
2. INSTRUMENTALITY
The concept of what makes an object a musical instrument
has been brightly tackled by Sarah-Indriyati Hardjowirogo:
 “[...] instrumentality, or simply being a musical
instrument must not be understood as a prop-
erty an object as such has or has not. Rather, it
seems to result from using something in a par-
ticular way which we think of as instrumental.
Consequently, an object is not per se a musi-
cal instrument (ontological definition) but it be-
comes a musical instrument by using it as such
(utilitarian definition) [...] the term must not be
understood as denoting a property an object per
se has or has not, but it is rather intended as a
means of capturing the instrumental potential of
a given artefact.” [16]
The “degree of instrumentality” is a dynamic quality of an
object, the result of cultural negotiation [41]. This degree
depends on a series of factors that characterize the possibili-
ties of human interaction with the artifact. The author lists
a number of important criteria for the instrumental poten-
tial of an artifact: sound production, Intention / Purpose,
Learnability / Virtuosity, Playability / Control / Immedi-
acy / Agency/ Interaction, Expressivity / Effort / Corpore-
ality, “Immaterial Features” / Cultural Embeddedness and
Audience Perception / Liveness.
What makes a musical instrument not as binary true or
false choice, but a continuous range that depends on the
player’s relationship to the object. This definition helps the
instrument designer to look at the user experience and to
target the intended criteria to improve the objects instru-
mentality.
3. ARTS INTEGRATION
Our main motivation to study the creation of DDMI is the
demand of artists who are at the intersection between mu-
sic and dance, interested in expressing themselves in both
languages at the same time.
The integration of the arts is an ancient quest in the hege-
monic western history. The Florentine humanists of the sev-
enteenth century introduced the concept of opera (opere),
referencing the theatrical traditions of ancient Greece which
combined music, dance and poetry. Wagner, in the nine-
teenth century, brought the concept of unification of the
arts (Gesamtkunstwerk) [26].
There are countless artistic traditions in which dance and
music performance are inseparable. In the Brazilian tra-
ditions of samba, forró, frevo, vanerão, maracatu, cavalo
marinho, dance and music play equally important roles. In
several artistic contexts, the practice of each artist is of
both languages at the same time. Some other examples are
the various tap dance traditions (such as Fandango, Coco
de Arcoverde, American tap dancing, Irish step dancing).
The flamenco dance cannot be done without the rhythmi-
cal handclapping, foot stomps and castanets playing. Other
examples are the traditions of the Portuguese Pauliteiros [1],
the European fiddling and dancing traditions [36], or even
Rock ’n’ Roll guitar performance.
4. DANCE AND MUSIC
Some empirical studies have been conducted to prove the
importance of body expression for musical perception [4,
10, 14, 22, 40, 42, 44]. These experiments have proved how
music is not restricted to sounds and also includes visual
and bodily aspects. They also reveal the importance of
the musician’s body movement to the audience’s musical
perception.
The movements of modern and contemporary dance marked
a rupture with previous traditions, where the dance was
subordinated to musical compositions previously made. Martha
Graham, one of the founders of modern dance, was a chore-
ographer who marked this break by interfering in the pro-
cess of musical composition, commissioning or suggesting
changes in the music pieces from her choreography [26].
This detachment process was intensified after the inter-
action between the choreographer Merce Cunningham and
the composer John Cage, where the choreography was con-
ceived independently of the sound composition. Each had a
different narrative, but they were presented together. The
audience had the freedom to make their connections be-
tween dance and music and could switch attention between
one and the other [26]. In Cage’s words:
“[...] in working with Merce, the first thing we
did was to liberate the music from the necessity
to go with the dance, and to free the dance from
having to interpret the music.” John Cage [21].
This rupture was important to consolidate dance as a lan-
guage independent of music, that is, without a hierarchical
distinction. Being able to look at both equally opens pos-
sibilities to think about new possible relationships between
them.
5. CONCEPTUAL FRAMEWORK FOR DDMI
The instrumental control of music by the dancer is not usu-
ally taken into account by the IDS researchers. On the other
hand, musicians’ body expressiveness is not frequently con-
sidered as a relevant asset for the DMI literature. This is
a natural consequence from the musician’s focus on sound
production, and dancer’s main concern in body movements.
These bias, nevertheless, can generate much confusion to
the DDMI designer, hindering its process of ideation and
development of instruments for corporal and musical ex-
pression. For that, the conceptual framework is consisted
of:
• The set of concepts that structure the process of re-
search and development of an artefact for the commu-
nity;
• The categories of artifacts considered relevant;
In this paper, we present a conceptual framework tai-
lored specifically to the DDMI designer with a classification
system relevant to the idiosyncrasies of these instruments
and a set of structuring concepts that take into consider-
ation important aspects for the architecture of a DDMI.
This framework was based on elements of the DMI and IDS
literature and also on research about IDS or DMI that we
consider to be also DDMI. It is important to point out that,
even though we are focusing in this paper on instruments
designed to be danced and played, an object that wasn’t
made to be used in such a way can be considered a DDMI
depending on the user’s relation to it. A guitar on the hands
and body of Chuck Berry, for example, is a dance and music
instrument, since he uses it intentionally making music and
dancing with it.
Here is a list of some examples of DDMI that meet the
main focus of this research:
• Very Nervous System - being one of the first IDS
developed1 , it used cameras in a very simple and ex-
pressive way, being an important reference for the cre-
ation of interactive installations with an instrumental
interaction [45].
1
A DDMI can also be an IDS
 • Karlax2 - this instrument looks like a clarinet, but
the absence of a nozzle and several other controls allow
much greater freedom of movement [28, 32].
• Miburi - launched by Yamaha Experimental Division
in 1994 in Japan, the system consists of a blouse with
six bending sensors in the arm joints, two hand con-
trollers (each with ten pressure-sensitive buttons) and
continuous controllers for the toes. [25]
• Expressive Footwear - it is a gestural interface in
the form of a shoe with 12 sensors that are intended
to capture the majority of gestural possibilities with
the feet. [35]
• Prosthetic Instruments - they are a series of phys-
ical artifacts used by dancers that can be attached to
the body or freely manipulated, alternating their con-
ceptual model of instrument as body to instrument as
object. [18]
Figure 1: Representation of the inheritance of
DDMI framework from DMI and IDS
This conceptual framework is intended to help on the
ideation and development phases of new DDMI. The struc-
turing concepts can help the ideation process, defining the
central aspects the designer should define first. The classifi-
cation system can help on the development of these instru-
ments, helping to predict the main challenges and allowing
to systematise common solutions to a set of instruments
that share some aspects. This framework can also serve as
a bridge between the areas of DMI and IDS facilitating the
exchange of knowledge between these communities.
5.1 Structuring Concepts
For the development of DDMI, aspects related to musi-
cal production and body expression are important. The
paradigms of DMI and IDS can provide references, but the
former presents few contributions to the corporal expres-
siveness and the latter brings few inputs on an instrumen-
tal relation with the musical production. We, therefore,
propose these structuring concepts taking elements that we
considered relevant to the design of DDMI.
We propose the diagram in Figure 2 to describe the archi-
tecture of a DDMI. This kind of representation was inspired
by the representation of a DMI in [29] and the IDS dia-
gram in [6]. It is composed of a Gestural Perception Unit,
a Gestural Processing Unit, a Mapping Unit and a Sound
Production Unit. The interface consists of sensors that per-
ceive information from the body. The gestural processing
2
http://www.karlax.com/
Figure 2: DDMI’s Architecture
unit is responsible for both the processing and interpreta-
tion of raw sensor data at different levels of abstraction.
The mapping unit must process these qualities for the sound
production unit in order to obtain interesting gesture-sound
relationships. This last stage is composed of sound synthe-
sizers and/or electromechanical actuators (loudspeakers or
robotic instruments).
Gestural feedback is responsible for communicating to the
artist that the gesture was perceived by the instrument and
can be given by passive interface elements (such as a click
of a button or the elasticity of the instrument’s structure)
as well as by active elements (such as actuators luminous,
mechanical or sound). The sound production feedback is
responsible for returning information to the user about the
sound produced by the instrument and consists mainly of
the sound output of the instrument, but also can have anal-
ogous productions to this sound with projections, light in-
dicators (such as VU bars) or mechanical (such as motors
that vibrate at the frequency of sound).
In this diagram, the concern of the feeling of control for
the indication of the feedback (characteristic of the instru-
mental approach) is emphasized. The gestural processing
unit is a key piece of a DDMI, suggesting that they should
be concerned with the expressiveness of the gestures.
The representation of the mapping through several con-
nected blocks indicates a different approach to the DMI
representation by arrows. Our approach suggests some-
thing closer to the IDS programming environments where
in addition to mapping ”raw” sensor data, it is possible to
”cook” this data through different processes. Expressive
mappings between gesture and sound are not always those
where sound parameters are directly related to the qualities
of movement. Another reference for this mapping proposal
is the systematization of Filipe Calegario to decompose the
mapping unit of a DMI into several levels: raw, cooked,
adapted, and combined [5].
The proposed mapping architecture for DDMI is directly
associated with programming environments with the patch-
chords metaphor of both IDSs (Isadora and EyesWeb) and
sound synthesis (Max / MSP, Pure Data). Following the
example of IDS, however, one has to worry about easy access
to programming, looking for a low floor and a high ceiling.
The more this language is adapted to the specific abilities
and purposes of the potential community, the greater the
adoption of new creators for this paradigm.
5.2 Categories / Classification System
We propose a categorization of DDMI according to 3 criteria
that are the most sensitive to common challenges and so-
lutions. We proposed the following categories: attachment,
tactile feedback and interaction metaphor. It is worth not-
ing that this classification does not have an encyclopedic
purpose to list all types of instruments that already exist or
may exist. We sought with this categorization an approach
aimed at inspiring the creation of new DDMI, in a man-
ner analogous to the categories of the movement analysis
of Rudolf Laban, who sought to contribute to the creative
process of dancers.
Figure 3: DDMI’s Categories
5.2.1 Attachment
This category is related to the movement freedom the in-
strument affords. If the instrument is fixed only to a limb it
provides greater freedom than if it is fixed on a table. This
class was inherited by IDS classification between inside-in,
inside-out, or outside-in systems [30]. We believe, however,
that the main structural difference relates to the place where
the instrument is attached and not to the nature of the phe-
nomenon measured. We are therefore interested only in the
first term of this classification (outside or inside).
It assumes four possible values:
• attached to an object;
• attached to the body (inside);
• attached to the environment (outside);
• not attached (loose);
Instruments that are loose usually need to be held with
the hands or supported on other parts of the body. This
can prevent a number of movements but generally can en-
compass the choice to easily release it from the body. An
instrument attached to the environment can bring greater
flexibility of movement, but usually restrains the region of
interaction. If the instrument is wired to a tabletop de-
vice it is also considered attached to the environment. The
wearable instruments, which are attached to the body, can
generally be used in a larger region of the space and it is
usually of interest that they are small. There is also the
possibility of the instrument attached to another object, in-
creasing its expressive capacities.
Some examples of DDMI attached to the body are the
MidiDancer [11], the Biomuse [20, 38] or The Hands [43].
Sensory cameras and carpets, such as Magic Carpet [34],
are examples of sensors attached to the environment. And,
examples of instruments attached to objects are accelerom-
eters attached to guitars to change their sound with their
inclination [31]. Examples of loose instruments are Karlax
and T-Stick [24].
5.2.2 Tactile Feedback
The lack of tactile feedback isn’t an issue for IDS, since the
cenesthetic/proprioceptive feedback is usually sufficient to
a dancer’s own-body perception[37]. This category is more
important to the musical instrumentality of the DDMI, since
an instrument you can touch affords greater precision and
speed of control. The turning of a knob, pressing of a string
or key can allow a greater degree of instrumentality to a
DDMI.
Since both the dance and musical expressions are impor-
tant, it can be a designer’s choice to decrease the precision
of control in favour of a greater freedom to move, so it is not
a category that the more tactile interaction with the object
the better, but a set of choices that the designer can make
to a new DDMI. The classes of tactile interaction are:
• tactile interaction with an object;
• tactile interaction with the body;
• tactile interaction with the environment;
• no tactile interaction (free gesture);
The second and third are possibilities that can give pre-
cise tactile feedback without obstructing movement, allow-
ing the body to interact with itself (i.e. body percussion)
or with the environment (i.e. foot-stomping). These in-
teractions bring together the best of both worlds, allowing
precise musical control without obstruction of movement.
This criterion corresponds to the mechanical feedback of
the instrument, mainly to the tactile component of passive
gestural feedback. Mechanical actuators can also contribute
to give a tactile feel of active feedback and sound production
[17]. One example of instruments without tactile interac-
tion is the Biomuse. Instruments of tactile interaction with
objects are those that most resemble musical instruments
that are usually manipulated like Weisvisz’s The Hands, or
Da Fact’s Karlax.
Instruments whose tactile feedback comes from the body
itself are body percussion instruments such as the instru-
ments used by Laurie Anderson in the 1986 ”Home of the
Brave” show in which percussion in the body itself triggered
digital samples. Another example of tactile feedback with
the body itself is the bio-acoustic interfaces that perceive
the acoustic propagation in the bones of subtle percussion
gestures with the fingertips [2].
The tactile interaction with the environment corresponds
to gestures of interaction with the ground, with walls or
with very large objects that are not easily displaceable. Ex-
amples are the ”Magic Carpet” by Joe Paradiso, which per-
ceives the position and pressure of the feet on the floor [34].
Another example is the ”Expressive Footwear” [35], which
controls the musical production with different gestures of
the interaction of the foot with the ground.
5.2.3 Interaction Metaphor
This category is subjective and has to do with the inten-
tionality in the instrument’s gestural interaction. The in-
teraction metaphors that the user has when playing an in-
strument are derived from the “instrument as object” and
“instrument as body” categories presented by Malloch when
describing the “Prosthetic Instruments” [23]. We added
the “instrument as an environment” class to deal with in-
teractive environments that have an instrumental relation-
ship. We consider the following categories of interaction
metaphors:
• instrument as object;
• instrument as body;
• instrument as environment;
The “instrument as object” corresponds to gestures which
evoke the interaction with an artifact or physically do so.
These gestures can refer directly to the repertoire of acous-
tic instruments, such as air gestures [15], or even to the
interaction with everyday objects. In the case of a tactile
interaction with the instrument, the interaction metaphor
will necessarily be an instrument as an object. This interac-
tion metaphor allows precise and subtle gestures of control,
usually with the hands, but can be done with any part of
the body.
The “instrument as body” metaphor refers to an instru-
mental relationship where it appears that the body itself
is producing sound. In the great majority of dance tradi-
tions, the body is the main instrument of expression, and
there are several repertoires of movement associated with
styles and categorized according to various qualities, such
as Laban/Bartenieff Movement Analysis’ movement quali-
ties. Many IDS have this metaphor of interaction and its
interfaces of gestural input seek to perceive the dancer’s
movement as if there were no interfaces.
The “instrument as environment” metaphor refers to an
interaction with the space around the player. The boundary
between what is an object and what constitutes an environ-
ment is difficult to define, but it is associated to its immo-
bility. The floor, a wall, the ceiling or a tree are clearly per-
ceived as environment, while a chair, a table or a computer
are usually considered objects. The instrument metaphor as
environment is related to the restriction of the interaction
space or to the interaction by parameters related to space.
6. CONCLUSION
The DDMI conceptual framework, composed of structuring
concepts and a system of categorization, is a proposition
to help the design of new instruments to make music and
dance. It aims to include in the DMI creation the body
expressivity and in the IDS area the concept of musical in-
strumentality. This is an open proposal to be in constant
evolution by the community interested in the development
of (DDMI).
7. REFERENCES
[1] A. P. Acitores. La gestualidad en los pauliteiros de
Miranda do Douro: Usos, funciones y tipolog{\’\i}as.
Trans. Revista Transcultural de Música, (19):1–14,
2015.
[2] B. Amento, W. Hill, and L. Terveen. The Sound of
One Hand: A Wrist-mounted Bio-acoustic Fingertip
Gesture Interface. CHI ’02 extended abstracts on
Human factors in computing systems - CHI ’02, page
724, 2002.
[3] R. Aylward and J. A. Paradiso. Sensemble: A
Wireless, Compact, Multi-User Sensor System for
Interactive Dance. In N. Schnell and F. Bevilacqua,
editors, Proceedings of the 2006 International
Conference on New Interfaces for Musical Expression
(NIME-06), pages 134–139, Paris, France, 2006.
[4] M. Broughton and C. Stevens. Music, movement and
marimba: an investigation of the role of movement
and gesture in communicating musical expression to
an audience. Psychology of Music, 37(2):137–153,
2009.
[5] F. Calegario. Probatio: Towards Different Levels of
Mapping. Technical report, INRIA Lille, Lille, França,
2016.
[6] A. Camurri. Interactive Dance/Music Systems.
Computer Music Conference ICMC, pages 45–252,
1995.
[7] A. Camurri, G. De Poli, A. Friberg, M. Leman, and
G. Volpe. The MEGA Project: Analysis and
Synthesis of Multisensory Expressive Gesture in
Performing Art Applications. Journal of New Music
Research, 34(1):5–21, mar 2005.
[8] A. Camurri, G. D. Poli, M. Leman, and G. Volpe. A
Multi-layered Conceptual Framework for Expressive
Gesture Applications. In Proceedings of MOSART:
Workshop on Current Directions in Computer Music,
(1):29–34, 2001.
[9] A. Camurri and G. Volpe. Multimodal Analysis of
Expressive Gesture in Music Performance. In S. J.
and N. K., editors, Musical Robots and Interactive
Multimodal Systems. Springer Tracts in Advanced
Robotics, chapter 4, pages 47–66. Springer, Berlin,
Heidelberg, 2011.
[10] J. W. Davidson. Visual Perception of Performance
Manner in the Movements of Solo Musicians.
Psychology of Music, 21(2):103–113, 1993.
[11] S. DeLahunta. ISADORA ”almost out of beta” tracing
the development of a new software tool for artists,
2002.
[12] J. Eaton, W. Jin, and E. Miranda. The Space
Between Us. A Live Performance with Musical Score
Generated via Emotional Levels Measured in EEG of
One Performer and an Audience Member. Proceedings
of the International Conference on New Interfaces for
Musical Expression, pages 593–596, 2014.
[13] C. Frisson, S. Dupont, J. Leroy, A. Moinet, T. Ravet,
X. Siebert, and T. Dutoit. LoopJam : turning the
dance floor into a collaborative instrumental map.
NIME 2012 Proceedings of the International
Conference on New Interfaces for Musical Expression,
pages 278–281, 2012.
[14] D. Glowinski, N. Baron, D. Grandjean, T. Ott,
K. Shirole, K. Torres-Eliard, and M.-A. Rappaz.
Analyzing expressive styles and functions of bodily
movement in violinist performance. Proceedings of the
2014 International Workshop on Movement and
Computing - MOCO ’14, pages 154–155, 2014.
[15] R. Godøy, E. Haga, and A. Jensenius. Exploring
musicrelated gestures by sound-tracing-a preliminary
study. In 2nd International Symposium on Gesture,
number April 2016, 2006.
[16] S.-I. Hardjowirogo. Instrumentality. On the
Construction of Instrumental Identity. In
T. Bovermann, A. de Campo, H. Egermann, S.-I.
Hardjowirogo, and S. Weinzierl, editors, Musical
Instruments in the 21st Century Identities,
Configurations, Practices, chapter Instrument, pages
9–24. Springer, Berlin, New York, 2017.
[17] I. Hattwick, I. Franco, and M. M. Wanderley. The
Vibropixels: A Scalable Wireless Tactile Display
System. In S. Yamamoto, editor, HIMI 2017: Human
Interface and the Management of Information:
Information, Knowledge and Interaction Design,
pages 517–528. Springer, New York, 2017.
[18] I. Hattwick, J. Malloch, and M. Wanderley. Forming
Shapes to Bodies: Design for Manufacturing in the
Prosthetic Instruments. Proceedings of the
International Conference on New Interfaces for
Musical Expression, (August 2016):443–448, 2014.
[19] A. R. Jensenius and M. J. e. Lyons. A NIME Reader,
volume 3. Springer, 2017.
[20] B. Knapp and H. Lusted. A Bioelectrical Controller
for Computer Music Applications. Computer Music
Journal, 14(1):42–47, 1990.
[21] R. Kostelanetz. Conversing with Cage. Limelight,
New York, 1988.
[22] C. Krahe, U. Hahn, and K. Whitney. Is seeing
(musical) believing? The eye versus the ear in
emotional responses to music. Psychology of Music,
10(8), oct 2013.
[23] J. Malloch. A Framework and Tools for Mapping of Cross-modal interactions in the perception of
Digital Musical Instruments. PhD thesis, McGill emotions in musical performance. Cognition,
University, 2013. 118(2):157–170, 2011.
[24] J. Malloch and M. M. Wanderley. The T-Stick : From [41] F. Visi. Methods and Technologies for the Analysis
Musical Interface to Musical Instrument. In and Interactive Use of Body Movements in
International Conference on New Interfaces for Instrumental Music Performance. PhD thesis,
Musical Expression, pages 66–69, New York, USA, Plymouth University, 2017.
2007. [42] J. K. Vuoskoski, M. R. Thompson, E. F. Clarke, and
[25] T. Marrin. Toward an Understanding of Musical C. Spence. Crossmodal interactions in the perception
Gesture: Mapping Expressive Intention with the of expressivity in musical performance. Attention,
Digital Baton. PhD thesis, MIT, 1996. Perception, & Psychophysics, 76(2):591–604, 2014.
[26] P. H. Mason. Music, dance and the total art work: [43] M. Waiswisz. The Hands, a Set of Remote
choreomusicology in theory and practice. Research in MIDI-Controllers. Proc. of the 1985 International
Dance Education, 13(1):5–24, 2012. Computer Music Conference., pages 313–318, 1985.
[27] M. V. Mathews. The Digital Computer as a Musical [44] A. Williamon and G. Waddell. Eye of the Beholder:
Instrument. Science, 142(3592):553–557, 1963. Stage Entrance Behavior and Facial Expression Affect
[28] T. Mays and F. Faber. A Notation System for the Continuous Quality Ratings in Music Performance.
Karlax Controller. In B. Caramiaux, K. Tahiroglu, Frontiers in Psychology, 8(April):1–14, 2017.
R. Fiebrink, and A. Tanaka, editors, Proceedings of [45] T. Winkler. Creating Interactive Dance with the Very
the International Conference on New Interfaces for Nervous System. Proceedings of Connecticut College
Musical Expression, pages 553–556, London, United Symposium on Arts and Technology, (April), 1997.
Kingdom, jun 2014. Goldsmiths, University of
London.
[29] E. R. Miranda and M. M. Wanderley. New Digital
Musical Instruments: Control and Interaction Beyond
the Keyboard. A-R Editions, Middleton, 2006.
[30] A. Mulder. Human Movement Tracking Technology.
In Report 94-1 of the Hand Centered Studies of
Human Movement Project., Burnaby, British
Columbia, Canada, 1994. Simon Fraser University.
[31] D. Newton and M. T. Marshall. Examining How
Musicians Create Augmented Musical Instruments. In
A. R. Jensenius, A. Tveit, R. I. Godoy, and
D. Overholt, editors, Proceedings of the International
Conference on New Interfaces for Musical Expression,
pages 155–160, Oslo, Norway, 2011.
[32] G. Paine. Designing the techno-somatic. Proceedings
of the 2nd International Workshop on Movement and
Computing - MOCO ’15, pages 48–51, 2015.
[33] J. Paradiso and E. Hu. Expressive footwear for
computer-augmented dance performance. Digest of
Papers. First International Symposium on Wearable
Computers, (October):20–21, 1997.
[34] J. a. Paradiso and N. Gershenfeld. Musical
applications of electric field sensing. Computer Music
Journal, 21(2):69–89, 1997.
[35] J. a. Paradiso, K. Hsiao, a. Y. Benbasat, and
Z. Teegarden. Design and implementation of
expressive footwear. IBM Systems Journal,
39(3.4):511–529, 2000.
[36] C. Quigley. Locating the Choreomusical: The Case of
European and American Dance Fiddling. Český Lid,
103(4):515–536, 2016.
[37] W. Siegel. Dancing the Music: Interactive Dance and
Music. In R. T. Dean, editor, The Oxford Handbook of
Computer Music, chapter 10, pages 191 – 213. Oxford
University Press, Oxford, 2009.
[38] A. Tanaka. Musical Technical Issues in Using
Interactive Instrument Technology With Application
to the BioMuse. In International Computer Music
Conference, pages 124–126, 1993.
[39] Z. Vamvakousis and R. Ramirez. The EyeHarp: A
gaze-controlled digital musical instrument. Frontiers
in Psychology, 7(JUN):1–14, 2016.
[40] B. W. Vines, C. L. Krumhansl, M. M. Wanderley,
I. M. Dalca, and D. J. Levitin. Music to my eyes: