Exploring Musical Textures By Computational

Creativity and Visual Art Interactivity: The

Pendulum’s Sway on Overlapping Sounds

First Author1[0000−1111−2222−3333] , Second Author2,3[1111−2222−3333−4444] ,

Third Author3[2222−−3333−4444−5555] , and Fourth Author4[2222−−3333−4444−5555]
1
Xxxx Xxxxx, Xxxx
2
Yyyy Yyyyyy xxxx@xxx.xxx
3
Zzzz zzzzz,
{zzz,zzzs}@zzz.zzz

Abstract. This paper explores a particular contribution to the field of

ubimus studies by exploring the synergies between music, computational
creativity, and visual art through a project called ‘Orbital Overlaps’.
Utilizing the P5.js JavaScript library, the project employs a double pen-
dulum model to create an interactive and acoustically engaging visual-
ization. The model is inspired by studies from Daniel Shiffman’s ‘Na-
ture of Code’ and allows users to manipulate various visual and acoustic
features such as angles, lengths, gravitational forces, and sound intensi-
ties through a GUI interface. Additionally, the current model integrates
the Tone.js library for sound synthesis techniques applications, offering
multiple features like Frequency Modulation (FM), Stereo Image, and
Feedback Delay. A pilot study involving university students indicates
the project’s potential impact on creative activities. The study employs
the CSI-NAP Performance Questionnaire to gauge the effectiveness of
the project in fostering activities, a key objective in ubimus research.
The study serves as a preliminary assessment for future developments in
this interdisciplinary field.

Keywords: Musical texture · Computer Music · Creative coding · Pen-

dulum music · P5.js.

1 Introduction

We explore the connections between novel music creation strategies and a visual
pendulum mediated by creative coding. Our study features the use of pendulum
music visual renderings, coded within the P5.js framework. Section 2 delineates
the related works. In Section 3 we outline the process of building the protoype.
The following Section 4 documents its deployment on a web browser. Section 5
delves into a discussion of its creative potential leading to the conclusion, which
draws together the threads of the discussion and proposes future developments.
2 F. Author et al.

2 Related Works
Pendulum Music The impact of rhythm and imagery of oscillatory motion was
investigated in various studies, such as [10]. Pendulum-movement simulations
have been applied in the analysis and realization of works such such as Schubert’s
Piano Sonatas [6], poeme symphonique (1962) for one hundred metronomes by
Ligeti [3][4] and Steve Reich’s Pendulum Music [11]. ‘Pendulum Music’ features
swinging microphones that generate weakly phased feedback pulses [12]. The
pendulum metaphor has been used by several artists and researchers within the
context of creative programming [7][1]. Coco recreated Reich’s experiment using
the visual language Pure Data [2]. Østergaard developed the artistic project
Pendulum Dialogues [9].

3 Methods
Computer Environment Architecture. ‘Orbital Overlaps’ makes use of the
p5.js JavaScript library. P5.js puts an emphasis on accessibility and diversity
and may be used from within a browser, eliminating the requirement for instal-
lation4 . P5.js promotes interactive and immersive experiences by emphasizing
open-source principles and increased sketching skills, giving artists greater ver-
satility in computer-generated art. The browser is employed as a canvas, giving
programmers and artists a wide range of options for computer-generated mate-
rials.

The Very First Pendulum Sketch Version: ‘Orbital Overlaps’ ‘Orbital

Overlaps’ was derived from Daniel Shiffman’s Nature of Code’ book (Chapter
3.9), which focuses on Trigonometry and Forces [13]. Shiffman’s book presents
Java Processing code examples. The similarities between Java and JavaScript
facilitate porting the examples to P5.js sketches.
A pendulum consists of a bob suspended from a pivot linked by an arm,
influenced by gravitational forces. The pendulum bob doesn’t fall due to its
connection to a pivot point, its behavior is based on on both the gravitational
forces and the angle of the pendulum’s arm.
An update from our current version is to adapt this to a double pendulum
context, which can be presented as one pendulum attached to another. Subse-
quent iteration of our model will integrate the double pendulum: one pendulum
attached to another one. As noted by Shiffman, while such an adaptation might
yield captivating outcomes, it risks significant physical inaccuracies [13, pp.
133]. In our model, the first pendulum’s arm is attached to a fixed point, and
the second pendulum’s arm is attached to the end of the first arm. Each arm
has a bob (or mass) at its end, featuring the following properties:

– origin: The point of attachment for the arm.

4
https://editor.p5js.org/. Date of access: 08/08/2023
 Musical Textures, Creativity, and Visual Art: The Pendulum’s Sway 3

– position: The end point of the arm or where the bob is located.
– length: The length of the pendulum arm.
– angle: The angle of the pendulum arm with respect to the vertical.
– aVelocity: Angular velocity of the pendulum arm.
– aAcceleration: Angular acceleration of the pendulum arm.
– mass: Mass of the bob at the end of the pendulum arm.
– gravity: Gravitational force.

Given some cornerstone elements for different motion results, our simulation
considers the angle and arm length as the variables, determining motion using
angular velocity and acceleration. The force exerted by the pendulum Fp is
perpendicular to its arm, with gravity Fg acting downwards. By constructing a
right triangle, we deduce that:

F p = F g × sin(θ) (1)
A central concern is deriving the angular acceleration of the pendulum. Ac-
cording to Newton’s laws and trigonometry, the angular acceleration is:

gravity × sin(θ) (2)

However, the pendulum behavior also depends on its arm length l, leading
to the following:

−1 × G × sin(angle)
aAcceleration = (3)
l
For enhanced realism, damping is introduced, decreasing the angular velocity
by 1% each cycle, represented by:

aV elocity× = 0.99 (4)

Our model uses the following structure for the angular motion:

let force = (this.gravity / this.length) * sin(this.angle);

this.aAcceleration = -force;
this.aVelocity += this.aAcceleration;
this.aVelocity *= 0.99;
this.angle += this.aVelocity;

Subsequently, the initial position, length, and orientation of the arm were
configured. These parameters are then updated with each successive frame re-
fresh, culminating in the simulation of pendulum movement. The following script
depicts the structure of the arm motion:

this.position.set(
this.origin.x + this.length * sin(this.angle),
this.origin.y + this.length * cos(this.angle)
);
4 F. Author et al.

In the current model, each arm is delineated as a line emanating from its
origin to its specified position. Simultaneously, the bob, characterized by its
inherent mass which correlates with the dimensions of the ellipse, width and
height, is illustrated as an ellipse positioned at the terminal end of the arm as
depicted in the following:

ellipse(this.position.x, this.position.y, this.mass, this.mass);

To facilitate the nesting required for the double pendulum, a class titled ‘Pen-
dulum’ was developed. The initial position of ‘arm2’ is conjoined to the terminal
position of ‘arm1’, visually superimposing the bob. In the current model, the
bob’s ‘mass’ attribute does not influence motion, but is exclusively utilized for
sound processing, as delineated in Section 3. The very first ‘arm1’ is initially
positioned at the center of the screen, e.g., width/2 and height/2. Subsequently,
we present the parameters incorporated in the pendulum constructor, where col1
and col2 indexes are related to the color definition for each bob:

{
this.arm1 = new Arm(x, y, length1, mass1, angle1, col1, gravity);
this.arm2 = new Arm(
this.arm1.position.x,
this.arm1.position.y,
length2,
mass2,
angle2,
col2,
gravity);
}

The following Figure 1 depicts the double pendulum at rest on the center
of the screen. In this first model, the background color is arbitrarily fixed to a
grayscale index of 92 where 0 is full black and 255 is full white. Arm colors were
also set initially to black.
Lastly, we created a function named addPendulum(), in which we can de-
fine the number of double pendulum instances at the same time on the screen.
For computational efficiency, the quantity of overlapping pendulums is capped
at five. From our empirical trials and computer resources it is enough to grant
enough results. Based on our preliminary empirical assessments and the con-
straints of available computational resources, this limitation suffices to yield
satisfactory outcomes.

pendulums[];
function addPendulum() {
if(pendulums.length >= maxPendulums) {
pendulums.shift();
}
}
 Musical Textures, Creativity, and Visual Art: The Pendulum’s Sway 5

Fig. 1. An exemplar representation of a double pendulum frame, positioned at equi-

librium, is centrally displayed on the screen.

Interactivity The parameters of the double pendulum, namely angles, lengths,

masses, and gravitational forces, are user-defined. The pendulum properties are
determined through a randomized instances ensuring diversity and offering var-
ied and contrasting configurations triggered by the ‘Add Pendulum’ functional-
ity.
‘Add Pendulum’ instantiates a new double pendulum (Section 3). There may
be a maximum of five concurrent pendulum instances. Upon reaching this thresh-
old, the system automatically eliminates the earliest instance.
GUI buttons trigger the pendulum actions (Figure 2). To accommodate
variations, specific functionalities were introduced. Zoom In and Zoom Out can
be used when the length exceeds the screen dimensions. Click and drag allow the
users to reposition the pendulum. A toggle button labeled ‘Show/Hide Controls’
conceals or reveals all GUI sliders. The ‘Reset Sketch’ button initializes the
interface, restoring all parameters to default settings.
Dynamic controllers make use of sliders 3. The angles, in ‘degrees’, span
a range of 0 − 360. An index of 0 or 360 degrees corresponds to the lowest
position on the screen, while an index of 180 degrees signifies the uppermost
point. Lengths are denoted in pixels and are commensurate with the size of the
screen. The masses for each bob are set to a range between 5 − 50. Gravity
is set within the range −10 and 10. A negative value reduces the weight of
the pendulum’s lower section, triggering an upward motion. Additionally, we
assembled a gradient of red and blue hues to diversify colors, spanning the range
1 − 255, features transitions from red to blue hues.

Sound Structure Architecture ‘Tone.js’5 . Tone.js is a browser-integrated

Web Audio framework oriented towards interactive music. It integrates conven-
tional DAW functionalities with components for specialized audio applications.
This includes capabilities like global transport for event synchronization and
pre-configured tools for tailored synthesizer and digital signal controllers. The
5
Further details can be found at: https://tonejs.github.io/. Accessed on: 08/10/2023
6 F. Author et al.

Fig. 2. Two double-pendulum settings. On the left, all dynamic controls for pendulum
features are visible. Conversely, on the right, once the ’Show/Hide Controls’ is acti-
vated, GUI sliders are concealed, and pendulum attributes are randomly assigned.

audio architecture was integrated into the ‘Pendulum’ class, encompassing arms
and bobs for the nested pendulums. Here is a list of audio features.

– Frequency Modulation Synthesis (FM): This is employed to augment the

character and diversity of the sound results generated by each double pen-
dulum instance.
– Stereo Imaging: This pertains to the stereo position of each pendulum.
– Feedback Delay: This enhances the audio possibilities.

Within the Tone.js library, there exists a built-in FM synthesis function de-
noted as ‘Tone.FMSynth’. This was incorporated for each bob within the double
pendulum instances, and in the extant model iteration, it adheres to the subse-
quent structure:

// Create the FMSynth objects

// for bob #1
this.synth1 = new Tone.FMSynth({
modulationIndex: 12,
harmonicity: 1,
modulation: { type: ’sine’ },
modulationEnvelope: {
attack: 0.25,
decay: 0.2,
sustain: 0.25,
release: 0.3},
envelope: {
attack: 0.25,
decay: 0.20,
sustain: 0.25,
release: 0.3}
}).toDestination());
 Musical Textures, Creativity, and Visual Art: The Pendulum’s Sway 7

// for bob #2
this.synth2 = new Tone.FMSynth({
modulationIndex: 12,
harmonicity: 1,
modulation: { type: ’sine’ },
modulationEnvelope: {
attack: 0.25,
decay: 0.2,
sustain: 0.25,
release: 0.3},
envelope: {
attack: 0.25,
decay: 0.2,
sustain: 0.25,
release: 0.3}
}).toDestination());

Regarding the parameters for FM synthesis, we designed an integration by

the bob size to delineate the carrier frequency. As stipulated arbitrarily, bob
sizes span a 5 − 50 pixel range, mapping sound frequencies to a 60Hz − 2000Hz
range.

// Map the mass values directly to the frequency ranges

let mappedFreq1 = map(this.arm1.mass, 5, 50, 60, 2000);
let mappedFreq2 = map(this.arm2.mass, 5, 50, 60, 2000);

The Frequency Modulation and the Modulation Index were based on the bob
colors. Red was employed for the frequency modulation and blue was used for the
modulation index. Frequency Modulation range is set between 1Hz − 1000Hz
and the Modulation Index is 1 − 10:

// Map the red color index to the frequency modulation

let modulationFrequency1 = map(r1Slider.value(), 1, 255, 1, 1000);
this.synth1.harmonicity.value = modulationFrequency1;
// Map the blue color index to the modulation index
let modulationDepth1 = map(b1Slider.value(), 1, 255, 1, 10);
this.synth1.modulationIndex.value = modulationDepth1;
// Map the red color index to the frequency modulation
let modulationFrequency2 = map(r2Slider.value(), 1, 255, 1, 1000);
this.synth2.modulationIndex.value = modulationIndex2;
// Map the blue color index to the modulation index
let modulationDepth2 = map(b2Slider.value(), 1, 255, 1, 10);
this.synth2.modulationIndex.value = modulationDepth2;

The pan positioning employed the ‘Tone.Panner’ function, which seamlessly

integrates with each arm’s position within the double pendulum. Panning was
mapped to the spatial location of each arm on display.
8 F. Author et al.

// Add properties for panPosition1 and panPosition2

this.panPosition1 = 0; // at the center of display (width/2. height/2)
this.panPosition2 = 0; // at the center of display (width/2. height/2)
// pan1
this.synth1 = new Tone.FMSynth({
... // presented above
}).connect(new Tone.Panner(this.panPosition1).toDestination());
// pan2
this.synth2 = new Tone.FMSynth({
... // presented above
}).connect(new Tone.Panner(this.panPosition2).toDestination());
// and then map the function to the double pendulum location
// Calculate the pan position based on the x-coordinate of the ellipse
let panPosition1 = map(this.arm1.position.x, 0, width, -1, 1);
let panPosition2 = map(this.arm2.position.x, 0, width, -1, 1);

Feedback Delay was the primary audio processing mechanism employed. Its
integration served not only to introduce a subtle depth but also to mitigate
abrupt sound transitions in synchrony.

/ Add the FeedbackDelay effect to the synthesizers

this.feedbackDelay1 = new Tone.FeedbackDelay({
delayTime: 0.15, // Adjust the delay time as per your preference
feedback: 0.4, // Adjust the feedback amount as per your preference
wet: 0.65 // Adjust the wet/dry balance as per your preference
}).toDestination();

this.feedbackDelay2 = new Tone.FeedbackDelay({

delayTime: 0.15, // Adjust the delay time as per your preference
feedback: 0.4, // Adjust the feedback amount as per your preference
wet: 0.65 // Adjust the wet/dry balance as per your preference
}).toDestination();

Then, we connect the feedback Delay function to the FM synth:

// Connect the synthesizers to the FeedbackDelay effect

this.synth1.connect(this.feedbackDelay1);
this.synth2.connect(this.feedbackDelay2);

4 Prototype Description

The Orbital Overlaps prototype provides an interactive and acoustically engag-

ing visualization based on the double-pendulum creative-action metaphor. The
‘Show/Hide Controls’ activates the double pendulum’s display as delineated in
Section 3. The pendulum’s movements and the associated red and blue color
variations are mapped to FM-synthesis parameters: frequency modulation and
 Musical Textures, Creativity, and Visual Art: The Pendulum’s Sway 9

modulation index. The GUI controllers provide a handle to the pendulum’s prop-
erties Section 3.
Two interface layouts were implemented: one with a visible dynamic control
panel and another without controllers. Figure 3 depicts an example of the dou-
ble pendulum interface. There is also the option of visual pendulum elements
without sonic output.

Fig. 3. A single frame from the double pendulum enactment in random mode.

A Tone.Panner function was used to map the stereo auditory location of the
soutrce to the bob spatial position. Tone.FMSynth provided support for flexible
audio results. The decision to use the bob’s size as the determining factor for the
carrier frequency produced a spectrum spread between 60Hz and 2000Hz. The
Feedback Delay added nuances to the sounds, and helped to smooth abrupt audio
transitions. The ‘Orbital Overlaps‘ prototype can be accessed in the following
link: https://rb.gy/i8e89.

4.1 Study
A pilot study involving sound creation and interaction with the ‘Orbital Over-
laps’ prototype was conducted. To gather volunteers’ assessments, we used the
CSI-NAP Performance Questionnaire v. 0.8, complemented by the ISE-NAP v.
1.0, which focuses on the Identification and Socio-Economic Profile of the re-
spondents 6 .
NAP’s Creative Support Index, a.k.a. CSI-NAP, is a tool designed to gather
objective information through subjective queries on the participants’ perfor-
mance during creative activities [8]. This instrument comprises various facets
of individual experiences and material resources [5]. The data is used for the
evaluation of the impact of the resources and the activity [5, pp. 10].
6
The discrete categories are derived from the following items: ‘the activity location’,
‘the amount of sound materials’, ’ease to select sounds’, ‘ease to modify the sounds’,
‘ease to mix’, ‘fun to select sounds’, ‘fun to modify sounds´, ‘ease to collaborate´,
‘quality of sonic result´, ‘originality of sonic result’, ‘quality of experience’, ‘original-
ity of experience’. Participants answer the instrument’s questions through a 5-points
Likert scale between -2 and 2. Each level features the following terms: ‘I fully dis-
agree’; ‘I partially disagree’; ‘I don’t know’; ‘I partially agree’ and; ‘I fully agree’.
10 F. Author et al.

Participants Five university students participated as volunteers. Four were

male (mean age 24.2 years, SD = ± 5.97). All of them were music students at
Universidade Federal do Acre. Most of the volunteers were hard-level mobile
devices and computer users (8.40 ± 2.27 years). Additionally, there were few of
them who had music studies formally (4.80 ± 4.21 years), thus, almost all were
considered as moderate ‘lay people’ 7 . Figure 4 summarises the profile of the
participants.

Descriptives
Descriptives
N Mean SD Minimum Maximum

Age 5 24.20 5.97 18 33

Mean age of music training 5 4.80 4.21 0.00 10.0
Mean age of computer and/or mobile device usage 5 8.40 2.27 4.50 10.0
Interaction time (minutes) 5 24.40 9.32 10 32

Fig. 4. Participant profile

Session Location Participants engaged with the prototype within their re-
spective domestic environments. Of the participants, one utilized a tablet for
interaction (20%), two opted for notebooks (40%), while the remaining two em-
ployed mobile devices for the study (40%).

Session Results The participants engaged with the model for 24.4 minutes
(SD = ± 9.32 minutes). Figure 5 presents the median values of all variables,
along with their respective minimum and maximum ratings. Notably, there were
no missing values in the study.
In the Following Figure 6, all the median categories are stacked in a single
barplot.

5 Discussion

Our creative-action metaphor ‘Orbital Overlaps’ stands at the intersection of

oscillatory motion, generative graphics and browser-based interaction. The heart
7
We apply the criterion of a minimum of two years of formal music training to consider
a participant as a music student. This classification is based on data provided by
the subjects in the ISE-NAP tool. It is possible to apply a more refined protocol to
determine the level of musical training. However, given the focus of the research on
the inclusion of lay people, the declaration of the subjects themselves is sufficient.
 Musical Textures, Creativity, and Visual Art: The Pendulum’s Sway 11

Descriptives
Descriptives
N Median Minimum Maximum

the activity location is good 5 1 -1 2

the sound materials are sufficient 5 1 -1 1
it's easy to select the sounds 5 1 1 2
it's easy to modify the sounds 5 1 1 2
it's easy to mix 5 1 0 1
it's fun to select the sounds 5 2 -1 2
it's fun to modify the sounds 5 2 -1 2
it's easy to collaborate 5 2 1 2
the sound result is good 5 2 -1 2
the sound result is original 5 1 0 2
the experience is good 5 2 0 2
the experience is original 5 2 1 2

Fig. 5. Descriptive Statistics

Fig. 6. Barplot from median values in each category

of ‘Orbital Overlaps’ is a double pendulum - a classic system in physics that may

display periodic or chaotic behavior, depending on the initial conditions. The
12 F. Author et al.

double pendulum’s motion produces both fairly predictable and unpredictable

patterns over time, opening chances to interact and experiment with its initial
settings. By rendering its motion both in the visual and auditory modalities, its
ludic and experiential potential is amplified.
Our initial findings from the pilot study presented in Section 4.1, align the
proposal with an expanding palette of ubimus creative-action metaphors. Given
its limited sample size, we classify these results as preliminary. The overall out-
come was positive. The design choices, especially those targeting visual feed-
back, enhanced user engagement. According to the participants’ assessments,
this creative-action metaphor promotes an immersive experience. This is re-
flected by the high scores given to factors such as ‘Fun’, ‘Collaboration’, and
‘Attention’. The attributes ‘Ease’ and ‘Productivity’ also yielded positive out-
comes.
Ubimus design aims to incorporate musical experiences into daily activities
and social interactions. The Orbital Overlaps prototype features functionalities
to support activities in everyday contexts: it is accessible from any internet-
enabled device (home computer, a smartphone while traveling, or a public dis-
play). The design is built upon a well-known phenomenon: the oscillating motion
of a double pendulum. Supporting multimodal interaction (even with material
phenomena that may be considered insignificant) carries the potential for more
inclusive designs, making it possible for people of varied musical and sociocul-
tural backgrounds to participate and engage in creative practices.

References
1. Bisig, D., Bogner, F.: Pendulum – exploiting simple physics for gen-
erative art. In: 20th Generative Art Conference GA2017 (2017),
https://www.generativeart.com/GA2017/DanielBisig full paper.pdf
2. Coco, R.: Minimalism and process music: a pure data realization of ”pendulum mu-
sic”. In: Proceedings of Sound and Music Computing 0́6. Marseille, France (2006),
http://smc.afim-asso.org/smc06/papers/24-Coco-pendulum.pdf
3. Cone, E.T.: One hundred metronomes. Australian Journal of Music Education (26),
19–24 (1980), https://search.informit.org/doi/10.3316/informit.227677257417966
4. Drott, E.: Ligeti in fluxus. Journal of Musicology 21(2), 201–240 (2004).
https://doi.org/10.1525/jm.2004.21.2.201
5. Freitas, B., Bessa, W., Costa, D., Nazaré, P., Keller, D.: A ecologia soundsphere
desde um viés culinário: Atividades criativas em um restaurante universitário. In:
Anais do VII Simpósio Internacional de Música na Amazônia (2019)
6. Guez, J.: Process and Pendulum in Schubert’s Expanded Type 1 Sonatas. Music
Analysis 40(2), 147–177 (2021). https://doi.org/10.1111/musa.12172
7. Hansen, A.M.S., Overholt, D., Burleson, W., Jensen, C.N.: Pendaphonics: A
tangible pendulum-based sonic interaction experience. In: Proceedings of the
3rd International Conference on Tangible and Embedded Interaction. pp. 153–
160. Association for Computing Machinery, New York, NY, USA (2009).
https://doi.org/10.1145/1517664.1517701
8. Keller, D., Capasso, A., Tinajero, M.P.: Knowledge transfer in ecologically
grounded approaches to ubimus: Inmesh 1.0. Journal of New Music Research 48(4),
397–411 (2019). https://doi.org/10.1080/09298215.2019.1642361
 Musical Textures, Creativity, and Visual Art: The Pendulum’s Sway 13

9. Østergaard, E.: Pendulum dialogues and the re-enchantment of the world. In: Pio,
F., Varkøy, Ø. (eds.) Philosophy of Music Education Challenged: Heideggerian
Inspirations: Music, Education And Personal Development, pp. 185–198. Springer
Netherlands, Dordrecht (2015). https://doi.org/10.1007/978-94-017-9319-3 11
10. Peckel, M., Pozzo, T., Bigand, E.: The impact of the perception of rhythmic
music on self-paced oscillatory movements. Frontiers in Psychology 5 (2014).
https://doi.org/10.3389/fpsyg.2014.01037
11. Reich, S.: Writings about music. In: Lexikon Schriften über Musik: Band 2:
Musikästhetik in Europa und Nordamerika, pp. 684–689. Bärenreiter-Verlag, Kas-
sel, Germany (2022). https://doi.org/10.1007/978-3-7618-7287-1 33
12. Schwarz, K.R.: Steve Reich: Music as a gradual process part II. Perspectives of
New Music 20(1/2), 225 (1981). https://doi.org/10.2307/942414
13. Shiffman, D.: The Nature of Code (2012), https://natureofcode.com/